Collaborative Rule Generation: An Ensemble Learning Approach

Significance Statement

This paper introduces a new approach of ensemble learning called Collaborative Rule Generation (CRG). The new approach involves multiple base algorithms learning from a single data set to generate a single rule set, which aims to enable each single rule to have a higher quality. In other words, the new approach of ensemble learning is designed to improve the quality of each single rule generated and thus to improve the overall classification accuracy through scaling up algorithms.

The proposed approach addresses weaknesses in current ensemble learning approaches, as outlined below.

Firstly, the collaborative rule generation approach only generates a single rule set and rule based models are highly interpretable. In contrast, some popular methods of ensemble learning, such as Bagging, Boosting and Random Forests, suffer from incomprehensibility of the predictions made by different rule sets and are thus poorly  interpretable. Therefore, the collaborative rule generation approach would fit better the purpose of knowledge discovery especially on interpretability.

Secondly, Bagging, Boosting and Random Forests all aim to improve accuracy for prediction through scaling down data. However, there is nothing done by scaling up algorithms for improving accuracy. It is necessary to deal with the issues on both algorithms and data sides in order to comprehensively improve the accuracy. Another ensemble learning approach, called Collaborative and Competitive Random Decision Rules (CCRDR), has been recently introduced by the authors in order to fill the gap. However, the authors argue that the CCRDR approach only enables each rule set (as a whole) to be of high quality on average, which indicates that there may be still some single rules of low quality. The authors conclude that the collaborative rule generation approach would be useful and effective to help the CCRDR approach fill the gap relating to the quality of each single rule and thus also complements the other three popular ensemble learning methods mentioned above.

This paper includes an experimental study validating the CRG approach and discusses the results in both quantitative and qualitative terms. In particular, the experimental study is set up to validate that the combination of different rule learning algorithms usually improves the overall accuracy and the quality of each single rule on average compared with the use of a single base algorithm. 20 data sets from the UCI repository were used for the validation.

We compare the CRG approach with other single base algorithms in terms of classification accuracy, and provide average metrics for the quality of each single rule. The results indicate that the CRG approach is useful for improving the quality of each single rule generated, thus improving the overall accuracy of classification.

 

collaborative rule generation (advances in engineering)

About The Author

Han Liu received a BSc in Computing from the University of Portsmouth in 2011, an MSc in Software Engineering from the University of Southampton in 2012, and a PhD in Machine Learning from the University of Portsmouth in 2015. He is currently a research associate in the School of Computing at the University of Portsmouth. He has previously been a research assistant and a demonstrator in the Department of Operations and Systems Management and the School of Computing respectively at the University of Portsmouth.

His research interests include data mining, machine learning, rule based systems, granular computing, intelligent systems, fuzzy systems, big data and computational intelligence. He was awarded a PhD Studentship in Computing from the Faculty of Technology at the University of Portsmouth in August 2012. He has also been awarded Member of the Engineering and Technology with designatory letters MIET since February 2016.

He published a research monograph with the Springer Series “Studies in Big Data” in the third year of his PhD. He received a nomination for his paper to be a candidate of the Best Paper Award in the 15th International Conference on Machine Learning and Cybernetics. He has also been registered as a reviewer for several established journals such as IEEE Transactions on Fuzzy Systems, Fuzzy Sets and Systems (Elsevier) and International Journal of Fuzzy Systems (Springer). He is also a Member of the Programme Committee for the 16th UK Workshop on Computational Intelligence (UKCI 2016), the 25th International Joint Conference on Artificial Intelligence (IJCAI 2016), the 19th International Conference on Human System Interaction (HSI 2016) and the 9th International Conference on Advanced Computational Intelligence (ICACI 2017).  

About The Author

Alexander Gegov has a BSc in Automatics (1987) and an MSc in Robotics (1989) from the Technical University of Sofia, Bulgaria. He also has a PhD in Control Systems (1992) and a DSc in Intelligent Systems (1997) from the Bulgarian Academy of Sciences. In addition, he has a Postgraduate Certificate in Learning and Teaching in Higher Education (2000) from the University of Portsmouth, UK.

He is currently a Reader in Computational Intelligence in the School of Computing at the University of Portsmouth, UK. He has previously been an EU Visiting Researcher at the Delft University of Technology in Netherlands and Alexander von Humboldt Guest Researcher at the Universities of Duisburg and Wuppertal in Germany.

His research interests are in the area of Computational Intelligence with a focus on rule based systems and networks using data and knowledge to the purpose of modelling, simulation and decision making. He has received funding through the UK Engineering and Physical Sciences Research Council, the South East England Development Agency, the UK Technology Strategy Board, the Bulgarian National Research Fund, the University of Portsmouth Research Sabbatical Scheme, the Erasmus Teaching Mobility Programme, the International Federation of Automatic Control and the Association of European Operational Research Societies.

He has over 100 peer-reviewed research publications including 4 research monographs, 2 edited books, 10 book chapters and about 50 journal articles. He has presented over 20 invited research tutorials and plenary lectures at international research events including IEEE conferences, symposia and congresses. He is Associate Editor for the journals ‘IEEE Transactions on Fuzzy Systems’, ‘Fuzzy Sets and Systems’, ‘Intelligent and Fuzzy Systems’, ‘Computational and Intelligent Systems’ and ‘Robotics and Automation’. He is a Member of the IEEE, the IEEE Technical Committee on Soft Computing, the IEEE Computational Intelligence Society and the IEEE Systems, Man and Cybernetics Society. 

About The Author

Mihaela Cocea has a BSc in Computer Science, a BSc in Psychology and Education and an MSc in Communication and Human Relations from the University of Iasi, Romania. She also has an MSc by Research in Learning Technologies from the National College of Ireland (2007), a PhD in Computer Science from Birkbeck College, University of London, UK (2011), and a Postgraduate Certificate in Learning and Teaching in Higher Education from the University of Portsmouth (2012).

She is currently a Senior Lecturer in the School of Computing at the University of Portsmouth, having previously been a Lab Demostrator at Birkbeck College, Associate Lecturer at the National College of Ireland and a Teaching Assistant at the University of Iasi.

Her research interests are in the area of Intelligent System, focusing on intelligent techniques using data and knowledge engineering to provide adaptation and personalisation, as well as decision support. She has received funding through: (a) scholarships from the National College of Ireland and Birkbeck College, University of London, UK; (b) an internship through the EU Leonardo da Vinci programme; (b) a mobility fellowship from the European Network of Excellence in Technology Enhanced Learning (STELLARnet); (c) research development funds from the University of Portsmouth and (d) travel grants from EATEL (European Association for Technology Enhanced Learning), User Modeling Inc. and NSF (National Science Foundation).

She has published over 60 peer-reviewed papers and has received a Best Project Award at the Summer School on Personalized e-Learning, Dublin (2006), a Best PhD paper award from KES (2010) and was runner up for the 2011 Best PhD Thesis in the School of Business, Economics & Informatics, Birkbeck College, University of London. She is a member of the IEEE and the IEEE System, Man and Cybernetics Society.

.

Citation: Han Liu, Alexander Gegov,  Mihaela Cocea. Collaborative Rule Generation: An Ensemble Learning Approach. Journal of Intelligent & Fuzzy Systems, 2016, 30 (4). pp. 2277-2287. 

Affiliation: School of Computing, University of Portsmouth, Buckingham Building, Lion Terrace, Portsmouth, PO1 3HE, United Kingdom.

Go To Journal of Intelligent & Fuzzy Systems