This project aims at developing a comprehensive conceptual, analytical and empirical framework for the critical assessment of currently employed research and education evaluation practices.
This objective will require overcoming a number of theoretical and methodological challenges, among which: i) designing a wide-ranging framework extending and integrating existing conceptual, analytical and empirical frameworks taking into account recent contributions from moral philosophy -especially normative ethics; ii) integrating existing datasets with case studies, questionnaires and use cases; iii) developing new (more robust) methodologies to estimate the performance of education and research; iv) exploration of what is inside the black box of rankings to include financial data; v) developing a visual analytics environment to implement and test the new performance indicators.
Despite the relevance of this topic for research and educational policy making, there are very few contributions on this issue.
To face these challenges we will adopt a multi-methods interdisciplinary approach. In particular, the research activity foreseen in this project will undertake a number of coordinated actions including the following: 1) Build a Proof of Concepts of the wide-ranging integrated framework. 2) Integrate existing relevant data on research and education (which are available thanks to the other projects in which the research team is involved) within an artificial intelligence based visual analytic environment. 3) Develop a prototype of the visual analytic system for the assessment of performance models. 4) Carry out case studies (on the Italian balance sheets of universities in comparison with UK universities) to assess the feasibility and the improvements of existing rankings by integrating financial data. 5) Developing a questionnaire for the inclusion of emotional intelligence in the traditional evaluation framework.
WHAT THIS PROJECT WILL ACCOMPLISH THAT IS NOT BEING DONE IN THE EXISTING LITERATURE
The first accomplishment of this project relates to the availability of a Proof of Concepts of a comprehensive conceptual, analytical and empirical framework for the critical assessment of currently employed research and education evaluation practices.
The inclusion of normative ethics in our framework will allow us to operationalize the notion of > evaluation of research and education practices making a step further in the measurement of virtues and providing some hints on how to transfer (teach and learn) them.
The Visual Analytics approach that will be developed in this project will allow us to move from Performance indicators development to Performance model development, by exploring and exploiting the modelling and the data features within the flexibility of a Visual Analytics environment. In this research we will leverage on the research based on Sapientia and OBDA combining it with a Visual Analytics approach. The new approach proposed allows us to move from Performance indicators development to Performance model's development, by exploring and exploiting the modelling and the data features within the flexibility of a Visual Analytics environment. This allows a multi-stakeholder viewpoint on the model of Performance Indicators, the assessment of the sensitivity and robustness of the Performance Indicators model in a multidimensional framework.
Finally, the investigation about how rankings can benefit from the inclusion of financial data can provide the European policy makers with helpful suggestions to promote a path to accounting harmonization for the university public financial reports.
HOW WILL THE PROJECT CHANGE OUR UNDERSTANDING ABOUT THE EVALUATION
The prototype of visual analytics environment that will be realized within this project will allow us to make an empirically grounded reflection on the evaluation of evaluations.
Firstly, it will offer a self-assessment tool for understanding the functions of research and teaching activities. This is an important step towards the improvement of research and teaching practices.
Secondly, it will help institutions to collect and describe the main functions of the research and teaching practices (highlighting their special features) developed by its researchers and academic staff, and their motivations. This information may be very useful for institutional strategic plan. This is a further important step for the development and improvement of the organizations involved.
To conclude, this project has the potential of introducing a paradigm shift in the evaluation of research and teaching practices. From an evaluation focused on the
Finally, the results achieved by this project may be the starting point for the preparation of an Advanced ERC grant proposal on the issues investigated within this project.