This project proposes a conceptual and empirical framework for developing models of indicators of knowledge dynamics and to map their co-production and co-evolution over time.
This objective will require overcoming a number of methodological and theoretical challenges, among which: i) the design of a general Co-Creation Indicators Factory embracing all the actors involved in the interaction between knowledge and society, and in their co-production and co-evolution; ii) the integration and consolidation of large and heterogeneous sets of micro-level data with meso and macrodata; iii) the development of a sound visual analytic approach to specify, evaluate and visualize suitable and timely indicators of knowledge co-creation, including contextual connections and feedbacks from the general public (society); iv) the development of new methodologies to estimate 'socially robust' impact indicators.
Despite the relevance of this theme for evidence-based policy making, there are few analyses on this topic.
To face these challenges, the research activity foreseen in this project will undertake a number of coordinated actions: 1) Build a Proof of Concepts of the Co-Creation Indicator Architecture. 2) Develop a prototype of the visual analytic system on data already available on research and higher education systems and test it on the University of Rome La Sapienza , extending it where possible to the Italian and European level. 3) Integrate these data (which are available thanks to the other projects in which the research team is involved) with a pilot investigation on the relevant and available data from the civil society. 4) Deliver the system usage requirements, including indications of the policy of the usage and the related requisites; 5) Carry out a few case studies of how data are used to derive indicators in the context of the European Research Area (ERA), including indicators to assess interdisciplinarity and map their co-evolution over time.
WHAT THIS PROJECT WILL ACCOMPLISH THAT IS NOT BEING DONE IN THE EXISTING LITERATURE
This project proposes a radically innovative conceptual model for the generation of indicators, which can be described in four steps (see Bonaccorsi, Catalano, Daraio and Moed, 2016; Daraio and Bonaccorsi, 2017).
1: conceptualizing education, research and innovation systems as complex systems formed by more elementary entities. There are several possible ways of doing this, as proposed in the literature. One distinction is between an actor-centred perspective, which defines elementary action units in the system, and a function-based perspective, which, in contrast, works more on structural relations and flows within the system. We will try to reconcile these two perspectives. A starting point is to think in terms of actors.
2: developing concepts associated with entities. The development of concepts will be enriched by the consideration of users' requirements for indicators. Rather than following strict definitions, actors should be conceptualized in terms of a list of associated concepts.
3: looking for lists and standards which are enormously important for automatic searches.
4: establishing links between concepts. We obtain a graph of actors, each with an associated graph of concepts (direct and indirect link).
Then we are ready to implement a principle for the integration of education, research and innovation information systems. The informal principle we follow is: put your information in the region of the graph in which the potential for generation of other information is greater.
This approach is orthogonal to the commonly employed approaches in indicator-based analyses, which starts from user requirements (usually very demanding) and builds up an ad hoc integration of existing databases. This approach is based on a thorough conceptual analysis of concepts and their relations, after which the combination of data is greatly facilitated. In this way, the integration of datasets becomes a smooth process, because the underlying conceptual structure has been clarified ex ante (see Poggi et al., 2008; Lenzerini, 2011).
In principle, indicators should be designed according to the needs of the users. However, these needs are highly fragmented and idiosyncratic. Indicators must follow the informational needs of sophisticated and heterogeneous users closely. To design a conceptual framework able to account for a very broad user requirements perspective, as starting point (to extend during the first phase of the project), we will refer to the work of a group of experts (AUBR) of the European Commission (EC, 2010) which considered as main user groups the following:
- Higher education (HE) management and governance
- Governments and government agencies
- Academic organizations and academies
- Individuals
- Peer HEIs
- Industry partner organizations
- Civic society and civic organizations
- Ministries of higher education in developing countries
- Sponsors and private investors
- The wider public.
HOW WILL THE PROJECT CHANGE OUR UNDERSTANDING OF INDICATORS¿ CO-CREATION DESIGN
We aim at introducing a change in the way indicators are designed. A departure from the more traditional logic is suggested, namely to:
- start top¿down with a thorough conceptual analysis of the research and the higher education system;
- draw all linkages between concepts, particularly the sharing of concepts between entities;
- examine the linkages down to the lowest possible level of granularity, in order to ensure that the sharing is complete;
- generate new indicators through the combination of shared concepts;
- implement the new indicators with existing data, or identify areas that may produce the largest increase in connectivity among concepts.
Following this train of logic, once indicators are created, their feasibility is immediately visible. If there is an area of the infrastructure uncovered by existing data, it is clear where to invest in the production of new data.
Furthermore, the process is recursive. Once new indicators have been generated, they will share the concepts that have originated them. Then they enter, again, into the process of identification of shared concepts. Indicators built over other indicators become feasible.
An open-data platform, which we pursue in this project, encourages and supports new research developments in the generation of new indicators carried out by scientists, which exploit the accessibility and transparency of data. In this way, there will be opportunities for the creation of new indicators beyond the short-term needs of policy-makers.
In the open-data framework we propose, communities of users can, in fact, contribute to the improvement of the documentation and identify pitfalls and shortcomings of indicators.