In the spring of 2016, a discussion regarding research funding policies was raised by tenured professor Atte Oksanen, University of Tampere, and professor Pekka Räsänen, University of Turku. Oksanen and Räsänen have analysed the output from projects funded by the Academy of Finland’s Research Council for Culture and Society over a ten year period. Their analysis, which was published in the journal Tieteessä Tapahtuu, showed that the 391 million euros investments in a total of 1841 research projects have resulted in surprisingly little output in terms of publications. I wrote a piece about the discussions surrounding this study in the Curie journal, published by the the Swedish Vetenskapsrådet. (read it here)
Oksanen’s and Räsänen’s study raised a lot of discussion and was also reported on in the mass media, for example in Helsingin Sanomat (10.4.2016) Both study proceedings and results were questioned, and the text was also addressed in a lengthy reply on the web page of the Academy of Finland.
Especially the approach of measuring academic productivity by appearance in the Scopus data base was critisized as a doubtful proceeding when comparing disciplines. While I do sympathize with critique of a fixation on publication metrics when assessing academic performance, I want to here take the opportunity to discuss two questions in the core of the evaluation of such performance: the scientific article, on the one hand, and the balancing between mediating traditions and new investigations, on the other. Both of these aspects are relevant for the evaluation of academic quality that is needed in funding and appointment processes.
In their critique of the current national funding system Oksanen and Räsänen argued that in comparison to the Finnish system, some international evaluation processes, such as the one of the ERC grants, pay much more attention to scholarly skills. In the next I will discuss the potential role of the peer reviewed article as well as the balancing between tradition mediation and new investigations in the assessments of such skills.
The peer reviewed article
The reader of Oksanen’s and Räsänen’s study will inevitably ask on what basis Scopus appearance even strictly hypothetically speaking could serve as a measurement of scholarly skills? The answer may partly lie in the status of the peer reviewed international scientific article, which has become the main measurement of academic performance in the social sciences. While in its short report-mode often claimed to be mostly serving the natural sciences (and imitating approaches in social sciences), the article has also become increasingly appreciated in more descriptive meaning- and theory-based research traditions.
Writing a scientific article involves a demanding comprehensive, technical and cognitive workload regarding integrated aspects of content and format. Not only need scholars stick strictly to their focus on the research topic, but must also have the ability to squeeze their message into content and format requirements of the journal, not to mention getting it passed the peer reviews. If a researcher has the ability to get an article published, it is, no doubt, a sign of great competence. If a scholar is able to publish many articles, it might, in addition, reflect an impressive continous drive in their abilities to transfer scholarly contributions into digestible output rations in a focused, well-organized and well-communicated manner. This is generally regarded as a valid measurement of scholarly skills (1).
The high status of the article may also be a result of how this genre has developed over time: Gross et al. (2002) have tracked the progress of the article as a genre of discourse over a 300 year period, showing that it increases in impression of objectivity, it changes towards greater cognitive complexity, and more efficient communication. Furthermore, they see an increased complexity in the genre’s inherent style dimensions.
One important explanation of the high status of the scientific journal article in English is undoubtedly the global boost in innovation and application that science policy has increasingly come to aim at (2). The article is thought to ensure the speed of progress needed, while – at least seemingly – guaranteeing some standard. This is, of course far from the truth. There are a number of critical evaluations of this genre both per se, and as an academic performance measurement (see Cianflone 2014; Cronin & Sugimoto 2014)
The idea that the article serves as a more aprehensible format for bringing out new results and ideas than for example the monography book brings me to my second topic for this blog text: namely a dichotomy in the current Finnish university culture between the mediation of tradition and the exercise of research activities. I suspect that an explanation for some of the commotion that Oksanen’s and Räsänen’s stirred up may be due to the cultural clash between the old tradition-bearing and the new inquiry-oriented profiles of scholarly personas at Finnish universities today. In a comment to my piece in Curie, it is speculated whether the study by Oksanen and Räsänen may just be yet another case of researchers in disciplines that spit out large amounts of publications trying to prove their own superioirity to those disciplines that do not have the same modus operandi.(3) (Personally, I think such an interpretation is not very interesting at all, as the reply would simply be that those who do not agree are also just people who want to stress their own kind of excellence position in the same field.)
Leaning on tradition, contributing with new
I like to think that one can illustrate the different amount of emphases on publications by different researchers by the help of a four-field created by higher education pedagogy researchers Annala and colleagues (2016). The four field, depicted in the figure below, originally illustrates the kind of teaching that research-integrated curriculum typically involve. But it serves also as an illustratation of differences between the lower area of the four-field concerned with the university’s research mediating tasks and the upper field’s research producing tasks.
While scholars’ academic performance can be evaluated in both fields, one can claim that the lower field functions as a base of the house: one cannot do new research without any knowledge of previous results or sans any integration of existing theories and techniques in this perfromance. Basically, — and yet again simplifying a great deal – one can say that doing research is automatically invested, to some degree, with the lower field’s tradition mediation-functions. However – and interestingly enough — it is possible to mediate research and traditions without being able to contribute to them in new inquiries.
I read Oksanen’s and Räsänen’s lines of argumentation as the one of the national funding system in social and cultural research not being enough focused on researchers’ “doing” and the appreciation of their skills in this “doing”. In such a situation, appreciation of scholarly performance is downplayed while single project plans, their thematic focus and theoretical position, as well as researcher’s personae, acquire more attention.
Specualting further: Assuming that the tradition mediation of the lower field of the figure would automatically be brought along on the ride, skill assessments of performance in the upper field could be seen as some sort of guarantee for system progress and an adaption to relevant historical circumstances as well as some sort of preservance of tradition. Well aware of that this is very sensitive and controversial point that I am making, I would nevertheless claim that, without hesitation, one measurement of such strengths by scholars and their institutions is bound to be the international peer reviewed scholarly article, easily accessed through the Scopus or other data bases.
…conclusions
Oksanen and Räsänen introduced Scopus citations as a research-political measurement of the level of output of research projects. Their attempt was openly declared as just an attempt: their instrument would in any case need further operationalisation and additions in order to become more valid and nuanced. Still, their discussion raised about productivity is important for the academic community’s understanding of itself, and also for its credibility in society. Amounts of publications can function as some sort of thumb measurement for assessing level of activities in projects. As such, it is certainly not an irrelevant measurement of output from investments into research. It should by no means be the only one, but, as it stands, it does hold some validity.
ADDITION 20/09/2016: Please, read also Björn Hammarfelt’s blog insert on how Wennerås and Wold (1997) have shown that bibliometrics also can serve to increase fairness and point out lack in equality between how candidates are treated in academic review processes. Hammarfelt is a researcher specialized in bibliometrics and science policy.
Footnotes
(1) provided that the authors are really contributing – see Claxton (2005) for “ghost names” as an academic phenomenon.
(2) Around the turn of the millennium, such a paradigm shift of research was described in classic pieces on modes of research production such as Nowotny et al. (2003) and Nowotny (2003).
(3) At least Oksanen’s scholarly belonging is originally very strongly situated in cultural studies, so I am a bit skeptical to a clear-cut interpretation from this point of view.
References
Annala, J., Mäkinen, M. & Lindén, J.: Tutkimuksen ja opetuksen yhteys yliopistossa – opetussuunnitelmatyön näkökulma.Kasvatus & aika 3/2016. Available at: http://www.kasvatus-ja-aika.fi/site/?lan=1&page_id=719
Cianflone, E. (2014). Communicating science in international English: scholarly journals, publication praxis, language domain loss and benefits. Círculo de lingüística aplicada a la comunicación, 57, 45-58.
Claxton, L. D. (2005). Scientific authorship: Part 2. History, recurring issues, practices, and guidelines. Mutation Research/Reviews in Mutation Research, 589(1), 31-45.
Cronin, B., & Sugimoto, C. R. (2014). Beyond bibliometrics: harnessing multidimensional indicators of scholarly impact. MIT Press.
Gross, A. G., Harmon, J. E., & Reidy, M. S. (2002). Communicating science: The scientific article from the 17th century to the present. Oxford University Press.
Nowotny, H., Scott, P., & Gibbons, M. (2003). Introduction: Mode 2’Revisited: The New Production of Knowledge. Minerva, 41(3), 179-194.
Nowotny, H. (2003). Democratising expertise and socially robust knowledge. Science and public policy, 30(3), 151-156.
Oksanen, Atte och Räsänen, Pekka (2016) Suomen Akatemian rahoittaman tutkimuksen tieteelliset tuotokset kulttuuri- ja yhteiskuntatieteissä. Tieteessä tapahtuu 3/2016. sid 16-23.
