Peer review in scholarly publication: A cornerstone or a stone in the shoe?

By Maria Kuteeva (Erik Allardt Fellow at HCAS, Fall 2021)

Photo of a door knocker made of metal and shaped like a lion

Photo: Pixhere

Peer review is a cornerstone of academic activity, a marker of quality research and publication. As HCAS Director Tuomas Forsberg (2021) points out in the HCAS Newsletter 2020–2021, peer review is paramount to various activities at the Collegium, starting with the evaluations involved in the fellows’ selection process. In a broad sense, peer review can take many forms, ranging from formal written evaluations to informal spoken interaction. One of the most memorable and enjoyable experiences during my stay at the Collegium was the Fellows’ Seminar, where we discussed our research from different disciplinary perspectives and gave and received constructive feedback in a supportive and collegial environment.

At the same time, we are all engaged in other peer-reviewing activities in our own fields of research, and the demand for peer-reviewing keeps growing. For me, the year 2022 started with several requests to review two journal articles, a book, a research grant application, and a tenure application. Requests for peer reviews of journal submissions have been increasing, as a peer-reviewed journal article is now the most prestigious and desirable type of publication in many fields of research.

In this blog post, I focus on the ‘occluded’ genre (Swales 1996) of the anonymous peer review for academic journals. In today’s world of academic publishing, the journal and the complex of norms associated with its activities represent the main centres of authority. As concrete representatives of this authority, journal editors act as gatekeepers in the process of knowledge production, and peer review is meant to inform and support their decisions. The way in which peer review is currently set up often results in a hierarchical and structured activity, geared largely towards journals as centres of authority rather than our research peers. Has peer review become an oxymoron? What role do reviewers play in maintaining or challenging the authority of academic journals? How is this power dynamic manifested in the discourse of peer review? My discussion below engages with these questions.

The discourse of peer review: The reviewer, the author, and the manuscript

As we all know, not all peer review is conducted in the same spirit as our HCAS Fellows’ Seminar. The discourse of anonymous peer reviews is not always transparent, as criticism can be hedged and requests for essential changes can be phrased as polite suggestions (e.g. the author(s) might want to edit …). On the other hand, not all reviewers are as polite and tactful: harsh criticism can take rather personal undertones (see https://shitmyreviewerssay.tumblr.com). Hyland (2021) points out several other potential pitfalls associated with the peer review process: long decision delays, bias and subjectivity, and even dishonesty. For example, during the reviewing process for what would become a well-cited article co-authored with my then doctoral student for a top journal in our field, one reviewer provided comments littered with capitals, exclamation marks, expressions such as ‘gee what a finding’ and ‘what does this mean, honestly?’, and even threats ‘I really want to see this out of this paper and if it is not, I cannot recommend it for publication’. Although this was not the case with our submission, when taken to the extreme, peer review can be a mechanism for censorship.

As the sheer number of scholarly publications keeps growing, they seem to become increasingly standardised. Academic genres, such as the journal article, have evolved to reflect the rhetorical norms of research communities. Behind the façade of these normative genres is a process involving dialogue and negotiation, the main primary purpose of which is to advance knowledge in a research field. Journal reviewers and editors play a key role in this dialogic process, and their comments reflect the disciplinary, socio-economic, socio-cultural, and socio-political contexts in which they operate.

Research on the discourse of peer review is scarce because data are notoriously difficult to obtain. Previous studies conducted in applied linguistics have drawn on a limited dataset of reviews from one journal (e.g. Paltridge 2017). At the same time, interest in this topic is growing, and researchers are being asked to examine their own practices in order to demystify the peer-reviewing process. I have recently contributed one such study to a forthcoming volume titled The Inner World of Gatekeeping in Scholarly Publication (Kuteeva, in Habibie & Hultgren, forthcoming). The idea was to critically reflect on my own trajectory and practices as a peer reviewer, based on an analysis of 50 reviews that I have written over the last decade in response to manuscripts submitted to 15 academic journals.[i]

A black-and-white photo of a group of blindfolded people sitting at a table

Photo: Library of Congress, Prints & Photographs Division

Although the peer review is an academic genre, its discourse features are not the same as those of published genres like journal articles or books. The dialogic and evaluative features are more pronounced, as evidenced by the abundance of stance expressions and directives (Paltridge 2017; Samraj 2021). In line with previous studies, my initial corpus-assisted analysis points towards the centrality of evaluative and attitudinal stance in the discourse of my peer reviews, manifested in the frequent use of the pronoun I accompanied by various stance verbs (e.g. I agree, I believe, I (cannot) recommend, I find, I suggest, I wonder). All stance-taking acts involve two subjects (e.g. the speaker and the audience) and an object, thereby forming the stance triangle (Du Bois 2007). In peer review, this triangle involves, above all, the reviewer, the author of the journal submission (the audience), and the manuscript as the object of the stance-taking act.

Unlike in spoken interaction, the dialogue in peer review is written and asynchronous. In this context, the centrality of the journal adds a communicative dimension in line with Bakhtin’s (1986) concept of ‘superaddressee’, a metaphor used to describe a complex of norms or a larger body of authority. In academic publishing, this dimension involves journal readers as individuals, the research community as a collective and abstract entities, such as research ethics and language standards. Thus, the reviewer’s utterances are not only directed at the author of the manuscript: they are also shaped with reference to a higher evaluating authority of a perceived centre, in other words, the editors or the journal readership more broadly.

Although the object of the stance-taking act – the journal submission – remains the same, the reviewer role can change from that of evaluator to advisor, peer or (proof)reader. Accordingly, the audience of the reviewer’s utterances may expand to include the editor, the journal readership or the research community more broadly. Evaluations are accompanied by different stances towards the journal submission. Shifting between different reviewer roles can also involve different alignments with the audience to either include or exclude the author or the editor. For example, the reviewer may address the author directly (I suggest swapping the order of the subsections) or else choose to align with the journal editor by mentioning the author in a dependent clause (I recommend that the authors review some more recent literature).

The shifts between different reviewer roles and audiences are manifested through register variation, in which ‘clustered and patterned language forms (…) index specific social personae and roles’ (Blommaert 2007: 117). Resorting to a particular register is a way to index belonging to a particular group with its own repertoire of voices, e.g. the reviewer as an evaluator and expert in the field (e.g. I cannot recommend this article for publication) or a friendly and supportive peer (e.g. have you considered doing it this way?). For example, the reviewer as evaluator can resort to conventional ‘reviewer speak’, aligning with the journal editor (e.g. I cannot recommend this article for publication) or indicating to the author how the manuscript can be improved (I found this section surprisingly short). The expert role concerns the reviewer’s knowledge of the field and what research is needed to advance knowledge in the field (the article has the potential to offer new insights into…). As an advisor, the reviewer is likely to either directly or indirectly address the author (I suggest swapping the order of these subsections). The peer role is similar to that of advisor but involves more proximity with the author, for example through the use of the pronoun you. As (proof)reader, the reviewer may comment on the quality of the text and point out specific infelicities.

The reviewers’ role: Maintaining the status quo or challenging the journal’s authority?

Authority in accepting an article for publication resides with the journal editors, and the peer review process is meant to inform and support the editor’s decision. To make a convincing recommendation for the journal, the reviewer is likely to resort to register features that index their proximity to the journal as the centre of authority. In this context, the very concept of peer review appears to be an oxymoron, as it is, in fact, a hierarchical and structured activity oriented primarily towards journals as perceived centres of authority and expertise rather than our colleagues and research peers.

Although academic journals hold strong authority and have established gate-keeping mechanisms, they may also have their caveats. Since these journals represent both real and perceived centres, their practices risk becoming too centripetal and inward-looking. Based on the existing literature and my own experience, I would argue that there are two main limitations, which can be broadly described as anglo-centricity and ‘disciplinary navel-gazing’. The question of anglo-centricity has been debated in connection with the reported challenges experienced by non-anglophone researchers in getting their work published (e.g. Canagarajah 2002; Hyland 2016). For example, a great deal of debate in the applied linguistics research community has revolved around questions concerning linguistic (in)justice caused by the dominance of English and the need to move away from norms based on established varieties of Standard English (see, e.g. Hynninen & Kuteeva 2017; Kuteeva & Mauranen 2014; McKinley & Rose 2018).

There are also more subtle and serious biases that extend beyond language issues. In 2020, the journal Applied Linguistics (OUP) hosted a debate about knowledge production in the field, challenging the dominance of certain modes of enquiry and raising awareness of the need to decolonise scholarship, for example by engaging with epistemologies of the Global South. Hultgren (2019) shows how the controversies around the dominance of English disregard the importance of socioeconomic factors in shaping publication practices. Finally, we cannot underestimate the importance of languages other than English in increasing societal impact and sustaining language diversity in research activities, not only publishing but also assessment, funding and so forth. The Helsinki Initiative on Multilingualism is at the forefront of this movement by promoting equal access to research findings in a variety of languages, offering support to national publishing infrastructures and providing equal rewards for publications in different languages.

‘Disciplinary navel-gazing’ appears to be a side effect of the need to provide journal publication outlets in specific fields, which can ultimately lead to (re)producing the same kind of knowledge. Academic journals have a key role to play in maintaining or challenging this status quo. Much too often, publications end up harping on the same note, as the authors are encouraged to cite previous research that appeared in the same journal, whose authors then act as reviewers of new submissions. The need to increase the journal impact factor also makes it easier for highly-cited authors to have (any) work accepted in quality journals because it is likely to attract more citations. The two perceived limitations – anglo-centricity and disciplinary navel-gazing – may be intertwined, particularly in fields where English is the dominant language of research communication and where evaluation is based on quantitative indicators.

Three marble statues depicting people who gaze at their navel

Satyres en atlante, Unknown Artist, II century after J.C., Rome. Photo by Gregg Tavares. https://www.flickr.com/photos/greggman/4305490456/in/photolist-7ysLj5-a5Vsnw-hGT6T-hGT8v. Photo uploaded on 16.1.2010. Accessed on 1.2.2022.

Where does the peer reviewer stand in this landscape? Is it possible for peer reviewers to contribute to addressing the aforementioned caveats regarding academic journal publishing? I would argue that it is possible. The reviewer is in a position to challenge centripetal journal practices without misaligning with the journal and its editors. This involves a balancing act between aligning with the journal as an established centre of authority (e.g. through the use of an appropriate register) while at the same time questioning aspects of research that are associated with the two limitations.

In my experience, one straight-forward strategy is to alert the authors and editors of journal submissions to cutting-edge research carried out in non-anglophone contexts (sometimes in languages other than English) and to encourage them to engage with it more thoroughly. It is not unusual for authors working in an anglophone context to assume that their readers would share background knowledge about their research context. However, the same cannot be said for authors who write in English but are based in non-anglophone contexts. This kind of indirect benchmarking can be limiting. In my peer reviews, I have encouraged authors working in both kinds of contexts to take a reflective approach and discuss their own positionings.

Last but not least, it is important – particularly for scholars in the humanities – to keep an open mind about writing conventions and not to be overly confined by the straitjacket of templates and increasingly standardised academic genres. I am sure there are other good strategies for overcoming limitations of the perceived centres of authority in knowledge production and would be delighted to hear your views.

To sum up, although academic journals function as the main centres of authority in writing for research publication, they come with their limitations, as centripetal trends in the practices surrounding knowledge production and publication can be counterproductive to moving the research field forward. By mediating the dialogue between the authors and the journal, peer reviewers have a key role to play in both maintaining and challenging the journals’ authority as centres of knowledge production.

References:

Bakhtin, M. (1986) Speech Genres and Other Late Essays (C. Emerson and M. Holquist, eds) (V. McGee, trans.). Austin, TX: University of Texas Press.

Blommaert, J. (2007) Sociolinguistics and discourse analysis: Orders of indexicality and polycentricity. Journal of Multicultural Discourses 2 (2), 115–130.

Canagarajah, A. S. (2002). A geopolitics of academic writing. Pittsburgh: University of Pittsburgh Press.

Du Bois, J. (2007). The stance triangle. In Englebretson, R. (ed.), Stancetaking in discourse: Subjectivity, evaluation, interaction. Amsterdam / Philadelphia: John Benjamins.

Forsberg, T. (2021). “The Quality of Academia Depends on the Quality of Reviewing”, Tuomas Forsberg (HCAS Newsletter 2020–2021, https://www2.helsinki.fi/sites/default/files/atoms/files/hcas_newsletter_2021.pdf)

Hultgren, K. (2019). English as the language for academic publication: On equity, disadvantage and “non-nativeness” as a red herring. Publications, 7 (2), 31.

Hyland, K. (2016). Academic publishing and the myth of linguistic injustice. Journal of Second Language Writing, 31, 58–69.

Hynninen, N. & Kuteeva, M. (2017). Good” and “acceptable” English in L2 research writing: Ideals and realities in history and computer science. Journal of English for Academic Purposes, 30, 53-65.

Kuteeva, M. (forthcoming). Polycentric peer reviewing: Navigating authority and expertise. In Habibie, P. and A.K. Hultgren (eds.). The inner world of gatekeeping in scholarly publication. Palgrave Macmillan.

Kuteeva, M. and Mauranen, A. (2014). Writing for international publication in multilingual contexts: An introduction to the special issue. Journal of English for Academic Purposes, 13, 1-4.

McKinley, J. & Rose, H. (2018). Conceptualizations of language errors, standards, norms and nativeness in English for research publication purposes: An analysis of journal submission guidelines. Journal of Second Language Writing, 42, 1-11.

Paltridge, B. (2017). The discourse of peer review: Reviewing submissions to academic journals. London, UK: Palgrave.

Samraj, B. (2021). Variation in interpersonal relations in manuscript reviews with different recommendations. English for Specific Purposes, 62, 70-83.

Swales, J. (1996). Occluded genre in the academy: The case of the submission letter. In E. Ventola and A. Mauranen (eds.), Academic writing: Intercultural and textual issues. Amsterdam/Philadelphia: John Benjamins.

[i] The reviews were written for the following journals: Applied Linguistics, Discourse, Context and Media, English for Specific Purposes, Higher Education, International Journal of Bilingual Education and Bilingualism, Journal of English for Academic Purposes, Journal of English for Research Publication Purposes, Journal of Multilingual and Multicultural Development, Journal of Pragmatics, Journal of Second Language Writing, Linguistics and Education, Multilingua, Nordic Journal of English Studies, Studies in Higher Education, System.

On Excellence

By Tuomas Forsberg

The stated mission of the Helsinki Collegium is to carry out high-level research in the humanities and social sciences. Given this key purpose, it is essential that in the international research assessment of the entire University of Helsinki in 2019 that focused on the past decade, the Collegium received the grade of “excellent” for both the quality of research and the research environment. As excellence ought to be recognised by others, it is important that what we say we are aligns with what we do.

So I would like to take this opportunity to thank my predecessors and all our former fellows who have contributed to this success.

We may, of course, ask how excellent is “excellent”. Something would be terribly wrong with the concept of an institute of advanced study if the Collegium were not recognised as a better research environment than teaching units and if the quality of the research environment did not translate into quality of research. However, following the academic good practice of doubt and self-criticism, there is no justification for resting on one’s laurels. Even excellence can be improved.

Societal impact from the bottom up

The Collegium received the grade “very good” in the assessment of societal impact. “Very good” is not a bad achievement but already literally a very good result. Yet, given the available resources, to what extent can we realistically improve our societal impact without also jeopardising our excellence in research? Many institutes for advanced study worldwide have reckoned that the old idea of the “usefulness of useless research” is not sufficient. Accordingly, they have started to pay more attention to societal impact to meet the expectations or even demands of the authorities, funding bodies and sponsors.

The Collegium’s visibility and outreach have emerged both locally and internationally. For example, it has been active in social media, through blogs and in public events organised at the new Think Corner of the University of Helsinki and streamed worldwide.

Speakers of the spring 2019 Kollegium Talks on Think Corner Stage

Public Kollegium Talks event “Unexpected turns in research paths”, with Erkko Professor Jane Cowan, Core Fellows Michael Langlois and Emilia Mataix Ferrándiz and Research Coordinator Kaisa Kaakinen on the Think Corner Stage, March 11, 2019 (Photo: Minerva Juolahti)

However, probably the best way for the Collegium to foster societal impact is by facilitating the activities of its researchers. Just as the research carried out at the Collegium is bottom-up by nature, so should its societal impact be. Given that Collegium researchers are exempt from major administrative and teaching duties, they can in fact address new topical issues much faster of their own initiative as well as find more time for societal interaction. Many researchers already know how they can reach out to the relevant audiences. In addition, some researchers are better positioned for societal interaction than others. Moreover, research and societal interaction are typically sequential, since impact is based on research that first has to be carried out. Therefore a kind of division of labour should apply to institutions. Given the diversity of fields and issues represented at the Collegium, it is not easy to identify a core audience other than those interested in knowing what is going on and what is new in academic research in the wide sense.

Can impact be measured?

Societal impact, while definitely important, is difficult to measure reliably. In fact, attempts to do so, particularly when it affects funding directly, may lead to unintended consequences. As is well-known, measuring the societal impact of academic research is difficult because that it may take a long time before the impact becomes visible, and it is often impossible to attribute the impact of scientific knowledge to particular research outcomes. A related question is whether we should reward research that could or should have had an impact, but has failed to have one. Politicians and other decision-makers still make choices on the basis of their preferences and they may discard the scientific evidence. What if we reward outcome, in other words research that has had impact, but for reasons that may have nothing to do with the quality of research? Scholars are expected to be active in the society and broaden their expertise beyond their own academic research. We should reward researchers for their societal impact based on their scholarly expertise, but it is very difficult to do so without rewarding them also for their societal impact that is based on mere civic activism. By the same token, there is no objective way of separating good impact from bad. And even if there were a clear definition of societal impact, it can remain a secret:  some of the most significant instances of societal impact – when advice is given to key decision-makers – are not meant to be publicly acknowledged.

Societal impact should definitely be part of the academic ethos that guides our research. This should not imply that research should be evaluated in terms of its short-term goal or that the societal impact of research can be measured accurately. Moreover, there is no contradiction in claiming that we should pay attention to the societal impact of research, and that we still need places where that is not the primary concern. The more universities and research institutes are required to demonstrate their relevance by addressing immediate societal concerns defined in a top-down manner, the more important it becomes that at least some institutes can focus on basic, curiosity-driven research.

Group photo of Helsinki Collegium fellows standing outside on grass

HCAS Fellows and staff in September 2019 (Photo: Veikko Somerpuro)

Tuomas Forsberg has been the Director of HCAS since August 2018.

This article has also been published in the HCAS Newsletter 2019-2020.

On population genetics, emotions, and entangled differences

Venla Oikkonen

My first year at the Helsinki Collegium for Advanced Studies was intense, inspiring and rewarding. While starting a new research project on cultural debates about vaccines, I had the opportunity to finish my second monograph on the cultural dimensions of human population genetics. The book, Population Genetics and Belonging: A Cultural Analysis of Genetic Ancestry, was published in October by Palgrave Macmillan. Many of the final details of the book reflect discussions I had with my wonderful colleagues in the Collegium hallways, at coffee breaks, and in our Brown Bag seminars.

As the title suggests, the book is an account of population genetics as a means of making communities, identities, and belonging. Human population genetics is a field of science that studies genetic variation within and between populations. The book explores technological practices and cultural imaginaries that population genetics has engendered in contemporary societies. For example, population genetics underlies commercial genetic ancestry tests, which promise to trace our personal roots to prehistoric communities and connect us to people who share genetic markers with us. Population genetics provides the basis for national genome initiatives. Population genetics informs the study of DNA retrieved from ancient human remains, playing a crucial role in attempts to imagine human evolution. Population genetics structures the ongoing building of biobanks, which are meant to accelerate scientific research and pharmaceutical development through faster circulation of samples and information. Population genetics is also present in the marketing of pharmaceutical products to specific ethnic communities on the basis of assumed population-level genetic differences. Moreover, population genetics has been invoked in both pro- and anti-immigration campaigns, revealing the ambivalent relationship between population genetics and politics of inclusion and exclusion.

How has population genetics become part of all these diverse projects? How can it support opposite political and social agendas? Why are the tensions and connections between these projects and practices seldom discussed? In Population Genetics and Belonging, I investigate the mutability and persistence of population genetic imaginaries by tracing shifts and continuities in the uses of population genetics from the late 1980s until today. The book explores these shifts and continuities through a range of materials including scientific articles, journalism, popular science books, online genetics websites, and fiction.

Throughout the book, my analysis of shifts and continuities in the uses of population genetics focuses on two issues: affect and intersectional differences. Affect refers to the cultural circulation of emotions and the emergence of emotional intensities around objects such as genetic technologies or biological samples. By intersectional differences I refer to how technologies are entangled with assumptions of gender, sexuality, race and class, and how these differences are constituted through one another. I argue that in order to understand the potential impact of population genetics in society, we need to pay attention to how population genetic technologies shape gendered, racialized, classed and sexualized differences, and how these differences become emotionally charged.

The theory of “Mitochondrial Eve” developed in the 1980s provides an illustrative example. “Mitochondrial Eve” is the most recent maternal ancestor of all currently living humans traced through mitochondrial DNA inside our cells. Eve is not a specific woman but rather a statistical point of origins in the past where mitochondrial variation among modern humans originates. Through an analysis of scientific, media, and fictional texts, I trace how the theory of Mitochondrial Eve becomes contested, celebrated, and gradually routinized in the early 1990s. I also explore the appearance of another gendered figure, “Y-Chromosome Adam”, in the mid-1990s. While Mitochondrial Eve draws on the idea of an unbroken maternal chain between us and our evolutionary past, Y-chromosome Adam is the most recent patrilineal ancestor of currently living men traced through Y-chromosome DNA. In cultural discourses surrounding population genetics, Y-Chromosome Adam became almost immediately portrayed as Eve’s counterpart, partner, and even boyfriend. This portrayal questioned the initial cultural representations of Eve as a strong and independent woman, as well as contradicted the scientific evidence that placed Eve and Adam at different prehistoric times.

Population Genetics and Belonging shows how the figures of Eve and Adam mobilize cultural assumptions of gendered, sexual and racialized differences while traveling across science and culture. The first chapter demonstrates how the pro-feminist and multicultural undertones of the figure of Mitochondrial Eve evoked strong emotional responses in science journalism, mainstream media, popular science, and fiction, and how some of the underlying cultural anxieties were alleviated through heteronormative narratives involving Adam. In subsequent chapters, I ask what happens to these emotionally charged differences when mitochondrial and Y-chromosome DNA is used in commercial genetic ancestry tests, ancient DNA research, or debates about the roots of national populations.

All in all, the book argues that genetics and emotions are thoroughly entangled: emotions shape as well as reflect developments around population genetics in society. In particular, emotions arise around cultural perceptions of how genetic accounts of human evolution may reshape social categories of difference such as gender, class, sexuality, and ethnicity. Finally, the book suggests that formations of affect and difference around biotechnologies call for critical exploration also beyond the field of population genetics.

Venla Oikkonen is Core Fellow at the Helsinki Collegium for Advanced Studies. Her research interests include evolutionary theory, genetics, vaccine controversies, pandemics, and theoretical questions related to affect and intersectionality. Her first book Gender, Sexuality and Reproduction in Evolutionary Narratives was published by Routledge in 2013.

TUHAT research profile: https://tuhat.helsinki.fi/portal/en/person/voikkone