The editor’s challenge in peer review

Peer review is the core quality control mechanism of science, determining what gets published and funded. Yet, the system is surprisingly outdated and inefficient. Editors face mounting pressure to find the right reviewers, but often rely on outdated tools or author-supplied information that can be biased. The result is a process that struggles with inefficiency, inconsistency and a lack of transparency, especially in how reviewers are chosen. Here, Professor Philipp Koellinger, co-founder and CEO of open science startup DeSci Labs, argues that peer review needs urgent reform and that AI can help science live up to its ideals.

Peer review remains the cornerstone of validating scientific research, but the process of selecting reviewers is outdated, slow, and flawed. Editors, who rely heavily on traditional methods such as scanning reference lists or databases of past referees frequently end up reusing the same reviewers rather than finding the most suitable experts.

This limited and recycled pool can lead to biases or inadequate evaluations, as the chosen reviewers may not always have the ideal expertise or impartiality needed to rigorously assess the work. Despite advances in technology, many journals still depend on these aging systems, which restrict the diversity and quality of peer review and ultimately impact the fairness and reliability of how science gets published and funded. In a recent survey across four scientific publications, editors described finding multiple suitable reviewers as the most frustrating part of their job above all other frustrations ^[1]. The editors also expressed frustration with selecting the most appropriate expert for each manuscript ¹.

Choosing the right reviewer

The quality and integrity of the peer review process hinges on choosing reviewers with the right expertise, impartiality and motivations. A well-qualified referee helps editors make informed and fair decisions by providing thorough, competent, and unbiased assessments of a submission.

Conversely, selecting inappropriate reviewers can introduce bias, reduce the quality of the review and ultimately damage the credibility of the entire publishing process. Ensuring the right experts are involved is essential for maintaining trust in scientific research.

An ideal reviewer should have no conflicts of interest, they must not be recent co-authors, past or current supervisors or supervisees, nor affiliated with the same institution as the authors. They should possess relevant expertise, demonstrated by recent publications on the same topic or employing similar methods, ensuring they understand the nuances of the research.

Beyond expertise, reviewers must be objective and fair, and able to set aside personal biases or rivalries that could influence their judgment. Lastly, the best reviewers are motivated and thorough, willing to dedicate the necessary time and effort to carefully assess a submission.

Limitations of current procedure

Editors and funding agencies play a crucial role in identifying and managing potential biases among reviewers. They carefully select and oversee the process to ensure that reviews are balanced, fair and focused solely on the quality of the research. Currently, editors typically spend up to two weeks identifying suitable reviewers within their specific field and reaching out to them ^[2].

In addition, they often depend heavily on the references cited within a manuscript to identify potential reviewers, which saves time from searching elsewhere. However, this approach has notable drawbacks; authors can manipulate reference lists to steer suggestions toward certain reviewers, and important experts who are not cited may be overlooked entirely.

This method therefore results in a narrow or biased pool of reviewers, missing out on broader expertise that could improve the quality of peer review.

Additionally, the demands placed on editors and funding agencies are significant. Finding suitable reviewers is not just time-consuming but made more difficult by peer reviewers typically serving on a voluntary, unpaid basis. Consequently, editors face ongoing challenges in securing timely and thorough reviews, which can delay the publication process and affect the overall rigor of scientific assessment.

Revolutionising the procedure

AI tools have the potential to transform the reviewer selection process by rapidly analysing vast amounts of scientific literature to identify the most relevant experts.

These advanced systems can incorporate important features such as conflict-of-interest checks, expertise mapping that distinguishes between topic specialists and methodological experts and integration of researcher metadata like H-index and career stage. This also greatly expands the pool of qualified and capable reviewers, something DeSci Labs is actively addressing through its AI-driven Referee Finder tool.

Editors can tailor these algorithms to meet the unique requirements of each submission, ensuring a more precise match between reviewers and manuscripts. By addressing long-standing inefficiencies, AI-powered solutions can significantly enhance both the fairness and speed of peer review.

These tools move beyond the traditional reliance on references, which can be manipulated by authors or overlook key contributors and open the door to a more transparent and robust scientific publishing process.

To explore how DeSci Labs is reshaping science with the help of AI on DeSci Publish, visit the website and sign up.