摘要：

【目的】 基于国际表征学习会议（International Conference on Learning Representations,ICLR）同行评议的文本分析,利用自然语言处理方法探索同行评议中审稿意见效用的评估方法。【方法】 首先定义审稿效用的概念,然后对审稿文本进行标注,应用表示学习与深度学习模型提出一种审稿效用度智能识别方法。【结果】 所提出的智能识别方法能够快速、有效地进行分析,获得审稿文本的效用度。负面审稿意见以及录用论文对应的审稿意见的审稿效用更高,论文水平与审稿专家经验是审稿效用的重要决定因素,中国审稿专家的审稿水平可能与世界平均水平相当。【结论】 应用审稿效用度智能识别方法能够及时发现异常的审稿文本,辅助编委和编辑做出审稿最终决定,改善同行评议评价结果,进而健全同行评议机制。

关键词: 同行评议, 审稿效用, 智能识别, 文本分析

Abstract:

[Purposes] Based on the text analysis of peer review of International Conference on Learning Representations （ICLR）, natural language processing methods are used to explore the method for evaluating the effectiveness of peer review comments. [Methods] We first defined review effectiveness, then annotated the review texts, and proposed an intelligent identification method for review effectiveness with representation learning and deep learning models. [Findings] The proposed intelligent recognition method can efficiently analyze and yield the effectiveness of review text. Negative review comments and review comments on the papers accepted are more effective. The level of papers and the experience of reviewers are determinants of review effectiveness. The reviewing ability of Chinese reviewers reaches the world average. [Conclusions] The intelligent identification method can help identify abnormal review texts in time, assist the editorial board members and editors in the final decision-making, and improve the evaluation results of peer review and the peer review mechanism.

Key words: Peer review, Review effectiveness, Intelligent identification, Text analysis