摘要：

【目的】 在开放同行评议的大背景下,探讨会议论文同行评议得分与其被引频次的关系,从而分析同行评议结果与传统文献计量指标在科研评价中的关系,为完善科研评价体系提供一定的参考。【方法】 基于OpenReview平台提供的ICLR会议论文的公开评审数据,将全部论文划分为口头报告、海报展示和拒收论文三类,运用文献计量和统计分析方法探究论文的同行评议得分与被引频次之间的关系。【结果】 三类论文在评审得分和被引频次方面均存在显著差异,同行评议得分与被引频次存在较显著的正相关性。【结论】 同行评议与传统文献计量指标在科研评价方面的一致性较高,但并非相互替代的关系,文献计量指标应是对同行评议的重要补充。科研评价体系应该是建立在定性同行评议的质量评价基础上,融合定量文献计量指标,形成一种主客观相结合的融合评价模式。

关键词: 论文评分, OpenReview, 被引频次, 开放同行评议

Abstract:

[Purposes] Under the background of open peer review, this paper discusses the relationship between the peer review score of conference papers and their cited frequency, and further analyzes the relationship between peer review results and traditional bibliometric indicators in scientific research evaluation, which is expected to provide a reference for improving scientific research evaluation system. [Methods] The public review data of International Conference on Learning Representations (ICLR) papers were retrieved from OpenReview, and the papers were classified into oral reports, poster presentations, and rejected papers. Then, bibliometric method and statistical method were employed to explore the relationship between the peer review score and cited frequency. [Findings] Both the review score and cited frequency were significantly different among the three types of papers, and the review score was highly correlated with cited frequency. [Conclusions] Peer review and traditional bibliometric indicators have high consistency in scientific research evaluation, but they are not substitutes for each other. Bibliometric indicators supplement peer review. Scientific research evaluation system should be based on the quality evaluation of qualitative peer review and integrate quantitative bibliometric indicators, thereby combining both subjective and objective assessment.

Key words: Paper scoring, OpenReview, Cited frequency, Open peer review