中国科技期刊研究 ›› 2022, Vol. 33 ›› Issue (3): 305-310. doi: 10.11946/cjstp.202108030607

• 质量建设 • 上一篇    下一篇

OA期刊质量评价实证研究——双盲与开放评审模式对比分析

贺颖(), 徐小然()   

  1. 天津师范大学管理学院,天津市西青区宾水西道393号 300387
  • 收稿日期:2021-08-03 修回日期:2022-02-22 出版日期:2022-03-15 发布日期:2022-04-15
  • 通讯作者: 徐小然 E-mail:heying@tjnu.edu.cn;x17852002283@163.com
  • 作者简介:贺颖(ORCID:0000-0002-3631-0508),博士,教授,E-mail: heying@tjnu.edu.cn
  • 基金资助:
    国家社会科学基金一般项目“基于区块链的科学论文开放式同行评议质量控制研究”(20BTQ084)

Empirical study on quality evaluation of OA journals:A comparative analysis of double-blind and open review modes

HE Ying(), XU Xiaoran()   

  1. Management School of Tianjin Normal University, 393 West Binshui Road, Xiqing District, Tianjin 300387, China
  • Received:2021-08-03 Revised:2022-02-22 Online:2022-03-15 Published:2022-04-15
  • Contact: XU Xiaoran E-mail:heying@tjnu.edu.cn;x17852002283@163.com

摘要:

【目的】 对采用双盲评审模式和开放评审模式的OA期刊质量进行实证研究,验证“学术”与“开放”评价指标对OA期刊的评价效力以及“开放”指标的重要性。【方法】 将OA期刊的质量评价分为两个维度,即以引文分析为基础的传统“学术质量评价维度”和以开放程度为标准的“开放质量评价维度”;采用交叉列联分析和ROC(Receiver Operating Characteristic)曲线方法,使用双盲评审和开放评审模式的OA期刊数据来验证两个质量评价维度的关联及有效性。【结果】 双盲评审模式的学术质量与开放程度的关联很弱,开放评审模式学术质量与开放程度接近中度相关;不论是从双盲评审还是开放评审模式角度出发,开放评价维度都比学术评价维度更能有效评价OA期刊质量。【结论】 验证了DOAJ收录OA期刊的学术质量;OA期刊质量评价的“学术”和“开放”两个维度是互促与双赢的关系;开放评审模式可增加学术评价维度和开放评价维度的关联性;开放维度对OA期刊质量评价更有效。

关键词: OA期刊质量, 双盲评审, 开放评审, DOAJ Seal, 学术评价

Abstract:

[Purposes] This paper aims to conduct an empirical study on the quality of OA journals adopting double-blind review and open review and test the effectiveness of academic quality and opening quality in the evaluation of OA journals and the importance of opening quality. [Methods] The quality of OA journals was evaluated from the citation analysis-based academic quality and opening prevalence-based opening quality. The crosstab analysis and receiver operating characteristic (ROC) analysis were employed to explore the effectiveness of academic quality evaluation and opening quality evaluation in the assessment of OA journals adopting the two peer review modes and their correlation. [Findings] For journals employing the double-blind peer review, the academic quality was in weak correlation with opening prevalence. However, as for journals adopting the open peer review, the academic quality was in moderate correlation with the opening prevalence. In summary, the opening quality was more effective for the assessment of OA journals adopting double-blind review or open review than the academic quality. [Conclusions] We also carried out an empirical analysis based on the journals’ quality data obtained by web crawling. The results show that quality of the articles included in Directory of Open Access Journals (DOAJ) is assured. Moreover, academic quality evaluation is in a win-win relationship with the opening quality evaluation in the assessment of OA journals. The correlation between academic quality evaluation result and opening quality evaluation result is stronger for OA journals adopting open review and opening quality evaluation is more effective than academic quality evaluation in the assessment of OA journal quality.

Key words: OA journal quality, Double-blind peer review, Open peer review, DOAJ Seal, Academic evaluation