Abstract:

Purposes To conduct a cross-sectional evaluation of the AI-generated content （AIGC） detection capabilities of iThenticate， Wanfang Wencha， CNKI， and Jianziyuan AIGC detection tools on medical abstracts， aiming to provide a reference for the appropriate application of such tools. Methods A total of 518 randomly selected review article abstracts served as the original text group. Corresponding AI-polished and AI-generated text groups were created. The three sets of texts were assessed by the four detection tools for AIGC writing identification. Detection results were recorded， and accuracy， consistency， sensitivity， and specificity were calculated. Findings The results showed significant differences among the tools in terms of result presentation and decision thresholds. iThenticate exhibited limited applicability for Chinese texts， while domestic AIGC detection tools demonstrated higher detection rates for fully AI-generated texts but showed notable false positives and false negatives for AI-polished texts， with low consistency and sensitivity. Conclusions AIGC detection can be employed as an auxiliary technical tool for risk alert and manuscript triage， and should be integrated with author disclosure policies， editorial judgment， and industry standards to collectively uphold academic integrity and publishing order.

Key words: Artificial intelligence-generated content, Medical journals, Academic misconduct

摘要：

目的 对iThenticate、万方文察、知网、鉴字源AIGC检测工具在医学摘要中的识别能力进行横向评估，为同行合理应用AIGC检测工具提供参考依据。 方法 将随机选取的518篇综述文献摘要作为原文本组，并构建相应的AI润色组和AI写作组；通过上述4种检测工具对3组文本进行AIGC写作判别；录入检测结果并计算检测的准确率、一致性、灵敏度和特异度。 结果 各检测工具在结果呈现与判定阈值上差异显著。iThenticate对中文文本适用性有限；国产AIGC检测工具对AI写作检出率较高，但对AI润色文本误判与漏判突出，一致性与灵敏度较低。 结论 AIGC检测可作为一种辅助性技术手段，用于风险提示与审稿分流，需与作者披露、编辑判断及行业规范协同使用，以共同维护学术诚信与出版秩序。

关键词: 人工智能生成内容, 医学期刊, 学术不端