Objective To systematically review the accuracy and consistency of large language models (LLMs) in assessing risk of bias in analytical studies. Methods The cohort and case-control studies related to COVID-19 based on the team's published systematic review of clinical characteristics of COVID-19 were included. Two researchers independently screened the studies, extracted data, and assessed risk of bias of the included studies with the LLM-based BiasBee model (version Non-RCT) used for automated evaluation. Kappa statistics and score differences were used to analyze the agreement between LLM and human evaluations, with subgroup analysis for Chinese and English studies. Results A total of 210 studies were included. Meta-analysis showed that LLM scores were generally higher than those of human evaluators, particularly in representativeness of exposed cohorts (△=0.764) and selection of external controls (△=0.109). Kappa analysis indicated slight agreement in items such as exposure assessment (κ=0.059) and adequacy of follow-up (κ=0.093), while showing significant discrepancies in more subjective items, such as control selection (κ=−0.112) and non-response rate (κ=−0.115). Subgroup analysis revealed higher scoring consistency for LLMs in English-language studies compared to that of Chinese-language studies. Conclusion LLMs demonstrate potential in risk of bias assessment; however, notable differences remain in more subjective tasks. Future research should focus on optimizing prompt engineering and model fine-tuning to enhance LLM accuracy and consistency in complex tasks.
Our team proposed and constructed an Expert-knowledge and Data-driven Comprehensive Evaluation Model of Chinese Patent Medicine (EDCEM-CPM) using the machine learning algorithm. This model could improve the system of the comprehensive evaluation of the Chinese patent medicine in technology and provide measurement tools for Chinese patent medicine according to its characteristics. The model evaluates the multi-dimensional value of Chinese patent medicine by data pre-treatment, clustering algorithms, and data training steps, such as automatic learning weighting. This evaluation model is already in practice. In this paper, we introduced the establishment of the model with the calculation process for reference.
Objective To summarize and analyze the characteristics, advantages and disadvantages of the current governance framework for public health emergencies in China. Methods The CNKI, VIP, WanFang Data, CBM and PubMed databases were electronically searched to collect studies on the management of major infectious disease outbreaks in China from inception to April 15, 2023. The basic information and governance elements included in the study were extracted and analyzed. Results A total of 30 studies were included, and the time of issuance was from 2020 to 2022. Most of the studies were on COVID-19, focusing on the governance framework of big data governance, holistic governance, and multi-agent collaborative governance. The governance elements were mainly concentrated in three aspects: governance subject, governance cycle and institutional guarantee. The governance entities were concentrated on multi-agent collaborative governance, with the governance cycle mainly focused on in process governance, and the basic guarantee is a multiple guarantee with information technology big data as the main body. Conclusion The governance body of China's major infectious disease epidemic management framework has transitioned from a single entity to a multi entity collaborative governance. While increasing prewarning governance, attention should also be paid to governance during the post recovery period. In terms of system, comprehensive guarantees such as epidemic public opinion control system guarantees, privacy security guarantees, and psychological counseling guarantees should be added.
Objective To analyze the current research status, characteristics and development trends of traditional medicine-related clinical trials registration, and to provide ideas and directions for further development of traditional medicine clinical trials. Methods The International Traditional Medicine Clinical Trial Registry (ITMCTR) database was searched by computer from inception to June 30, 2024, with unlimited trial registration status, to collect all the clinical trials on traditional medicine, and analyze the basic information of the trials, the diseases studied and the interventions. Results A total of 4 349 clinical trials related to traditional medicine were included, with the number of registrations peaking in the second half of 2020, and showing a steady upward trend after 2023. The trial sponsors of the study covered 9 countries and a total of 34 provinces/autonomous regions/municipalities in China, led by Beijing, Shanghai, Guangdong, Sichuan, and Zhejiang provinces, accounting for 69.72% of the total. The financial support for the studies was dominated by local government funds in various provinces and cities, accounting for 29.66%. Disease types studied were mainly circulatory system diseases, musculoskeletal system or connective tissue diseases, and tumor diseases, accounting for 29.91% of the total. A total of 3 751 (86.3%) clinical trials were interventional studies, of which randomized parallel control was predominant, and 213 large-sample studies with a sample size of more than 1 000 cases were included. A total of 20 types of interventions were involved, of which 1 114 (29.86%) clinical trials utilized oral prescription of herbal medicine interventions. Conclusion Clinical trial enrollment in traditional medicine has increased overall, but with significant geographic unevenness. Oral herbal soup/granule intervention studies are the mainstream hotspots. It is recommended to strengthen international cooperation, enrich the types of interventions, refine the trial design, and raise the awareness of researchers about the registration of high-quality traditional medicine clinical trials.