Objective To elaborate on the statistical analysis methods for evaluating the accuracy of imaging diagnostic tests in a multiple-reader multiple-case (MRMC) design through formula derivation and real cases. Methods This study consisted of two parts: theoretical derivation and a real case study. The theoretical part discussed in detail the principles and procedures of MRMC statistical analysis methods, particularly the Obuchowski-Rockette (OR) and Dorfman-Berbaum-Metz (DBM) methods. The real case included 100 subjects, of whom 67 had disease. Four readers interpreted all the cases based on both traditional film imaging methods and digital imaging methods. OR and DBM methods were employed for data analysis. Results The real case showed that the OR and DBM methods had a high degree of consistency, with only slight differences in the confidence intervals. Conclusion It is recommended to use the OR and DBM methods for the statistical analysis of imaging diagnostic test accuracy, ensuring that the impact of reader factors on the evaluation results is fully considered. The results from the OR and DBM methods are relatively similar; when applying these methods in practice, one should consider the specific characteristics of the data and the research design to choose the appropriate analysis method. Besides, there are still challenges when applying the OR and DBM methods, such as software implementation and missing data handling, which require further exploration.
ObjectiveTo explore two methods of sample size estimation in multi-reader multi-case study of radiological diagnostic test and realize them by software. MethodsDemonstration programs were conducted in R software using the Van Dyke dataset, calculating combinations of readers and cases using the OR and DBM methods. These serve as pilot test results for multi-reader multi-case studies, providing a reference for parameter settings in subsequent formal experiments. ResultsWhen the effect size was 0.044, 6 readers and 247 cases could yield 0.80 power; while with an effect size of 0.088, only 6 readers and 44 cases were needed to reach 0.85 power. The sample sizes calculated using the OR method and the DBM method were consistent, and the same sample size calculation results could be obtained through conversion between the two methods. ConclusionFor the estimation of sample size in multi-reader multi-case studies, R software provides a convenient and mature software package for sample size estimation using multi-reader multi-case designs in radiological diagnostic tests, thereby offering a reference for selecting appropriate sample size estimation and statistical analysis methods in radiological diagnostic tests.