The COSMIN-RoB checklist includes three sections with a total of 10 boxes, which is used to evaluate risk of bias of studies on content validity, internal structure, and other measurement properties. COSMIN classifies reliability, measurement error, criteria validity, hypothesis testing for construct validity, and responsiveness as other measurement properties, which primarily focus on the quality of the (sub)scale as a whole, rather than on the item level. Among the five measurement properties, reliability, measurement error and criteria validity are the most widely used in the studies. Therefore, this paper aims to interpret COSMIN-RoB checklist with examples to guide researchers to evaluate the risk of bias of the studies on reliability, measurement error and criteria validity of PROMs.
ObjectivesThis study aimed to evaluate the validity and reliability of the clinical practice guidelines (CPGs) applicability evaluation tool, a preliminary revised tool, by using it to appraise specific clinical guidelines.MethodsMedical staffs were sampled from relevant departments in domestic medical institutions to use tool to evaluate the two guidelines. Spearman-Brown coefficient of odd-even split-half method and Cronbach's alpha coefficient were used to evaluate the split-half reliability and internal consistency reliability. The convergent and discriminant validity were evaluated by correlation analysis and correlation coefficient comparison hypothesis test, and the structural validity was investigated by confirmatory factor analysis based on structural equation.ResultsThe split-half reliability of the evaluation tool was 0.86, and the Cronbach's coefficient of the whole tool and each dimension were greater than 0.7 for two guidelines. The success rates of tool convergent and discriminant validity calibration were 100%. In the second-order confirmatory factor analysis model, the χ2 and df were 3.38 and 2.46, the comparative fit index (CFI) were 0.872 and 0.974, the goodness of fit index (GFI) were 0.954 and 0.983, and the adjusted goodness of fit index (AGFI) were 0.846 and 0.959 for two guidelines respectively. Both standard root mean square residual (SRMR) and root mean square error of approximation (RMSEA) were less than 0.09. Both P values of RMSEA hypothesis test were greater than 0.05.ConclusionsThe evaluation scale is a valid and reliable instrument for assessing the applicability of CPGs, which should be further evaluated in practical applications in the future.
ObjectiveTo evaluate the reliability and validity of the Quality of Working Life Scale (QWL7-32). MethodsThe QWL7-32 scale was used to survey 487 drilling workers. The presence of chronic diseases was regarded as an effector for evaluating physical health, and the result of SCL-90 measurement was regarded as an effector for evaluating psychological health. The reliability and validity of the scale were statistically analyzed. ResultsThe results of the Pearson correlation coefficient was 0.713, the Cronbach's alpha coefficient was 0.920, and the Splithalf reliability coefficient was 0.942. The result of confirmatory factor analysis showed that the construct validity of scale was good, and the accumulative rate of 7 variances was 62.59%. The results of correlation analysis and t test showed that the validity of scale criterion was also good. In QWL7-32 scale, each dimension showed a good correlation with its relevant item but poor correlation with any other items. ConclusionThe QWL7-32 has a good reliability and validity.
ObjectiveTo compare the performance of 36-item short form health survey (SF-36) and World Health Organization quality of life-bref (WHOQOL-Bref) in assessing quality of life (QOL) in patients with pulmonary tuberculosis (TB). MethodsThe WHOQOL-Bref questionnaire and the SF-36 questionnaire were administered to patients with tuberculosis undergoing treatment from July to September 2013. The statistical methods of reliability analysis, factor analysis and the Pearson correlation coefficient analysis were used. ResultsIt showed that the WHOQOL-Bref and the SF-36 both had good reliability (Cronbach α=0.863 and 0.920, respectively). Constructive validity of the two instruments were checked by factor analysis and the Pearson correlation coefficient analysis, which indicated that both the two instruments had good validity. Among scales measuring similar concepts, many subscales of the SF-36 and the four domains of the WHOQOL-Bref unexpectedly had a fair correlation with one another. For example, the physical QOL, psychological QOL, and social relation QOL domains of the WHOQOL-Bref and physical functioning, mental health, and social functioning of the SF-36 were 0.482, 0.745, and 0.572, respectively. ConclusionThe WHOQOL-Bref and the SF-36 have an approximately equivalent practicability in assessing the quality of life in patients with TB.
The COSMIN community updated the COSMIN-RoB checklist on reliability and measurement error in 2021. The updated checklist can be applied to the assessment of all types of outcome measurement studies, including clinician-reported outcome measures (ClinPOMs), performance-basd outcome measurement instruments (PerFOMs), and laboratory values. In order to help readers better understand and apply the updated COSMIN-RoB checklist and provide methodological references for conducting systematic reviews of ClinPOMs, PerFOMs and laboratory values, this paper aimed to interpret the updated COSMIN-RoB checklist on reliability and measurement error studies.
ObjectiveTo formulate the Chinese version of Fear of Progression Questionnaire-Short Form/Caregiver Version (FoP-Q-SF/C) and examine the reliability and validity of the scale.MethodsA questionnaire survey of FoP-Q-SF/C was conducted among the caregivers of melanoma out-patients in West China Hospital of Sichuan University from June 2019 to March 2020. Convenient sampling method was adopted. The validity and reliability of the scale were analyzed.ResultsA total of 247 caregivers of melanoma out-patients were investigated by the FoP-Q-SF/C, and 101 valid questionnaires were finally collected. The Cronbach’s α of the FoP-Q-SF/C scale was 0.919, and the Guttman Split-Half coefficient was 0.906. Using exploratory factor analysis to extract 3 common factors, the cumulative explainable total variation was 73.964%. The model fit was as follows: chi-square/degree of freedom was 1.950, standardized root mean square residual was 0.067, goodness of fit index was 0.859, incremental fit index was 0.939, comparative fit index was 0.938, Tucker-Lewis index or non-normed fit index was 0.918, and the root-mean-square error of approximation was 0.097.ConclusionsThe FoP-Q-SF/C scale formulated in this study is divided into three dimensions, which has good reliability and validity, meanwhile, it is relatively simple and can be used to clinically screen melanoma caregivers’ FoP-Q-SF/C levels. However, the application of this scale in other diseases still needs further testing.
ObjectiveTo translate the King’s Brief Interstitial Lung Disease (K-BILD) to Chinese, so as to provide an well reliability and validity assessment instrument for health status of patients with interstitial lung disease.MethodsBrislin’s transition model, six expert’s panel and pre-survey were used for initial Chinese version of K-BILD. Items analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), internal consistency reliability and test-retest reliability were used for validity and reliability test with 122 respondents.ResultsTen-item Chinese version of K-BILD were proved to have great psychometric qualities, two factors were extracted by EFA, which could explain 63.35% of the total variance. Furthermore, the CFA demonstrates the fit indices of two-factors mode: χ2/df=0.797, RMSEA=0.000, NFI=0.848, IFI=1.048, CFI=1.000, TLI=1.071. Cronbach’s α and Guttman Split-half were 0.893 and 0.861, respectively. Besides, the test-retest reliability of the scale was 0.805.ConclusionThe Chinese version of K-BILD scale has good validity and reliability, which is applicable for health status assessment in patient with interstitial lung disease.
ObjectiveTo evaluate the reliability and validity of the instrument of clinical applicability of guidelines (version 2.0). MethodsThe experts of domestic medical institutions were investigated by questionnaire, and the instrument of clinical applicability of guidelines (version 2.0) were evaluated the guidelines for the diagnosis and treatment of tinea mantis and tinea pedis (revised edition 2017) and the guidelines for the diagnosis and treatment of cerebral hemorrhage in China (2019). Using Cronbach's α coefficient and Spearman-Brown coefficient to evaluate the inherent reliability and split-half reliability. The content validity was evaluated by calculating the content validity index of the item level and the adjusted Kappa value. The correlation coefficient between each item and the dimension and the hypothesis test were used to evaluate the convergent and discriminant validity. The structural validity was evaluated by using structural equation model to evaluate the structural validity of the tool. ResultsThe Cronbach's α coefficient and Spearman-Brown coefficient of the instrument of clinical applicability of guidelines (version 2.0) were both greater than 0.7, the content validity index (S-CVI/Ave) were more than 0.8, the success rates of convergent were 100%, and the success rates of discriminant validity calibration were 100% and 96%. In the second-order confirmatory factor analysis model, the χ2/ df were less than 3, the fitting index (CFI), the goodness of fit index (GFI) and the adjustment goodness of fit index (AGFI) were all greater than 0.9. The root mean square residual (RMR) were all less than 0.05, and approximate error root mean square (RMSEA) were less than 0.09. The P value of RESEA hypothesis test were more than 0.05. ConclusionThe instrument of clinical applicability of guidelines (version 2.0) has good reliability and validity, which can be further verified in practical application in the future.
ObjectiveTo compare and evaluate the discrimination, validity, and reliability of different data envelopment analysis (DEA) models for measuring the effectiveness of models by selecting different input and output indicators of the model.MethodsData from health statistical reports and pilot program of diagnosis-related groups of tertiary hospitals in Hubei Province from 2017 to 2018 were used to analyze the discrimination, content and structure validity, and reliability of the models. Six DEA models were established by enriching the details of input and output on the basis of the input and output indicators of the conventional DEA model of hospitals.ResultsFrom the view of discrimination, the results of all models were left-skewed, the cost-efficiency model had the lowest left-skewed degree (skewness coefficient: -0.14) and was the flattest (kurtosis coefficient: -1.02). From the view of structure validity, the results of the cost-efficiency model were positively correlated with total weights, outpatient visits, and inpatient visits (r=0.328, 0.329, 0.315; P<0.05). From the perspective of content validity, the interpretation of model was more consistent with theory of production after revision of input and output indicators. From the view of reliability, the cost efficiency model had the largest correlation coefficient between the data of 2017 and 2018 (r=0.880, P<0.05).ConclusionsAfter refining the input and output indicators of the DEA model, the discrimination, validity, and reliability of the model are higher, and the results are more reasonable. Using indicators such as discrimination, validity, and reliability can measure the effectiveness of the DEA model, and then optimize the model by selecting different input and output indicators.
ObjectiveTo develop a survey questionnaire on preferences and values regarding perineal injury prevention measures during pregnancy and conduct reliability and validity tests. MethodsCombining literature reviews, qualitative interviews, and expert consultations, we summarized key elements of perineal injury prevention during pregnancy and synthesized the best evidence. Through multiple discussions within the core working group, a survey questionnaire on preferences and values regarding perineal injury prevention measures during pregnancy was formulated. Using convenience sampling, pregnant women were recruited, and a pre-survey was conducted using the questionnaire. Pre-survey results were analyzed using item analysis and reliability and validity testing methods to validate and refine the questionnaire. ResultsThe questionnaire was compiled based on the theory of evidence-based decision-making. The initial version of the questionnaire was developed by combining systematic evaluation, network meta-analysis, and semi-structured interviews. The questionnaire was modified and improved through expert consultation, group discussion, and pre-investigation, which ensured that the questionnaire had good reliability, validity, and practicability. The Cronbach's α coefficient was 0.87, the split-half reliability was 0.71, and the content validity index was 0.97 of the survey questionnaire. ConclusionThe present version of the perineal injury preventive measures preference and values questionnaire has good reliability, validity, and practicability. It can serve as a valuable tool for investigating preferences and values related to perineal injury prevention during pregnancy.