Development and Validation of a N.O.V. (New, Original and Valid) Tool for Assessing the Quality of Individual Patient Data Meta-Analysis
ABSTRACT OF THE THESIS
Evidence based Dentistry and Patient Centered Care are the two paradigms which influence the process of clinical decision making in modern dental care. Although they are complementary means to improve quality, they may seem at odds with each other. In the last decade, a need has emerged to examine the intersection of these two paradigms. Meta-analyses are a hallmark of evidence based dentistry as they succeed in showing statistically significant results by combining the results from individual studies. However, they are limited when the individual studies are heterogeneous. Meta-regression may help investigate this heterogeneity, but its limited ability to identify which patient features are related to the size of treatment effect is answered by using an individual patient data approach. Poor reporting of individual patient data meta-analysis diminishes its value to clinicians, policy makers, and other users. Our goal is to develop and validate a tool to assess the quality of an individual patient data meta-analysis.
We develop a tool based on the literature available in the field of Individual patient data meta-analysis. A modification of the PRISMA is done without altering its intent of use. In total, twelve Individual patient data meta-analyses in oral and maxillofacial medicine are identified to validate the tool and to establish its reliability. This is carried out by two independent readers who are standardized and trained to avoid misinterpretation of the Tool items. A Pearson's r coefficient is calculated to test inter-rater; intra-rater reliability and criterion validity.
A 9 item tool with a minimum of 3 criteria per item is constructed. The lowest score possible per question is 1 and the highest score possible per question is 4. The minimum total score possible is 9 and the maximum total score possible is 36. The new instrument has high content validity by virtue of its construction process. Testing the tool for criterion validity yields a Pearson's r of 0.957. The tool has construct validity based on the evidence of Test content, Internal structure, Response process, Relations to other variables, and Consequences of testing. It is also reliable with a Pearson's r of 0.927 for inter-rater reliability and a Pearson's r of 0.972 and 0.904 for intra-rater reliability.
Our study led to the development of a novel instrument specifically designed for assessing the quality of individual patient data meta-analysis. The tool scores for the 12 papers quantified the degree to which each paper satisfied certain criteria that are established in the literature as determinant factors of highest quality meta-analysis in general, and individual patient data meta-analysis in particular. The tool has significant implications for clinicians and researchers. We recognize the need for further testing of the Tool to increase its validity. New evidence in areas of methodology of IPDMA may call for a need to update the Tool.