Author Insights: Beware of Overestimating Biomarker Validity

John P. A. Ioannidis, MD, DSc, of Stanford University, Calif, and the University of Ioannina School of Medicine in Greece, advises against giving too much weight to biomarkers that have not been thoroughly validated. (Image: Stanford School of Medicine Office of Communication & Public Affairs)

Researchers and clinicians often use such biomarkers as the presence of a gene variant, the amount of an enzyme in the blood, or a physical trait measured in imaging studies as indicators of an individual’s risk of developing a disease, disease prognosis, or response to treatment. Many clinical trials, for example, use a biomarker to indicate if an experimental treatment is helping patients because a more definitive measure of a treatment’s efficacy (such as whether the patient experiences a heart attack, develops cancer, or dies) would require a study lasting at least several years. But not all biomarkers are equally useful, and a literature review presented today in JAMA warns against giving too much weight to biomarkers that have not been thoroughly validated.

An analysis by John P. A. Ioannidis, MD, DSc, chief of the Stanford Prevention Research Center in California and a professor at the University of Ioannina School of Medicine in Greece, and Orestis A. Panagiotou, MD (also of the University of Ioannina School of Medicine), found that highly cited studies that are often used to justify further research on a biomarker or its use in clinical practice frequently provide an estimate of the biomarker’s usefulness that is greater than is subsequently found in large studies looking at the same biomarker.

Ioannidis explains why it is important to keep in mind that new biomarkers that do not have considerable evidence supporting their utility should be regarded with caution.

news@JAMA: What should researchers who are investigating novel biomarkers that they think provide useful information about a disease or its treatment take from your findings?

Dr Ioannidis: The key message is that results that seem spectacular are very interesting, but researchers need to wait for further validation from independent research teams and large-scale evidence. It does not mean they should not publish the results or defend them, but there is a chance they have found something that ultimately does not get replicated.

news@JAMA: What about investigators who may use a highly cited paper to validate their research proposals?

Dr Ioannidis: Do not cite what seems to be the most impressive paper showing an association. Try to have a balanced view of the evidence and design your research accordingly to strengthen or refute what has been published.

news@JAMA: What about physicians who consider using the results from a highly cited paper to improve diagnosis and treatment strategies for patients?

Dr Ioannidis: Clinicians should keep in touch with the literature to the extent they have time to do so. They should not feel they have to move biomarkers immediately into their practice. They can wait for the evidence to crystallize.

news@JAMA: What about patients who may hear of new biomarkers and who want them measured to improve their care?

Dr Ioannidis: If biomarkers are introduced into clinical practice and they end up not giving the information they are suppose to provide, errors of risk assessment can occur. It is important to avoid using biomarkers in clinical practice unless we can show they are valid and not misleading—because once they are in, we are stuck spending money on them and gaining very little information.

Categories: Evidence-Based Medicine, Quality of Care, Statistics and Research Methods