Validation, use and interpretation

Print this article
Share this page:

Validation

How do new laboratory tests come into common use? New tests generally come first from research laboratories where scientists find a strong and consistent association between something they can measure, and a disease or group of diseases. The next step is to confirm that the test can distinguish between people with the disease and those without. Early studies typically compare people with the disease with a group of healthy controls. Often the test initially seems very promising and appears able to distinguish very well between the two groups and an article is published in a scientific or medical journal. However the great majority of potential new tests, although initially very promising, never make it from here into widespread clinical use. Why should this be the case?

Tests that are going to make it into common use need to fulfil the four criteria:

Does the test result reflect the reality in the patient's body?

Does the test result have a significant relationship with the disease in question?

Does the test result provide additional information that is not already available?

Is the test cost-effective? Is the cost of the test justified by its usefulness?

The most common reason tests fail is because they do not meet the first two criteria. This is because on further examination the test is found to be a poor discriminator between those with and without disease. When the test is compared in people with advanced disease and healthy people it may work well. However in real life in the clinic, the doctor is trying to distinguish between people with a variety of other diseases, often affecting the same organ, and people with the disease in question. To make it more difficult some people will have early stage disease (which is ideally when we want to detect it) rather than those with advanced stage disease who were used in the earlier comparisons. This is where most new tests fail. Either they are positive in too many people with other conditions (false positive) or negative in people with early stage disease (false negative) and sometimes they are both. There are almost no tests that can identify all people with a particular condition (100% sensitive) while being negative in all people without the condition (100% specific). Some genetic tests probably come closest to this ideal but even then both false positive and false negative genetic tests have been described. A “perfect” non-genetic pathology tests for common diseases is even harder to develop.

There are a variety of other reasons for test failing to make the grade including being too expensive, too difficult to perform, too unreliable or variable, too subject to interferences or requiring chemical reagents that are too toxic. The test may even be unnecessary because there are many conditions that doctors can diagnose and treat without needing to use any laboratory tests at all, so the new test may not be of any use or value.

Next »