Download Accuracy and Apparent Accuracy in Medical Testing - Exam | MATH 101 and more Exams Mathematics in PDF only on Docsity!
Accuracy and Apparent Accuracy in Medical Testing
Background Information
Today, testing for the presence or absence of a specific disease, medical condition, or illegal drug is common. The results of these tests are never as simple as they appear to be on many TV shows and movies. As patients become more and more critical consumers of medical information from their doctors, they must be aware of the quantitative and statistical reasoning that lurks behind the reported facts and figures. For example, if a medical test is reported as 99.9% accurate and you get a “positive” result, what is the chance that you have that medical condition? Medical researchers want to develop simpler, less expensive tests and screening tools for existing medical conditions. However, they do not want to sacrifice reliability when doing so. When a new test is developed, researchers need to compare its reliability with the existing “gold standard” test. In this activity, you can assume that the “gold standard” is perfect in detecting the medical condition under consideration. In this activity, you will explore the results of medical tests. The mathematics includes basic proportional reasoning, yet the real–world context is complex. Before beginning this activity, some definitions are needed.
Definitions
false positive (FP) : when a patient receives a positive test result for a disease but the patient does not have the disease false negative (FN): when a patient receives a negative test result for a disease when the patient actually does have the disease true positive (TP): when a patient receives a correct positive test result true negative (TN): when a patient receives a correct negative test result sensitivity: the probability that a test produces a positive test result when the patient does have the disease specificity: the probability that a test produces a negative test result when the patient does not have the disease
Part 1 ELISA, or Enzyme-Linked ImmunoSorbent Assay, is a biochemical technique to detect the presence of certain antibodies. An ELISA is developed to diagnose HIV infections. This new procedure will be compared to the “gold standard” test, the Western Blot, which is a time– consuming test for HIV. Determining the ELISA’s sensitivity and specificity for 20,000 patients: a. Assume 10,000 patients that tested positive by Western Blot (the gold standard) were tested with the new ELISA and 9990 were found to be positive. These are true positives. Similarly, ELISA was used to test serum from 10,000 patients who were found by Western Blot not to be infected with HIV. Of these HIV-negative patients, ELISA returned 9990 negative results ( true negatives ) and 10 positives ( false positives ). A two-way table is a very succinct way of organizing this information. Of the 20,000 patients tested with ELISA, fill in the number of patients in each of the four categories: true positive (TP), false positive (FP), false negative (FN), and true negative (TN)
Clinical Trials HIV–positive HIV–negative Totals
ELISA positive
(TP) (FP)
ELISA negative
(FN) (TN)
Totals
b. Fill in the cells for Totals. c. Of those patients who are HIV-positive, what percent were correctly diagnosed as HIV- positive? This is the sensitivity of ELISA. d. Refer to the definition box. Determine the specificity of ELISA. The high percents for sensitivity and specificity appear to indicate that the ELISA is an excellent test. Let’s investigate this a bit further in Part 2. .
Part 3 Let’s investigate two other situations where the tests still have a sensitivity and specificity of 99.9%. Investigating the ELISA’s false readings for 1,000,000 patients from a blood donor pool: Case 1: Assume we administer the ELISA to one million patients who are part of a real blood donor pool. Patients in this population have already been screened for HIV risk factors before they are even allowed to donate blood, so the prevalence of HIV in this population is closer to 0.1% (not the 1% used above). a. Create the appropriate two-way table. Remember : The sum of the row labeled Totals and the sum of the column labeled Totals should both be 1,000,000 (one million).
Blood Donor
Pool
HIV–positive HIV–negative Totals
ELISA positive
(TP) (FP)
ELISA negative
(FN) (TN)
Totals
(p) (n)
b. What percent of the positive readings are false? c. What percent of the negative readings are false? d. What is the only factor that has changed in this case, leading to these different percents?
Investigating the ELISA’s false readings for 1,000,000 patients from an “at-risk” pool: We will still assume that the test has a sensitivity and specificity of 99.9%. Case 2: The other case to examine is a drug–rehabilitation unit for I.V. drug users. In this case, the prevalence of HIV is 10%. a. Create the appropriate two-way table for this case.
I.V. Drug Users HIV–positive HIV–negative Totals
ELISA positive
(TP) (FP)
ELISA negative
(FN) (TN)
Totals
(p) (n)
b. What percent of the positive readings are false? c. What percent of the negative readings are false? d. What is the only factor that has changed in this scenario, which led to these different percents? Summary Questions:
1. Other than the general reliability of the test (measured by the sensitivity and specificity), what factor can greatly influence the number of false positive and false negative readings for any type of diagnostic test? 2. Is it possible to have a fairly effective screening tool (specificity and sensitivity values in the high 90%s) in which a large percent of those testing positive fail to have the disease? Explain.
Case 1 Tables: CAD positive
CAD
negative
CAD
positive
CAD
negative Stress positive (TP) (FP) Doctor Referre d Patients
Positive
Referrals (TP) (FP) Stress negative (FN) (TN)
Negative Referrals (FN) (TN) c. Of all patients who undergo exercise stress testing, only those who eventually also undergo the angiography are ever definitively diagnosed as having CAD or not having CAD. The entries in the right–hand table are often referred to as apparent true positive, apparent false positive, apparent false negative, and apparent true negative. These entries are called “apparent” because these patients are the only ones for which the presence or absence of CAD is known. 1. Using the values from the right–hand table, find the apparent sensitivity of exercise stress testing. How does it compare to the true sensitivity? 2. Using the values from the right–hand table, calculate the apparent specificity of exercise stress testing. How does it compare to the true specificity? Case 2: Let’s reverse the calculations. In the tables on the top of the next page, the right–hand side is filled in for you. This table represents the results of the definitive angiography on those patients who were referred by doctors after undergoing initial screening based on exercise stress testing. a. Verify that the apparent sensitivity of the screening tool is 75% and the apparent specificity is 85%. b. Assuming the referral rates are the same as in Case 1, Part 4. Work backward to find values for cells in the left hand table.
Case 2 Tables: CAD positive
CAD
negative
CAD
positive
CAD
negative Stress positive (TP) (FP) Doctor Referre d Patients
Positive
Referrals 200 (TP)
(FP)
Stress negative (FN) (TN)
Negative Referrals 66 (FN)
(TN)
c. Using the values from the left–hand table, find the true sensitivity and specificity of the screening tool. d. The differences between these true and apparent values is known as referral bias. Write a paragraph explaining the concept of referral bias to a friend who has not completed this activity.
