Observer variation in radiological assessment of pulmonary vasculature

Observer error in the interpretation of the pulmonary vasculature was studied. Ten experts commented on the presence of pulmonary venous and arterial hypertension, plethora, oligemia, thromboembolic disease, and the size of the pulmonary artery segment and aorta in 100 pairs (posteroanterior and lateral views) of chest radiographs. Apart from the age of the patient, no clinical data were provided. The observers, comprising 5 radiologists, 4 cardiologists, and 1 thoracic surgeon, viewed each pair of radiographs on 2 separate occasions. The 2000 assessments, each of which included 7 categories, were analyzed with the aid of a computer to determine the inter and intra individual variation and an attempt was made to assess the reliability of the diagnostic criteria. The greatest accuracy in diagnosis was shown in the category of thromboembolic disease, followed by that of pulmonary venous hypertension, whereas the assessment of pulmonary arterial hypertension, particularly when associated with the Eisenmenger syndrome, was the poorest. The diagnosis of oligemia was twice as accurate when a right to left shunt was present than when this was absent. The radiographic signs of pulmonary thromboembolic disease are highly diagnostic of the condition but are not always recognized on the plain chest film. The percentage overdiagnosis in all categories ranged from 5.6 to 21.1% (mean 9.6%) and that of underdiagnosis from 8.4 to 17.6% (mean 12.9%). With the exception of 2 of them, all observers were biased towards underdiagnosis. There was some correlation between the accuracy and consistency of each observer. The 5 clinicians were more accurate in all categories than the radiologists, though the difference was not always statistically significant. Accuracy of diagnosis did not correlate well with the age of the patient, the diagnosis, or the severity of the disease.