Experience matters in point-of-care ultrasound
By Erik L. Ridley, AuntMinnie staff writer
May 19, 2014 -- Experienced sonologists had significantly higher sensitivity for diagnosing appendicitis with point-of-care ultrasound than sonologists with less experience, in a study from Mount Sinai School of Medicine. Either way, though, it's important not to rely only on point-of-care ultrasound to rule out the condition.
In the prospective study of 150 patients, experienced sonologists' sensitivity was nearly 30% higher than that of their less experienced colleagues.
"To minimize the possibility of errors, sonologists should avoid ruling out appendicitis based on [point-of-care ultrasound] results alone," said Dr. James Tsung from the department of emergency medicine. "So you need the clinical picture, and if there's uncertainty, certainly proceed with radiology imaging."
He presented the research during a scientific session at the recent American Institute of Ultrasound in Medicine (AIUM) annual meeting.
The experience effect
While it's well-known that ultrasound is an operator-dependent imaging modality, the effect of operator experience on point-of-care ultrasound hasn't yet been studied, according to Tsung.
In medicine, misdiagnosis-related errors are much more common than medication errors and can lead to poor patient outcomes. These types of errors can be minimized, however, by understanding the relationship between operator experience and a test's performance characteristics, he said.
With that in mind, the Mount Sinai team sought to evaluate the effect of operator experience on the sensitivity and specificity of point-of-care ultrasound in a prospective study of 150 children.
For inclusion in the study, patients had to be 21 years or younger, have abdominal pain with nausea and/or vomiting, and require imaging or laboratory evaluation for suspected appendicitis. Patients were excluded if they required immediate resuscitation, had prior imaging for suspected appendicitis, or had known inflammatory bowel disease.
Point-of-care ultrasound exams were considered positive for appendicitis based on standard sonographic definitions for appendicitis, while negative results included a normal appendix finding and also nondiagnostic studies. For the purposes of the study, the gold standards were operating-room/pathology reports for patients who required surgical operations, and a three-week phone follow-up for nonoperative patients.
Experienced sonologists enrolled more than 25 patients in the study and had diagnosed appendicitis using point-of-care ultrasound prior to the study test, while novice sonologists enrolled fewer than 25 patients and hadn't diagnosed appendicitis yet using point-of-care ultrasound.
The researchers then stratified the test performance characteristics by novice versus experienced sonologists, analyzing the relationship between operator experience, prevalence of appendicitis, and the rate of nondiagnostic scans.
Of the 150 patients who received point-of-care ultrasound, 61 (40.6%) exams were performed by an experienced sonologist and 89 (59.3%) were performed by a novice. Patients went on to receive either follow-up radiology ultrasound or CT; those with positive imaging findings went on to the operating room, while the rest were admitted or discharged.
There was an overall appendicitis prevalence rate of 33.3% in the study, which is in line with prior literature for ultrasound and appendicitis. No missed cases were discovered at the three-week phone follow-up, and there were no negative laparotomies in the operative patients.
Higher sensitivity
The 61 studies performed by the experienced sonologists included 48 negative and 13 positive exams, while the 89 studies handled by the novice sonologists included 67 negative and 22 positive exams.
| Sensitivity and specificity of point-of-care ultrasound | ||
| Sensitivity | Specificity | |
| Overall point-of-care ultrasound (150 patients) | 60% | 94% |
| Experienced sonologists (63 patients) | 80% | 98% |
| Novice sonologists (89 patients) | 51.4% | 93% |
| Radiology ultrasound (117 patients) | 62.5% | 99.3% |
The overall sensitivity and specificity for point-of-care ultrasound is in line with the literature, Tsung said.
"If you look at the spread between sensitivity [for experienced and novice sonologists], you've got like a 28 [percentage point] spread, whereas the spread between novice and experienced in specificity is much smaller, about five [points]" he said. "If you look at radiology ultrasound, they had a relatively low sensitivity relative to what's in the literature, but their specificity was excellent."
Tsung noted that point-of-care ultrasound preceded the radiology ultrasound study, an order that will naturally bump up the specificity of the radiology ultrasound study. In addition, radiology residents performed radiology ultrasound at their institution, which is why sensitivity was lower than would be expected.
"A lot of the residents just weren't comfortable with the scan," he said.
| Additional point-of-care ultrasound results | ||
| Nondiagnostic studies | Appendicitis prevalence | |
| Overall point-of-care ultrasound | 69% | 33.3% |
| Experienced sonologists | 67% | 24.6% |
| Novice sonologists | 71% | 39.3% |
| Radiology ultrasound | 59% | 37.6% |
"What [the appendicitis prevalence numbers] suggest is that the patients the novices tended to enroll [in the study] probably had more apparent appendicitis," he said.
Based on the differences between the two sonologist groups, the researchers concluded that operator experience had a greater effect on sensitivity to rule out appendicitis compared with specificity.
"Our ability to rule out pathology is more operator-dependent than specificity," he said.
Tsung acknowledged a number of limitations to the research; for example, it was a single-center study, relied on a convenience sample, and utilized a small sample size for subgroup analysis, he said.
