Appendix A
ACR SURVEY METHODS AND ANALYSIS
The 2003 Survey was similar to its predecessor, the American College of Radiology’s (ACR’s) 1995 Survey of Radiologists and Radiation Oncologists (Deitch et al., 1997), but incorporated important improvements throughout the survey process. These ranged from more thorough canvassing of all ACR leadership in order to identify issues of importance and ascertain priorities among them, through use of a multifaceted “tailored design method” (Dillman, 2000) to maximize the response rate, to use of an expanded and more intensive array of steps to improve data quality.
The questionnaire for the 2003 Survey consisted of 36 items; many items in turn consisted of multiple subitems. Questionnaire items and topics were elicited from two rounds of canvassing ACR physician leaders and staff leaders, winnowed according to priorities indicated by top leadership, and pretested in two large pretests conducted in autumn 2002, with refinements made after each pretest.
The survey sample, a stratified random sample composed of four strata, was taken primarily from the American Medical Association’s (AMA’s) Physician Masterfile, a reasonably complete listing of all allopathic physicians in the United States, whether or not AMA members. The sample from the Masterfile consisted of a 16 percent sample of all those self-designated in the Masterfile as vascular/interventional radiologists, an 8 percent sample of all other radiologists, and an 8 percent sample of nuclear medicine specialists. The sample included residents, fellows, and retirees, not merely posttraining professionally active physicians, and it included physicians whether or not the Masterfile had usable addresses for them. The Masterfile sample was obtained from Medical Marketing Service, Inc. (Wood Dale, IL), the commercial firm designated by the AMA to provide Masterfile data, in January 2003. In addition, the sample included 92 osteopathic radiologists, selected at random by the American Osteopathic College of Radiology (AOCR) from its membership. Based on information supplied by the AOCR, this was an approximately 6.7 percent sample of all osteopathic radiologists in the United States, including non-AOCR members.
In March 2003, the ACR contractor, the Center for Survey Research (CSR) of the University of Virginia, mailed the survey. Nonrespondents were sent up to four remailings as necessary, at approximately monthly intervals. In addition, to boost the response rate: first-class stamps (not metered postage) were used on all outgoing and return envelopes; the survey was publicized in ACR hard-copy and electronic newsletters and those of other radiology organizations; the third remailing was conducted by U.S. Postal Service Priority Mail, which uses a large, attention-getting red, white, and blue envelope; nonrespondents for whom we had telephone numbers were telephoned after the third remailing (with a message left if not reachable after two calls) to urge them to complete the survey; and the third and fourth remailing had a handwritten note urging completion of the survey. The last remailing took place in mid-July; acceptance of responses ended a month later.
As in previous ACR surveys, among nuclear medicine specialists, the ACR was interested only in those who had major ties to radiology; this concept of a major tie to radiology was operationalized as holding American Board of Radiology (ABR) certification and/or being a member of the ACR (Sunshine et al., 2002). On this basis, approximately two-thirds of the original sample of nuclear medicine specialists were omitted from consideration.
The total sample of interest, which was composed of the four strata of interventionalists, all other allopathic radiologists, osteopathic radiologists, and nuclear medicine specialists of interest, consisted of 3,090 physicians. From these, 1,924 usably complete responses were received. In addition, not in the form of completed questionnaires, the ACR received information that 21 addressees were deceased, 6 were no longer practicing in the United States, and 6 were not radiologists. The response rate was thus (1,924+6)/(3,090−21−6)=63 percent.
Responses were weighted so that the weighted statistics would be representative of the answers that would have been received if all physicians in the United States in the four strata had been surveyed and had responded. The weighting process has been described previously (Sunshine et al., 2002). To begin, logistic regression analysis was employed to determine how many different sets of weights were to be used in each of the four strata. For the 2,743 physicians in the “all other allopathic radiologists” stratum, the analysis showed that ACR membership and age had statistically significant effects on the response rate, while sex, geographic region, and listing in the Masterfile as a “radiologist,” “diagnostic radiologist,” or “radiology subspecialist” did not. Accordingly, 10 weighting categories, based on whether or not a physician was an ACR member and his/her age, were used, and responses in each category were weighted by the reciprocal of the category’s response rate. A similar logistic analysis of the 202 interventionalists in the sample resulted in two weighting categories, based on whether or not the physician was an ACR member. Because logistic regression showed no statistically significant effect, only one weighting category was used for the nuclear medicine specialists of interest and one for the osteopathic radiologists. After all responses in each weighting category were given a weight equal to the reciprocal of the response rate for that category, these weights were multiplied by the reciprocal of the sampling rate to complete the process of making responses representative of the entire U.S. population of radiologists. For example, if a weighting category had a response rate of 65 percent and it was part of a stratum that had been sampled at the general 8 percent sampling rate, then all responses in that weighting category were given a weight of (1/0.65)×(1/0.08)=19.23.
Data Quality Improvement
Every survey has some deficient data—that is, missing items, responses not in accordance with directions given by the questionnaire, and responses that are inconsistent or have other problems. The leading tool to minimize data deficiencies in this survey was the designation of the 12 items on the questionnaire judged most crucial as “core questions.” When questionnaires were returned, CSR checked that these 12 items were indeed answered, and made three designated consistency checks involving them. If there were any problems with the core items, CSR telephoned the respondent to obtain the missing response(s) and/or resolve the consistency problems.
During the data entry process, CSR spot checked entries against the paper questionnaires and found an error rate of less than 0.1 percent. Judging this error rate satisfactory, the data were not double entered.
Data used in this report have been additionally cleaned and edited to further minimize deficiencies. An example of items with relatively extensive cleaning and editing is as follows: For two questions about how radiologists spend their time, answers to subparts were supposed to total to 100 percent. Actual totals were computed, and it was found that in the vast majority of cases in which the entries did not total to 100 percent, the total was slightly below 100 percent. Consequently, if the recorded percentages totaled 95 to 99, all recorded percentages were checked against the paper questionnaire and any errors corrected. The data for all respondents were then edited using an algorithm the ACR has long used with items that are supposed to sum to 100 percent: recorded percentages are summed. If the sum is 80 percent to 125 percent, each percentage is divided by the sum, which makes the revised percentages total to 100 percent. If the sum is <80 percent or >125 percent, the responses are deemed too deficient to use and all responses are set to missing.
REFERENCES
Deitch CH, Chan WC, Sunshine JH, Shaffer KA. 1997. Profile of U.S. radiologists at Middecade: Overview of findings from the 1995 survey of radiologists. Radiology 202(1):69–77.
Dillman DA. 2000. Mail and Internet Surveys: The Tailored Design Method. 2nd ed. New York: Wiley. Pp. 150–153.
Sunshine JH, Cypel YS, Schepps B. 2002. Diagnostic radiologists in 2000: Basic characteristics, practices, and issues related to the radiologist shortage. American Journal of Roentgenology 178(2):291–301.
ANALYSES AND REPORTS ON RADIOLOGISTS PERFORMING MAMMOGRAPHY
The American College of Radiology provided the following tables to the Institute of Medicine Committee on Improving Mammography Quality Standards. The following list of column and row headings for each table reflects the type of information provided to the Committee. Actual data is omitted, but is available on request.
TABLE I Number of Radiologists with Various Breast-Imaging-Related Characteristics
Rows:
-
All posttraining professionally active radiologists
-
Radiologists who interpret any mammograms (number of mammograms >0)
-
Radiologists with a fellowship in breast imaging
-
ACR’s 2003 Survey of Radiologists)
-
Radiologists who designated breast imaging as their secondary subspecialty
-
Radiologists who spend more than 30 percent of time in breast imaging
-
Radiologists who spend more than 50 percent of time in breast imaging
-
Radiologists who interpret less than 480 mammograms per year
-
Radiologists who interpret at least 480 mammograms per year
-
Radiologists who interpret at least 1,000 mammograms per year
-
Radiologists who interpret at least 2,000 mammograms per year
-
Radiologists who interpret at least 5,000 mammograms per year
-
Radiologists who do any nonmammo breast imaging (ultrasound biopsy, etc.)
-
Radiologists who do any other breast imaging, but no mammograms
Columns:
-
Unweighted number of responses
-
Weighted number=number of radiologists in the United States who meet the definition, with standard deviation
-
Weighted percentage of all radiologists, with standard error
TABLE II Combinations of Breast-Imaging-Related Characteristics
Rows: Same as Table I
Columns: Same as B through N
Note: Each cell indicates the percentage (and standard error) of those in the row who also meet the column definition. For example, this table will give the percentage of those who say breast imaging is their primary specialty who interpreted 2,000 mammograms a year, the percentage of those who did a breast imaging fellowship that now spend at least 30 percent of their clinical work time doing breast imaging, etc.
TABLE III Further Information About the Breast Imaging Activity of Those in Each Category, A Through N
TABLE IIIa Further Information on Mammography Activity and Other Breast Imaging Activity, by Radiologists’ Breast-Imaging-Related Characteristics
Rows: Same as Table I, definitions A through N
Columns:
-
Unweighted number of responses (Repeat from Table I)
-
Weighted number of radiologists (Repeat from Table I)
-
Weighted percentage of all radiologists (Repeat from Table I)
-
Percentage who interpret any mammograms
-
Weighted average number of mammograms for those who interpret any
-
25th percentile, 50th percentile, and 75th percentile of number of mammograms for those who interpret any
-
Overall average number of mammograms (not only for those who interpret any)
TABLE IIIb Further Information on Nonmammography Breast Imaging Activity, by Radiologists’ Breast-Imaging-Related Characteristics
Rows: Same as Table I, definitions A through N
Columns:
-
Percentage doing any nonmammography breast imaging
-
Average number of types (of the ones listed below) of nonmammography breast imaging done
-
Percentage who do each of the following types of nonmammography breast imaging, with standard error
-
ultrasound-guided breast biopsy
-
stereotactic breast biopsy
-
localizations for surgical breast biopsy
-
fine needle aspiration (FNAC)
-
computer aided detection (CAD)
-
full-field digital mammography
-
breast magnetic resonance imaging (MRI)
-
TABLE IV Geographic Variation
Rows:
National total
4 Census regions
1=Northeast
2=Midwest
3=South
4=West
9 Census divisions
1=New England
2=Mid-Atlantic
3=East North Central
4=West North Central
5=South Atlantic
6=East South Central
7=West South Central
8=Mountain
9=Pacific
Columns:
-
Percentage of all radiologists in the area who interpret any mammograms, with standard error
-
For those who interpret any mammograms, average number of mammograms
-
25th, 50th, and 75th percentiles of number of mammograms, for those who interpret any
-
Overall average number of mammograms (not only for those who interpret any)
-
Number of radiologists interpreting mammograms per 10,000 women age 40 and older in area
-
Percentage of all radiologists in the area who do any nonmammography breast imaging
TABLE V Information by Degree of Urbanness of Location
Rows: Same as Table I, definitions A through N
Columns:
For each of the following degrees of urbanness:
-
All locations
-
Large metro main city
-
Large metro suburb
-
Small metro main city
-
Small metro suburb
-
Nonmetro
Each of the following columns:
-
Percentage of radiologists who interpret any mammograms in each location type, with standard error
-
Average number of mammograms for those who interpret any mammograms, with standard error
-
Number of radiologists who interpret mammograms, per 10,000 women age 40 or older
TABLE VI Age Distribution
TABLE VIa Information by Age
Rows: Same as Table I, definitions A through N
Columns:
For each of the following age categories:
-
All ages
-
Ages <45
-
Ages 45–54
-
Ages 55–64
-
Ages 65+
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
-
Average number of mammograms for those who interpret any mammograms, with standard error
TABLE VIb Number and Percentage of Radiologists Who Interpret Mammograms and Mammography Volume, by Age
Rows:
Each of the following age categories:
-
All ages
-
Ages <45
-
Ages 45–54
-
Ages 55–64
-
Ages 65+
Columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
-
Average mammograms by those who interpret any mammograms, with standard error
-
25th, 50th, and 75th percentile of number of mammograms, for those who interpret any
TABLE VII Demographics
TABLE VIIa Number and Percentage of Radiologists Who Interpret Mammograms, by Practice Type
Rows: Same as Table I, definitions A through N
Columns:
For each practice type:
-
All practice types
-
Academic practice
-
Nonacademic multispecialty practice
-
Nonacademic private radiology practice
-
Solo practice
-
Nonacademic government practice
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
TABLE VIIb Number and Percentage of Radiologists Who Interpret Mammograms, by Site(s) Served by Practice
Rows: Same as Table I, definitions A through N
Columns:
For each type of practice setting in which radiologist works, or types of settings the radiologist’s practice serves:
-
All settings
-
Hospitals only
-
Nonhospital sites only
-
Both
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
TABLE VIIc Number and Percentage of Radiologists Who Interpret Mammograms Overall, for Females, for Those Who Work Full-Time, and for Those Who Are Board Certified
Rows: Same as Table I, definitions A through N
Columns:
For each of the following:
-
All radiologists
-
Male versus female radiologists
-
Full-time versus part-time radiologists
-
Radiologist board certified or not
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
TABLE VIId Number and Percentage of Radiologists Interpreting Mammograms, by Practice Size
Rows: Same as Table I, definitions A through N
Columns:
For each of the following practice size categories:
-
All sizes
-
2 to 4
-
5 to 7
-
8 to 10
-
11 to 14
-
15 to 29
-
30 and more
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
TABLE VIIe Number and Percentage of Radiologists Interpreting Mammograms, by Self-Reported Enjoyment of Working as a Radiologist
Rows: Same as Table I, definitions A through N
Columns:
-
Average (mean) enjoyment score (Scores are: enjoy very much=2; enjoy somewhat=1; etc.), with standard error
For each of the following enjoyment scores in Q9:
-
All scores
-
Enjoy very much
-
Enjoy somewhat
-
Neither like nor dislike
-
Dislike somewhat or very much
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
TABLE VIII Percentage Performing Mammograms and Number of Mammograms Performed, by Gender
Rows: Same as Table I, definitions A through N
Columns:
For each of the following categories:
-
All
-
Male
-
Female
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
-
Average mammograms by those who interpret any mammograms, with standard error
-
Estimated total number of mammograms
TABLE IX Number and Percentage of Radiologists Who Want More or Fewer Hours of Work and Amount of Increase/Decrease in Hours Desired
Rows: Same as Table I, definitions A through N
Columns:
For each of the following categories:
-
Those who want their work and income increased
-
Those who want their work and income decreased
Each of the following columns:
-
Number of radiologists who interpret any mammograms
-
Percentage of radiologists who interpret any mammograms, with standard error
-
Average desired percentage change in workload
TABLE X Work Status by Gender and 5-Year Age Group
Rows:
Each of the following age categories:
-
<35
-
35–39
-
40–44
-
45–49
-
50–54
-
55–59
-
60–64
-
65–69
-
70–74
-
75+
Columns:
For each gender category:
-
All
-
Male
-
Female
And for each of the following work status categories:
-
In residency training
-
In fellowship training
-
Working full-time in radiology
-
Working part-time in radiology
-
Not working in radiology
The following column:
-
Estimated number (weighted count) of U.S. radiologists in this category
NOTE: This is the only table that uses all survey responses, including trainees and retirees.