Read about our work below, or watch Saul Weiner’s explanation at Stanford Medicine X:
In planning a patient’s care, a physician must not only know evidence-based treatment guidelines derived from research on groups of patients, but must decide whether these guidelines will result in effective care given the life circumstances, or context, for each patient. Our research program, using innovative direct observations of physicians, has documented substantial failures in physicians’ ability to provide patient-centered, or contextualized, care. We have explored the causes, costs, and consequences of contextual errors, have developed and tested educational interventions designed to improve physician skill at patient-centered clinical decision making, and have demonstrated that patient-centered decision making is associated with better outcomes for patients. This research program harnesses translational social scientific study of clinical medicine—from experiments with hypothetical vignettes to controlled field studies of covertly observed physician behavior to the ultimate study of patient health outcomes.
Combining our skills as a physician/health services researcher (Saul) and a decision psychologist/methodologist (Alan), we first developed a method for creating and validating standardized patient cases to assess two critical physician behaviors: obtaining contextual information about patients (e.g. their financial situation, social support, or competing responsibilities) during the clinical interview, and applying contextual information when planning treatment. Our standardized patients (SPs) are professional actors trained to consistently portray a patient presentation. The cases challenge physicians by introducing clues to potential contextual factors that require alternatives to standard treatment as well as potential medical factors that also require (different) alternatives. If physicians probe these clues, the actors reveal further information that either confirms or disconfirms the presence of a complicating contextual or medical factor, depending on which variation of the case the actor is portraying. Observation of whether the physician probes the clues, and whether the final plan documented in the medical record proposes a treatment that accommodates the contextual and/or medical factor, if present, measure the physician’s performance at patient-centered care planning. Cases were validated by presenting each variation in writing (with complete information) to a panel of expert physicians, all of whom concurred on the correct treatment for the variation; we published the method and case validations in the journal Medical Decision Making.
With a grant from the U.S. Department of Veteran Affairs, we set out to determine how well physicians would contextualize care in their own setting. In a ground-breaking experiment, we sent undercover SPs wearing hidden audio recorders to engage in 399 visits with 111 physicians at 14 sites, presenting variations of four cases based on commonly encountered problems in Chicago, and obtained the medical charts from the encounters (in all our studies, physicians consent to enroll in the study but do not know which patients are actors or wearing audio recorders). The study, published in Annals of Internal Medicine, revealed that contextual errors were pervasive. Physicians were less likely to probe contextual clues than medical clues, and overall planned correct treatment 73% of the time in uncomplicated variants, 38% of the time in the medically complicated variant, 22% of the time in the contextually complicated variant, and only 9% of the time when both medical and contextual complications were present. This study received considerable publicity, not only for its findings, but for being the largest study using covert observation of physicians by SPs, a method receiving increasing attention by policymakers as a way to promote quality and patient safety.
View a video about this study:
Using the transcripts and charts from that study, and supported by another VA grant, we have since characterized the pattern of errors (underuse of care more frequent than overuse), and the costs of errors ($174,000 across all visits, with contextual errors more costly on average than medical errors in these cases) in a paper in BMJ Quality & Safety. Perhaps most strikingly, only $8,745 of the total costs of error could have been discerned from review of medical records alone, without the benefit of direct observation of the physician’s decision making. In a paper in Journal of General Internal Medicine, we concluded that differences in contextualization could not be explained by a widely-used measure of patient centered communication, and we have since published our system for measuring contextualization online and a study of its inter-rater agreement in Medical Decision Making.
With a grant from the National Board of Medical Examiners Edward J. Stemmler Medical Education Research Fund, we developed a workshop designed to improve the contextualization skills of medical learners, and tested it with UIC medical students in a randomized controlled educational trial, using (known) SPs. The results, published in JAMA in 2010 and featured in the JAMA Report, found that the rate of successful planning for patients with contextual factors increased from 22% in the control group to 69% in the trained group, and that this increase was specific to skill in contextualization (rather than general diagnostic ability).
View the JAMA Report video on the study:
The study also provided us with an opportunity to compare the undercover SP method with the known SPs in publications in Simulation in Healthcare and The Joint Commission Journal on Quality and Patient Safety.
Funded by a third VA grant, we examined the impact of contextualization on real patients receiving real care. In this study, also published in Annals of Internal Medicine, 774 patients wore hidden audio recorders during visits with 139 resident physicians in clinics at two VA medical centers; 208 confirmed contextual factors were identified. Health outcomes improved in 71% of patients whose physicians contextualized care, as opposed to 46% of those whose physicians did not—a significant difference. Additional analyses of this study with similar data from a VA performance improvement project and audio recordings of phone calls to telephone health assistants employed by Accolade also showed that providers are four times more likely to contextualize care in response to contextual factors they identify by probing clues than those which the patient spontaneously reveals, a finding we published in BMJ Quality & Safety.
In 2016, we received a $1.1 million dollar VA grant to expand our work as a quality improvement initiative to four new facilities in the VA health care system. We will study how best to implement and optimize collecting and analyzing audio recorded visits and providing feedback to clinicians, with the goal of increasing contextualized care, improving health outcomes, and lowering costs.