Press "Enter" to skip to content

The Evidence-to-Practice Gap: Teaching Clinicians Evidence-based Medicine (MtG)


Good morning thank you for coming to
today’s Medicine Mind the Gap lecture series featuring Dr. Scott Richardson
mine the gap is a lecture series that explores issues at the intersection of
research evidence and clinical practice areas in which conventional wisdom may
be contradicted by recent evidence from the role of advocacy organizations in
medical research and policy to off-label drug use to the effectiveness of
continuing medical education the seminar series aims to engage the NIH community
in thought-provoking discussions to tap to challenge what we think we know and
to think critically about our role in today’s research environment today Dr.
Richardson will discuss the various facets of evidence-based medicine what
it is why we need it and how to practice it he will engage in a discussion on how
clinicians can learn to practice evidence-based medicine even if they
have already completed their formal training he will also examine what it
would take to incorporate evidence-based medicine into the newer curricula of
medical schools illustrating how his teaching institution is doing this Dr.
Richardson is an academic general internist at the Georgia Health Sciences
University the University of Georgia Medical Partnership campus in Athens
Georgia where he is professor of medicine and campus associate dean for
curriculum his principal scholarly interests are in clinical epidemiology
evidence-based healthcare and medical education
he is co-author of the book evidence-based medicine how to practice
and teach it Dr. Richardson continues to work on the challenges of integrating
evidence into clinical decisions particularly in evidence-based clinical
diagnosis he also is working to incorporate evidence into the new
curriculum at his institution and in medical education at all levels please
join me in welcoming Dr. Scott Richardson thank you and good morning let’s see if
this works sure you’ll see about me a couple things
first of all I can’t read minds I’ve worked very hard over the years to learn
to do that but it hasn’t worked I’ve even gone so far as to turn my hair gray
and to lose a lot of it but still nothing gets in so if you have questions
I would be delighted to hear from them but you’re gonna have to flag me down
and say I have a question I have less than two hours of material so I’m hoping
we’ll fill whatever time with useful dialog and questions from the audience
the other thing is that I tend to move around a bit so you’ll have to get my
attention by actually moving your hand or something so I can see it so I hope
this won’t disturb you you’ll see I used visual metaphor from a popular culture
as I understand it although my late teenage your daughter has assured me I
don’t know anything about popular culture it’s okay
what we’ll tackle are these three topics in this order but as you’ll see we can
move back and forth so if your question comes out of sync it’s alright just go
ahead and ask it a first one to examine what we mean by evidence-based medicine
or practice or healthcare and think about why we need it and what it is
really that we are not doing maybe in some certain circumstances and what we
could be doing if we were to be using it and so to do this what I’d like to start
off with is if everyone could raise the right hand all right now keep it up in
the air if you would like your health system to routinely provide the
polyvalent pneumococcal vaccine not the new one but this one to high risk and
elderly persons to prevent pneumonia and death keep it up all right so that’s
most of you in the room I can’t see the webcast folks but guessing a lot of them
would of it their hands up as well and I’ll
pick up the thread of that story in just a little bit as we examine this I now
face the problem also of dealing with novice clinicians and we want to help
our first and second-year students before they head off to the wards to
understand the science that guides clinical decision making and so this
raises the additional question what we would want to do as a health system but
also what we should do educationally to help people understand how these
decisions get made and so start with what we mean I think of evidence-based
health care as the broadest of the terms evidence-based practice is the term that
has to do with the care of individual persons whereas healthcare might be the
care of all peoples in populations or population subgroups and of course
there’s evidence-based education evidence-based librarianship
evidence-based this and that I was recently sent a reprint that there’s now
evidence-based business management so the ideas of making well-informed
decisions are starting to appeal to people beyond healthcare and they’re
holding up healthcare as the example interesting and so what do I mean the
evidence-based it really has to do this idea that we conscientiously explicitly
and thoughtfully use current best evidence from clinical care research in
our work whether our work is actual clinical practice or in broader issues
of healthcare or education or elsewhere and it’s a recognition of the following
150 to 200 years ago the knowledge that was used for clinical
decisions was primarily of one sort clinical expertise those hard-won
lessons from doing it over and over and now at the now in this century we think
it’s more complicated there’s still clinical expertise and I want to be
clear I’m not arguing against the use of clinical expertise
you don’t want an expert healthcare provider you want somebody who’s been at
there what we’re in a sense arguing for is the
addition of consideration of all these other things the knowledge from the
studies of the human biology that’s why you exist right the net NIH there’s also
knowledge of studies of health systems and how translational research can
influence but in addition there’s the recognition in the last century and now
this entry of patients perspectives and how they can and should influence
decision-making and so forth their particular circumstances the health
systems are ethics so what we’re talking about is adding this one to the mix not
substituting okay and that’s the first point of confusion a lot of people say
how come here against clinical judgment I’ll preemptively say I’m not so if you
would like to talk more about that what I think we’re talking about is adding
considerations of results from clinical care research to the other knowledge
that we use and make judgments about and if you ask people like these researchers
did they did a survey of Americans and a few thousand were assembled and they
said should do you believe that health care services you receive should be
based on the current the best and most recent research available and it
probably won’t surprise you that most said yes
some said no and I’m not sure what these people were worried about but that’s
what we have it seems like a no-brainer in fact a lot of people when I start in
on this conversation say really it’s not already that way so why is it that we
need it let’s go a little bit more deeply and here are some of the issues
there’s an enormous amount of clinical care research that’s been published and
although people will complain about this I see this mostly as success success if
no other institution is success of the NIH and the enormous public and private
investment in research we have a lot of new knowledge now we have lots and lots
of though so much so that not all of it is
necessarily able to be understood by an individual in fact it cannot be done
over 28-thousand biomedical periodicals every year nobody can actually even read
them to say anything of understand and figure out how to use them so we have a
lot of research and importantly also its nature has changed dramatically whether
you think about individual randomized trials or systematic reviews in the
clinical care evidence or gene wide Association studies SNP research there’s
an enormous variety of types of research that wasn’t there 60 years ago so the
traditions of learning and keeping up to date that we had 60 years ago can be
expected to no longer apply sixty to a hundred years ago we had what
you might think of as the passive diffusion model which is to say new
knowledge as it was slowly made was published in biomedical journals and
talked about by the few who were expert in it and then that passively diffused
through the knowledge workers who needed to know that and what we’ve learned over
the years as I’ll show you is that model doesn’t work so well now particularly
when there’s so many voices and so many other things to consider here’s another
problem only a tiny fraction of this is sufficiently valid important enough and
directly applicable to care we’ll talk about applicability a bit more but
you’ve heard of this if you’ve invested as much as you have in translational
research you know that not every knowledge bit that we yield from biology
research is always directly applicable and you can think of a lot of what’s in
the research as communication from one group of scientists to others about
furthering science not so much from scientists to clinicians about major
advances in patient care so you’ve got this problem an enormous amount it’s
different than it used to be only a fraction of it is ready to hit the
street we also need it often now if you ask doctors how often they need it
they’ll say that they need new information two or
three times a week and that they usually get it from looking at textbooks or from
journal articles but if you follow them around and say do you need some now do
you need some now do you need some now it turns out that there’s been a wide
number of studies than they found a wide range but the average is something like
two bits of information for every three patients seen and what we tend to do is
we’re so busy that we either don’t pursue the questions or we ask experts
and why is that well the main reason is our usual sources of information don’t
work so well the usual passive diffusion model doesn’t work and what do I mean by
that well we’ve already talked about the overwhelming number in scope okay
imagine you need 1/2 a cup of water and to get it you have to go to Niagara
Falls and somehow get in there get your half a cup and get out too much for most
of us but in addition many of the forms of the usual sources have been shown to
be ineffective if you’ve heard from some of the people I think you’ve heard from
then you know that the standard didactic lecture not unlike what I’m doing here
is has been shown to not really change much in terms of knowledge and clinical
performance unless it motivates you to go off and learn and do something
differently so the main thing a large group presentation can do is to motivate
you to go off and learn and that’s useful but by itself it often doesn’t
change practice and there’s been over a hundred trials of cm II some showing
certain things effective some ineffective but the traditional forms
don’t work very well yet that’s what we require
so as time accumulates and we have all these questions that either don’t get
answered or get poorly answered our ignorant of current best care grows and
our clinical competence begins to shrivel and you may say well
see evidence for that assertion oh and did I leave out there’s too little time
how much time do you have reckon doctors have to learn new things
when people have done surveys they estimate among British general
practitioners they have half hour per week and then if you turn to individual
practitioners and say did you get your half hour last week only half or so half
which tells you maybe they’re not getting it very frequently and
importantly its half hour separate from work so here’s the thing
going back to that water analogy they need the knowledge at the point of care
they’re thirsty then but they don’t get it then they get it later like several
days later okay if this were water what would we do we build a plumbing system
before bottling where we’d go to a tap and out would come purified water
why isn’t knowledge like that why isn’t knowledge current best
knowledge delivered to the point of decision-making while working in a way
that it’s right there when we need it rather than quite a bit later yes sir ah
so there are products that are now starting to attempt to solve this
problem and up to date is one there’s several other point of care resources
but you can see why it’s only been in the last few years that people have said
oh we’ve got to redo the way we did knowledge deliver knowledge to the
knowledge workers so there has been research on this business about time and
its association with our quality of care there have been a number of studies 62
evaluations over time that were systematically reviewed in 2005 there
may be other newer evidence as well from John Ino DS we’ve learned that there are
several signals for updates every so often
sometimes as soon as a year after a systematic reviews published so I’d
reckon by now that’s quite out of date but I’d be surprised if it shows
something different because there was a strong and consistent negative
association between the year since graduation from medical school and our
knowledge as measured by tests our recommendations for diagnosis screening
and Prevention and our treatment decisions it turns out that obsolescence
is equal opportunity so that when you examine these data across subgroups us
medical schools international schools top the class bottle in the class
there’s a depressing similarity of the downward slope and the editorial was to
a company that says look it should be no surprise we need active methods to
deliver knowledge not passive methods and we can start to customize patients
care from evidence-based standards rather than from old textbooks going
back to the water analogy in a moment right here out-of-date think about this
there’s a three year or so time cycle from the time a textbook chapters
written to the time it’s published with some exceptions like the electronic
texts that are starting to come out if this were water we wouldn’t tolerate it
if you went over to the spicket and out came 30 year old water with the
accumulated detritus of 30 years of sitting in some tank what would we do
we’d raise a stink as they say and the government would intervene and we would
do something about it why isn’t that same societal outcry happening about the
out-of-date knowledge that we are delivering to our healthcare
professionals and unfortunately it’s often wrong so what’s that about let’s
have a look at these data you may have seen this earlier but let’s have a look
again these are data from Elliot ant-man were published in 92 where they examine
the number of treatment trial four treatments for acute myocardial
infarction and this is just one of them thrombolytics and yes I know we have
other newer treatments but work with me on this on the left most is the year
next in our the number of randomised trials and third column are the number
of patients who’ve cumulatively been randomized to them so that for instance
at this point in the mid early 1970s there were ten trials with just over
2,500 patients and so that in 1990 there had been 70 trials with those over
48,000 patients in the middle here this is a cumulative a systematic review with
a cumulative meta-analysis display of the effect size which is to say
mortality reduction from Trimble a tech therapy and here’s the line of no
difference where the ratio is one and this side favors the treatment the
mortality is lower and this side favors the control and I think you see that
after 23 patients there seemed to be a reduction but there was a wide
confidence interval that blob is the estimate of the effect size and that
line is the confidence interval and by the time you get to the tenth trial is
by then statistically significant showing them a reduction in mortality
and that was estimated around 1973 and for the next more than a decade 60 more
trials with nearly 45 of 46,000 patients entered into those trials and our net
effect has moved a little but not our qualitative belief that this treatment
saves lives now on the other side though let’s have a good look at the
recommendations of textbooks and experts as manifested by what they said roughly
the same years they checked sources either review articles or practice
guidelines or textbook chapters and count it up it was not mentioned
experimental etc and you’ll see that even though it was being used here it
was not mentioned in most places in 1969 it began to appear as mentioned but as
experimental and it wasn’t until this point
so roughly 1987 1990 that it was recommended as routine care that would
reduce mortality this demonstrated this article demonstrated that experts often
lag in their recommendation of life-saving therapy and the lag can be
measured in years sometimes decades and during that time there are millions of
people not getting this life-saving therapy and on the same in the same
article there were treatments that actually caused more harm and so there
were millions of patients receiving those potentially harmful therapies and
not just in this country all over the world so you have to start really asking
ourselves a society why is that what is it that we’re doing or not doing in
delivering knowledge to a knowledge intensive line of work in such a way
that were this far behind what we already know why can experts be wrong no
I have to say I have oops oh yes pick up the pneuma backstory why should we do
the pneumococcal vaccine really observational studies show its use was
associated with lower infections there’s certainly biological
plausibility you vaccinate against the strains you reduce it
there are definitely surrogate outcomes you can measure for instance that it’s
use is associated with increase in antibody titers and there are lots of
analogies there even cherished beliefs who wants to be again
prevention no one wants to really come out and say I’m against it and of course
there’s also these other things that are really more system issues low unit cost
each one doesn’t cost that much so people think it’s cheap even if the
aggregate cost is enormous if it has the property of being
measurable so that people could say oh yes check we did it we’re doing
preventive services and the experts recommend it practice guideline sure do
the manufacturers weren’t going to fight us and these extremist groups oh you
don’t know them by that okay well anyway these groups recommended the last one
gives you demerits if you don’t so you might quite reasonably ask what’s the
evidence that shows we should do this there at the time I look last summer
there’d been three systematic reviews for sort of 13 trials I don’t know if
their new trials since but in any case there might be three of those trials
were in healthy persons otherwise low risk but in high attack rate situations
army recruits South African Highlanders the excuse me the New Zealand
Highlanders the South African gold miners in other words you take a lot of
people have not been exposed you put them all live together in close quarters
and the infection sweeps through the camp and you can show that in that group
you can reduce mortality but in the group of the older and the high-risk
patients in whom it’s recommended there have been ten trials with over 24,000
subjects in all parts of the world and there’s no reduction in the poor
clinical outcomes total mortality pneumonia associated mortality
bacteremia and so forth so this gives us pause to wonder how it is now I’m not
sure whether we should give this stuff or not but it does strike me as a
remarkable that we universally note most developed nations recommend this
forget it but has it shown a difference not sure about that and this provides us
with what the learning science is called cognitive dissonance which is we think
of ourselves as making informed decisions and yet we’re making a
decision that seems uninformed by the information it should make us somewhat
uncomfortable and move us to action I have compassion for experts experts are
asked to do the nigh impossible they are asked to opine early when we have a new
thing that comes out we turn to experts and say well what do you think they’ve
often just seen the data just like we have we also don’t like our experts
wishy-washy we want them to be decisive to be definitive and so they’re asked to
have strong opinions formed early they also have the same problems of cognitive
overload little time too much information
they may also be involved in the research and it may or may not be free
of their own biases or conflicts of interest and of course experts being
human we have all the usual human frailties in addition it’s a recognized
phenomenon about the preliminary evidence it should work with definitive
evidence see we don’t really make a habit of doing trials of things we
already believe are stupid if we don’t think it’s gonna work that’s not what we
do trials and we take our best ideas where do those best ideas come from
animal studies in the early observational studies we take the things
we really think should work and try them out and that makes sense from the point
of view of when you’re about to embark on a trial but it means people already
have formed opinions about whether it should work
and so those formed opinions are hard to unlearn and we often confuse these two
it should work with it does work and many of the experts that I’ve talked to
focus on how something works rather than how well it works so they’ll tell me
about how the pneumonic Milana body alarms on to the receptor
interferes with the ligand and so forth and I’ll say well so how are patients on
this treatment are they better off – they have fewer joint exacerbations this
sort of thing and one of the problems is that up until recently many experts have
not used explicit systematic methods to gather the evidence and make sense of it
so experts are put in the jam but it does mean we can’t always rely on them
to know the truth of the matter but if we did if we did have good quality
information it might make a difference there have been a systematic review of
the treatment of cardiovascular disease from 2007 and they found that patients
who got the actual treatment based on trials at the optimal doses for the
right duration did better than those who didn’t get this treatment and that the
improvement was proportional to the number of treatments they got well
informed decisions might make a difference that’s part of the construct
here why are we doing this it’s because we can’t make well-informed decisions
without information and we’ve learned the hard way that not all information is
created equal and sometimes misinformation is worse I only got to go
back a little bit to the dub age i trial but also to the cast study there have
been a number of trials many of them sponsored by this institution where when
we finally put it to the test we learn the truth is different than the original
early studies and there is evidence that better information can lead to better
informed decisions and better outcome so now we face Morpheus choice do we take
the blue pill forget this leave this auditorium this morning and go back to
business as usual or do we take the red pill and remember
and try and make a difference in try and maybe teach ourselves or teach our
colleagues or junior colleagues in particular how to make a difference so
you’re feel free to walk out at any point but I’m gonna assume if you stay
you want to hear a little bit more about teaching so I’d be delighted to answer
questions as we go but let’s think about that how are we gonna get this taught
and the first part is to understand what the process is how it gets done and some
of the principles so in this slide I’ve tried to combine two kinds of things one
in white the process of evidence-based practice in which a patient dilemma or
if it’s evidence-based health care a population health care dilemma it could
be individual patients subgroups or whole populations leads us to ask
answerable questions that arise from that dilemma then we acquire evidence
that’s existing already about the answers to that appraise that evidence
for its credibility or validity importance and applicability and then
apply it now we were going with the A’s I think you can see but apply really
here means integrate this new knowledge with our existing knowledge of those
several kinds around the the human biology the patient preferences the
health systems and so forth and then we act on that and assess the impact of our
actions and this is meant to show sorry that’s a little dark meant to show that
since patients will vary those actions may vary let’s look at these two
principles here and these are the two sort of Biggie’s this one has a lot of
different names but I think the simplest idea and the least judgmental is that
evidence varies in terms of its robustness its risk for bias its risk
for error and so some things are more credible than others and a lot
this goes back to simple ideas of fairness you know if you’re if you watch
two boys throw snowballs and one of them throws a ball and hits a tree says that
see that tree I hit yeah that’s what I meant to hit and the other one says well
see that tree over there what’s the kind of crooked not coming out of it I’m
gonna hit that one throws it hits that one which one is more credible most of
our senses of fairness tell us the second one is more credible and that’s
the notion of pre-specified hypotheses in research and many of our ideas about
fairness and credibility come at the root from our understanding of fairness
what’s a fair test what would be falsifiable either way that this could
win that could win or we could turn out that they are equivalent this idea of
fairness though has some formalization from many methods so epidemiology both
classical and clinical biostatistics health economics lots of different
biological sciences so there are lots of more formal ways that we state
credibility validity and so forth but they at their root they have to do with
fairness then there’s importance which has mostly to do with how well something
works or how safe it is the size of the difference not it’s statistical
significance primarily but the clinical impact evidence can vary in its risk for
bias and error some people have said that that means there’s a hierarchy a
single hierarchy I’m not sure there’s a single hierarchy for evidence because
there’s lots of different questions and therefore the types of evidence you need
for those are different and therefore you might have multiple hierarchies
depending on your types of questions but anyway the idea is this one they vary
and we have to be able to sort it out the second principle is this when we’re
applying evidence evidence alone does not decide we have to combine with other
knowledge and values and more in that a bit but this basic notion is it’s
informing decisions not prescribing decisions that we’re
talking about so how are we gonna get started teaching this is Dumbledore
using his pensive to reflect and think and one of the things I hope that we’ll
realize is that early on in our reflection we often figure out you know
I’ve got to dust off my own skills so how you gonna get that done there’s some
courses to take I know Paris and others went to Colorado
to take a course there their courses in this country and elsewhere I’m
hopelessly biased in favor of taking a course because primarily we’re all so
busy where else do you get the time to concentrate on building a skill in an
area that’s so unfamiliar to you if you’re trying to do it a little bits and
pieces here and there but it can be learned without taking a course there
are course several books and I’m hopelessly biased in favor of several of
them not just the one I’m known to be a co-author on but I’ve contributed to
several so my encouraging you is to go off and learn from any of several what
I’m hopeful of is that you will dust off your own basic skills and start doing it
because that will help you in a few other areas another thing is to make
sure you have access to good evidence resources you’ve mentioned one
up-to-date but there are lots of others now starting to come to market and one
of them is none other than PubMed if you learn to use it well which you guys are
probably at least as familiar with as I am and this idea that we should learn to
optimize the flow of information from those resources is not just having them
a book on its shelf doesn’t share its knowledge it’s only when it’s in your
hand and open that it is working same with all the electronic resources and as
we build our own skill and actually do it and practice practice practice
another thing to think about is start building some allies and they may come
in all shapes and sizes of course you would think of your own colleagues in
your own practice but you might also think wow there’s a librarian and I need
her help or his help and i’ve got a clinical
cysts and I’ve got all these other players but there may be other allies
community groups healthcare journalists all sorts of people are now saying this
idea speaks to us and we want to work with people who want to do things I have
a lapel pin that I wear on my white coat and it’s in the shape of the yin and
yang symbol and you know those were meant to show you initially paradoxical
ideas that are turn out not to be paradoxical and one side it says trust
me I’m a doctor and on the other side it says ask me about the evidence because I
actually don’t think there paradoxic I want patients to trust me and to trust
that I will answer them about the evidence or we’ll look it up together
and I will help them interpret it it’s not a paradoxic statement it’s not I
don’t see it primarily as a challenge because I want them to feel like we’re
making well-informed decisions about their care now it’s not always easy or
fast we can come back to solving that with products like what you mentioned
but the idea is it’s part of our contract to make well-informed decisions
with our patients is to use current information now what do I mean by three
modes down here I’ve met a lot of teachers in evidence-based medicine a
lot of wonderful people I’m not gifted teachers and I’ve tried to learn from
each and every one of them one of the things I started doing several years ago
started watching how they teach and trying to categorize it into ways that
made sense to me and as I began to do it I noticed that although that I’d met
thousands and there were thousands of different individual styles that there
were three main methods or three main modes in which they talked and so I
thought to myself whoa what are those the first one is role modeling and this
is anyone who actually practices or actually does health care decisions at
the population level who uses evidence as part of their decision-making
and are seen by learners to do this so learners see that evidence and its use
that’s conscientious explicit injudicious is part of good patient care
or fill in population health or preventive care or whatever it is that
you do this idea being that evidence is not separate from it it’s right there
with everything out in addition they see who you do it with what you actually do
when you do it where you do it how you do it in other words they begin to see
through those pragmatics that it is actually possible to do that it’s not an
impossible pipe dream if you teach by doing they will learn by doing they
can’t learn by doing if you don’t teach by doing so this is the first one and
I’d actually wager that if we do lots of this then we have a certain moral
legitimacy when we ask them to learn to do the same if we don’t do this we’re being signaled okay the next one
is weaving or mixing or some other term that is when you’re teaching about
clinical practice or population health whatever you do that you integrate
evidence from clinical care research along with other knowledge you use in
how you teach so that learners see using that knowledge from clinical care
research that evidence as part of good clinical learning and by showing
explicitly how you make decisions in way this you can teach what I think of as
vinaigrettes now here it is separate the oil and the vinegar in the spice let me
give you an example as a general internist I’m often asked to teach in
the curriculum at my new school the stuff that nobody else wants to teach
and perhaps as a general internist I should look no better but I’m not afraid
of these I’m quite in fact fascinated by the problems of the recognition and
treatment of fluid problems in our patients and a lot of it comes down to
the combination of the physiology and it’s applied the applied physiology a
fluid movement across all these phases in our bodies with the evidence about
the different signs for fluid disorders and how accurate they are
and the evidence about the treatments for those fluid disorders so I take the
vinegar you’ll pardon me I hope of the physiology mix it with the oil of the
evidence from clinical care research add some spice with some cases that sort of
thing and we teach them together now remember if it’s wired together it’s
going to be fired together what happens now is that physiology is taught in the
first year when do they learn the evidence about the accuracy of these
findings if they learn it at all it’s in the third year they can’t remember those
things that way if you teach them together they will learn them together
no they may still have memory problems but at least you can hope that they will
wire it together in their memories and access it that way
as the scientists have taught us that we do the third one is what most people
think of that I mean when we say teaching evidence-based practice and
that is targeting specific skills in evidence-based practice and you know
whether it’s asking questions it’s anything about that cycle so I’d like to
have everyone raise their left hand and then keep it up if you’ve been to some
session where you thought this was happening where you were learning
something about evidence-based practice okaying lawyer you’re done if you have
it okay still that’s a lot of you where you’ve had something that you were
learning that you identified this and again what I’d say is if the other two
are happening if you see people role modeling if you see people weaving it
then you may be curious well how did you know that I’ll tell you a story I
occasionally teach on inpatient ending rounds and they’ve occasionally heard
before I get there that I’m interested in evidence-based medicine and so some
groups say ooh dr. Richardson yeah we’ve heard you like that stuff but we don’t
want to learn that this month we want to learn clinical medicine so I say okay
fine and I think their first shocked and we do it all month long we don’t call it
that we say oh we’re just trying to make sense of our patients test results which
of these is more reliable this clinical finding how do you gather it well how
accurate is it this treatment they seem to be having an adverse effect is that a
common one a less common one all of these things are ways to bring into
evidence from clinical care research into everyday decision-making without
necessarily the label but about halfway through the months the brighter teams
have figured me out and they say see what you’re doing you’re actually
teaching us clinical medicine using evidence and now we want to know how
you’re doing that when you said this test result was more reliable than that
one you seemed to pull out a table of numbers and those numbers spoke to you
in a way they don’t speak to us what do they tell you they don’t tell me how can
I learn to learn that way then you have them they want it and usually they’re
delighted to learn it if you show it’s direct relevance because why most people
arrive to me hating statistics already it’s not just neutral they’re in the
ground I’m in a hole where Roger said if you first find us if you find yourself
in a hole the first rule is to stop digging right so where are we in this
hole about statistics it’s often not that much about statistics and all if
it’s more interpreting the effect size or that sort of thing and my point of
this is if they see it as foreign to their careers they typically won’t learn
it if they see is directly relevant to their careers and you can show them how
it fits to the citizens are going to make they will happily learn it
particularly if it makes them faster and I hope to show you some of those ways
you have to start with their learning needs and then build towards what you
want to teach them and this is my reminder you know I showed you that kind
of round circle you can think of that as a whole pie a lot of people think that
to teach the whole thing all the time I don’t really think you do I think often
you can teach smaller slices and I’ll illustrate with just two because they’re
common but I don’t want you to get bogged down in the details one is this
idea of the probabilistic tradition or evidence-based diagnosis where there’s a
scale of probability from 0 to 1 if you’re a purist from 0 to 100% if you’re
a percent adjust I may have just made that word up
the idea being that it’s a way to represent our uncertainty about a
diagnosis whether it’s present not and that the this threshold approach
identifies that there is a probability above zero but not very high at which
we’re ready to accept that that diagnosis is not there even if it really
is there and being willing to take that error rate because usually the costs of
testing or the harm from testing is significant and similarly there’s a
threshold at this end where before we know for sure we are willing to treat it
what’s an example well you may know the story of herpes encephalitis that back
when I was an r1 I realize that’s ancient history although I have assured
my daughter there were Forks then when I was in r1 treatment for herpes
encephalitis was era C and the diagnostic test was brain biopsy and so
we didn’t really want to give era C if we didn’t have to and we sure didn’t
want to do that test if we didn’t have to so we had a pretty high threshold for
looking for it and a high threshold for treating for it now the diagnostic test
M R and the treatment intervenes acyclovir or some of the newer first
cousins so we’re much happier testing earlier at a lower threshold so that
test threshold has come down over time and the treatment threshold has also
come down because we’re willing to give acyclovir which is much safer than era C
even if we’re not entirely sure they have it yet so those concepts you
probably have a feel for even if you’re not sure where the numbers come from and
again I don’t want to get bogged down it but one thing’s it’s important I’ve
worked hard to do is a lot of people portray this probabilistic approach when
they’re teaching evidence-based medicine as the way to do diagnostic reasoning
but students have already learned five others they’ve learned the descriptive
traditions the criteria based for rheumatic conditions or for psychiatric
conditions they’ve learned a lot of the understand that a broken bone here
is different than a broken bone there so there’s an anatomic tradition and
there’s also pathophysiologic and biopsychosocial so we have to take peas
to add the probabilistic not substitute because they don’t want you to come and
take away what they already know they spend all this hard work learning that
and they’re going to defend themselves against you taking it but I don’t think
we have to adding evidence in its consideration doesn’t require us to
subtract the other knowledge in fact they’re probably better as doctors if
they can flawlessly oh did we go backward how fun now we went forward in
any case they’re probably better if they can flawlessly use all six and of course
all risk is in the context of realizing that sometimes we are doing analytic
thinking and sometimes we’re doing pattern recognition so putting
evidence-based diagnosis in the context of actual clinical diagnosis is easier
for most people to understand and accept rather than completely wiping the slate
clean and starting over for them so that’s a moderately large sized slice
now let’s tackle a smaller example and it’s from what I think of as the arc of
applicability here just this section how this is going to get done if you know
anything from the last 60 years of biologic research is that humans very
and they vary in many important ways and now we’re starting to understand why
they vary genetics epigenetics and lots of other explanations for some of it but
some of them may be more than that a big part though is that they vary in their
biology their psychology their sociology that define their circumstances as
broadly as you can think of it and the way they value and choose based on their
circumstances and those variations can be represented some of those variations
have to do with the representation of their baseline risk of bad outcomes from
a canoe and scaled without young units from low
to high so there could be biologic determinants social determinants of
health all these determinants of health there
are also some of these same things but maybe different things that could
determine their responsiveness to a treatment and still those same or others
could determine their vulnerability to adverse effects all of these scale
without units from low to high except this one from high to low and you’ll see
why now because you can create a cube that represents varying degrees from low
to high of risk responsiveness and vulnerability and start asking yourself
if you were the designer of a trial of a treatment which group would you want in
this trial if you wanted to maximize the chance to treat the find benefit a lot
of you would already go well we want people a high baseline risk so that
pretty frequently pretty soon we would start seeing people with a bad outcome
if the treatment didn’t work we’d also want people who are very responsive and
very low chance of having an adverse effect and so surprise surprise most
so-called efficacy trials are up in this corner and when we then try to broaden
the trials to see do they work in other groups of people those trials are often
more like this where we take high-end medium risk and we take high-end medium
responsiveness low and medium vulnerability and we accept them into
our trials clinical practice is quite a bit more like that
okay now we’re not really thinking of doing it in the absolutely low baseline
risk or impossible responsiveness or enormous lehigh vulnerability but most
of the time we’re trying to make judgments about that size from the
information generated from here and this issue of applicability
can be dealt with often at this conceptual semi-quantitative graphical
level without necessarily getting into the arithmetic of how this gets done so
if people understand the words and the concepts before they understand the
formula and the numbers that’s okay many people this is all they’re going to
get but many can go on to the numbers if they’ve understood this concept most of
judging applicability of results boils down to saying are we in this kind of
study this one or this one okay so that’s another small slice that
you could learn and then learn to teach as you begin to prepare your learners
for their work in evidence-based practice or healthcare and of course as
you can probably imagine I’m really recommending that you combine them you
learn each of these three modes role modeling weaving and targeting specific
skills and then teach in the natural context that you find yourself whether
it’s classroom or clinical settings and you adjust these to fit so that you’re
teaching what your learner’s need based on your strengths as well do you have
questions about this before we move on let’s pursue the problem of now how do
you prepare novice clinicians or basically senior undergraduates for
learning and practicing this way and that’s the dam I found myself in 2009
when I was asked to be the associate dean for curriculum in a way it’s people
basically saying okay you said some things about education now’s the time
for you to show us by putting your money where your mouth is
how are you really gonna do this and I should tell you that what we’ve
undertaken is not just about integrating evidence into the knowledge but
integrating much more broadly than that and much deeper than that and I want to
give you that context before we talk about how we’re integrating evidence so
here’s what we’re gonna tackle is showing an example I don’t think it’s
the example it is an a worked example from our work in Athens Georgia so if
you have a look at it and anytime this is our building it’s a 160 year old
mill building on the River Lea County River this right here is the mill race
where the water from the river goes under the building and it’s still there
and on rainy days the building howls with the noise of that going under
making large groups quite interesting actually in any case a lot of
distractions Athens a wonderful town University I’m told there’s a football
team there so we had to get our students attention and we were hired pretty much
in 2009 and in one year we were to open our door so in 2010 we had to have a
first year working curriculum and we’ve worked very hard to get this done and
yet it’s still a work in progress we’ve just finished our second year and we
took some time because it was night of 2010 to reflect on the prior 100 years
as to what we had learned from medical education if you were opening a medical
school in 1910 you’d have access to flex news report and his holding up of Johns
Hopkins as the model in which now has become almost enshrined this whole
business of the 2+2 curriculum 2 years of basic science and 2 years of clinical
rotations most people think of him as the person who made sure we had research
universities associated with medical schools and that’s true in addition he
was a big proponent of supervised door graded supervision on clinical
experiences that you couldn’t just lectured students then and push them out
with a degree they actually had to learn through doing and other things
attributed to him as well at the time though the shape of the medical schools
reflected these several beliefs that many people still hold these are
foundational beliefs that have been very tenacious in medical education and one
of them is that foundational subjects can be taught early and only once let’s
take a nap me we teach anatomy in the first year in
many places why do we only teach it once why is it that we overload our students
in the first year before they understand why they need it and never come back
after they’ve gone through the clinical rotations and go hey I want to become a
HS surgeon now I should know a lot more about chest anatomy than I did when I
went through Anatomy why do we teach it only once that there were that it was
good to inculcate general mental faculties that the study of disciplines
was primarily that it would stretch and strengthen your mind and these were
people who made sure we’d studied Latin and philosophy to stretch and strengthen
their mind and that also that to use knowledge students had to master its
methods of inquiry to use a beta-blocker you have to have done Studies on
beta-blocker receptors and ligands and we’ve learned that’s not exactly so but
anyway more on that and that also the the notion then was knowledge would
change slowly and that students would be able to integrate this knowledge all on
their own this is akin to thinking that you could go into a doughnut shop and
say I’d like a chocolate glazed doughnut and someone hands you a cup of flour
teaspoon of baking soda a tablespoon of cocoa and a heap of butter and say
there’s your doughnut well that’s what we’re doing right now in most medical
schools across the country is that me somehow that I’m doing Oh
somebody has a cellphone okay well as long as we’re not being signaled to
evacuate or something if that’s true you let me know and one of the biggest
problems this idea that transfer will happen automatically now what do i mean
by these last two words integration most people talk about horizontal versus
vertical but I think if integration is the deliberate putting together of any
knowledge anything that would guide decisions and that could be horizontal
disciplines that are taught in the same year or vertical the clinical and the
science and transfer is this idea of using knowledge acquired from one
context to solve different problems in other contexts and this is what the
learning scientists say is the holy grail of education they want people to
learn in one context and apply in another most of the learning science
studies have shown that we apply best in the context we learn it so for instance
divers have been checked on things as mundane as spelling words and they’ve
checked them on land and in sea and the ones they learn down in the deep they
spell better down in the deep than they do on land and the reverse the ones they
learn on land they spell better on land the knowledge we know in a classroom
setting we don’t necessarily know when we’re taking care of patients it doesn’t
transfer automatically again there’s this idea that these things will
passively and easily and automatically whereas the learning scientists the last
three decades have taught us that it takes a lot more active transport and
now that we have functional MRI and now that we have PET scans and other imaging
and better ways of measuring cognitive tasks they’ve been able to label the
glucose that shows us just exactly how much energy it takes to learn or relearn
things as fascinating that if you already have a belief it’s use less
energy than if something is counter to your belief and why is that it has to do
with neural commitment when we lay down a memory trace we’re putting tissue from
one cell to the next dendrites molecules were committing to certain memory traces
if we then find counter veiling arguments that make us change our mind
we have to Deacon at the D commit that and recommit something else and that
literally is a biologic contributor to resistance to change not only do we
psychologically want to defend the knowledge we spent so much time gaining
but we have to recommit neurons that’s hard for us all of us not just doctors
so here’s a first-year class they arrived very bright very eager and we
said we want to help you learn by engaging more in active learning that
there’ll be context for the content you do that we will have you learn in teams
like you’ll practice in teams later that we want you to build coherent
explanatory narratives not just memorize a lot of facts that are unrelated but
build explanatory narratives think about this if you were to walk up to a bunch
of kids on the street now and say who’s Professor Snape and Losey teach many of
them would know why because they’ve either seen the movie or they’ve read
the book and without meaning to learn just hearing the story just being
engaged by the story of these people they fought they come to care about they
learn a lot of things like who Professor Snape is and what he teaches it’s actually easier to understand
complex phenomenon when you see how all the pieces fit together you see how it
all works and you can see the pieces and their relationship to others so that you
can begin to name them and tell their story otherwise it’s a lot of facts to
be memorized separately we wanted to use cognitive scaffolding more on that if
you’d like of course this idea and this idea the idea that to become expert in
anything the studies that the expertise literature suggests as you probably know
that it takes over 10 years and 10,000 hours of deliberate practice to get good
at something and we wanted to integrate across many levels the issue about
active versus passive I’m here at the NIH I figure you can handle a biologic
analogy it’s like cardiac cycle diastolic learning is the passive
filling learning that many of us experience during undergraduate and
preclinical learning situations where systolic is more the active learning but
they do different things here you can acquire facts and you can identify what
you don’t know here you use the facts you have to organize them to use them
and by applying and transfer you lay down you commit new neurons to how we
use this knowledge so instead of being inert in our knowledge cupboards in our
brain we get practice opening those cupboards
and pulling it out and using it regularly enough so that now we can
transfer more easily more readily yes they do they do so the question has to
do with how they are doing in the testing and I want to show our sample
week and I’ll get back to your question about how are students learning this way
oh yes thank you I should have remembered we have doing ask it again
I’ll ask again I said I think this is a really good idea you know it’s
on-the-job learning if I wondered how the students do with testing because in
the end they care about their scores and placement when they graduate from
medical school they sure do thank you for your question and what I want to
show you is here’s our sample week so in the first year it looks like this in the
second year is very similar except the small group learning case based learning
happens in this time block they start in the first year with small group and they
have then large groups which you might call lecturers we have histology labs
anatomy labs community health projects clinical skills and so forth notice over
here it says weekly assessment and what this represents is a chance for students
to check their knowledge before they leave the week now takes the form of 30
multiple choice board style questions each week tests enhance learning the
learning scientists tell us means that if you have regular checks of your
knowledge alternating with cycles of learning more and relearning you do
better on final exams and tests so by the end of two years our students have
experienced whether it’s weekly assessments or module final exams over
3,000 nbme style multiple-choice questions that they’ve practiced
questions that they got credit for in the form of grades so there’s a certain
amount of facility that comes with doing it over and over
and being learning to apply your knowledge to assessments on a regular
basis people say are you teaching to the test
no the primary reason is tests enhance learning we’re helping them check their
knowledge a side effect of this is that by the time they hit their step one
they’ve done a lot of practice of this kind of testing and yes they’re nervous
like every other medical student all around the country is nervous right
about now because our second years are just have just finished their second
year and then the next month and a half they have to take step one I’ve yet to
meet a class of medical students who’s not nervous at this time are they any
more nervous I reckon not as a group individuals sure because humans vary but
I think there as well prepared as most groups as well let’s look at the other
stuff we’ve got a thing called essentials of doctoring or essentials of
clinical medicine and that’s where we do some learning about MS based practice a
thing called community health and I think called clinical skills more on
those in a moment I want you to show you that it’s a
composite thing and each week is organized around a week’s theme so
what’s an example well one week in the fall they have they start on Monday
morning with the case of a young man a senior on the football team who has an
injury to his shoulder and after that injury feels pain and he can’t move his
shoulder can’t move his arm and they realized to solve this problem they have
to learn about the anatomy of the shoulder and the anatomy of a thing
called the brachial plexus which is a complicated structure so that’s eight to
ten their very first large group is an anatomy presentation about this
complicated structure of the brachial plexus the next one is a histology
presentation about the histology of nerves the next day they learn about
action potentials and the physiology of nerves and their anatomy lab is the
anatomy of the shoulder in the brachial plexus and clinical skills is the exam
nation of the shoulder so that what we’re trying to do is integrate not just
by adjacency where these things are adjacent but that the weekly learning
objectives and all these things are different parts of the elephant so they
see all these pieces next to each other and they inform each other and because
we’re talking about various clinical decisions it allows us in this sort of
session to deal with the evidence about that kind of decision while we’re at it the cases are fairly
broad we’ve decided it’s not just the standard thing we’re going to include
from jeans to society all the determinants of health we can
throw in there there are authentic clinical predicaments and they unfold
over three days so in some schools problem-based learning is one case you
get everything on the first day and then you have to do follow-up in our
situation they get a part of it they discuss it they get another part discuss
it and so forth and more parts here and more parts there so there’s a sustained
mystery until Friday and that gives us a chance to learn between Monday and
Wednesday and between Wednesday and Friday new stuff that relates to the
decisions that are being made and we coach them on how to find knowledge so
we have a librarian who’s on our faculty and she teaches searching and she
teaches asking questions she teaches us she helps us teach appraisal and we have
sessions on the determinants of health fully biologic psychological
sociological all these educational notions on the side this is from a
presentation I gave recently to the College of Ed there’s a bunch of reasons
here educational science wise but these are boil down to the intermediate step
our surrogate marker of improving integration on the way to the
longer-term outcome of improving transfer so that’s how and why we teach
science in the cases and that provides in some of the context and we’ve
organized around a lots of doctoring tasks decisions problem-solving
conversations either you know a lot of times we say oh my doctor doesn’t know
how to talk to me and I wonder did they ever learn okay so we have sessions on
not only the easy ones the familiar ones how to break the bad news but even
talking about a nun diagnosis talking about the adverse effects of treatment
very specific and so then again you say well wow they we’ve just told
them here’s an adverse effects of treatment why not have the evidence
about the frequency of how often that adverse effects appear right then so
that when they need to use it is the time we bring the knowledge so that
that’s what we integration that it’s knowledge in time for their learning and
it comes from any discipline we’ve decided not to spend so much time
thinking about the root discipline that this comes from anatomy because we don’t
do this in language right you aren’t say you aren’t told that in
order to use the word quiz you have to learn its etymology you are simply
allowed to use the word and employ it the word pattern does it really matter
whether it’s Latin or Greek that it came from and it’s history over time we spend
a lot of time in medical education declaiming the origin of knowledge
rather than emphasize how that knowledge gets used in everyday practice a
tradition we should probably give up because our students recognize it
primarily as an elaborate two-year long hazing ritual that they then practice a
kind of intellectual bulimia on step one they disgorge and they don’t absorb and
they move on to the clinical years so then if you think of this way you could
organize yourself around these types of decisions and the types of evidence you
need if you’re going to talk about risk or harm talk about prognosis talk about
treatment or diagnosis and each of these have subgroups of K sub types and sub
categorizations but if there are four main areas and we’ve recognized that
those appear in a lot of our cases and that if we think about those skills we
talked about asking acquiring appraising applying and then acting and assessing
and then we add the third dimension of the content
either single patient groups of patients whole populations or what I think of as
future patients the skills for a lifelong learning if you map out that
three-dimensional grid then you can begin to say well actually there’s lots
of opportunity as long as we do this slicing rather than trying to teach the
whole thing we can reinforce in the cases in the science module the risk
from something so our very first week our very first case who I hope that
rising first years aren’t listening is a patient with sickle disease they don’t
know that they just see a young child with pain and we learn about the gene
associated and we learned about the risk of transmission and so forth and so
early on there are quantitative notions from genetics that are coming in and so
we begin understanding risk the quantitative expression of risk and how
to talk about it with our patients in the first week as they’re learning the
science of gene product and red cell making makeup and so forth and even
hemoglobin cooperativity or whatever it is that gives us that gorgeous s-shaped
curve is it cooperativity I may have lost track of that bit of biochemistry
in any case we try to weave it all together now for the stuff that needs
explicit teaching we’ve done some work in Community Health in the essentials of
clinical medicine and in the clinical skills part I don’t want to turn to
those in the first two years we talk about these skills with these headings
in those areas and they learn to appraise evidence in these types so that
they can find it and use it and make sense of it in addition in the community
health projects we’ve taken this idea of learning through doing instead of a just
a few lectures on Community Health in the first semester they work with one of
our partners in the community and they do a needs assessment and
the second semester they do a brief intervention targeted at those needs
first tier medical students working with the University Health that first year
was on sleep deprivation polypharmacy in the aging depression care this you may
know that Athens is home to a large creative community many of the musicians
but other types and there a lot of them are how what’s the polite term under or
uninsured and so we we provide they are provided care there
there’s also asthma at Head Start and this I think was geriatrics something I
can’t quite remember what that is now that I see it
the idea being that yes they have some introductory ideas but then they also
learned through doing and not least in clinical skills we have them as I
mentioned learn the examination how to do it but also learn stuff about how
accurate it is okay and it includes things like the delivering the messages
we were talking about are you familiar with this rationale clinical exam series
in the Journal of American Medical Association David Stein Merrill David
Sackett and Drummond Renia are among my many heroes and here’s why they’ve stuck
with since 1992 the deliberate collection and updating of what is known
about the accuracy of bits of bits and pieces of the clinical exam so history
or physical for the diagnosis of certain conditions it turns out for instance
that there are 43 different maneuvers for the diagnosis of chronic obstructive
pulmonary disease that have been described over the years three of them
have sufficient accuracy and reproducibility that we now teach our
students in those three because the others in even experienced hands aren’t
reproducible enough or accurate enough to be worth their time learning now when
we were talking earlier about speed here an example why do we teach eponym us
bits and pieces of the physical exam that have likelihood ratios of one make
no distinction between those with and without disease why do we make our
students learn them why do we insist on testing them in clinical skills the AH
skis and why do we insist on demonstrating them at the bedside is it
because we’re teachers of history is it because we’re related to those people
who may whose names are on there I would like to know why I want to know why we
can’t concentrate our learners attention on the few findings that matter and
really help us not only know how to gather them but how to interpret them
and how to use them in clinical medicine in a way that speeds them up as well as
makes them more accurate at diagnosis and so forth the bits of the history of
physical there are also other tests in that series so that’s how we put all
those pieces together into our curriculum and one of things that the
yes oh it may not be on speak into my
microphone now this this makes a lot of sense of I’ve always wondered though in
a curriculum like this how do you teach something that may not be amenable to
intermittent two or three hour sessions like immunology or genetics where
there’s a large amount of information and you know that it needs to be taught
sort of over a relatively short period of time because a lot of it may be new
and you know you may not work to teach two hours of immunology now and then two
hours the analogy in two months from there thank you for your question I
think it has several parts so how do we deal with the large groups so do we have
large groups and we clearly do and how do we teach new knowledge along with old
knowledge if you will and you know if you take the idea that we should be
teaching our best knowledge whether it’s new or old our most reliable most
understood we try to have that selection go in there and then the issue about
pacing is interesting I’ve found over the two years that at first many of our
scientists held the view that you appear to hold that you have to teach all these
things together and I think now some of them are starting to change that view
that if they see if they experience a case in which that knowledge is directly
relevant and then they hear that knowledge in the large group and they
see that case use that knowledge and so forth and they’re tested on it then they
can build that knowledge more longitudinally now I don’t mean once now
and then once a year from now we try to get some sequential nests out of it
but it isn’t necessarily so that in the only way to learn the next hour of
genetics is immediately following the first hour in fact many of our
scientists are coming to understand that if they go away and really learn those
things we just asked them in the first hour that a few days later or the next
week they can pick up the thread and move forward particularly if there’s a
case that makes that come alive so I take your
point yet what I see in our scientists is they’re seeing oh we should teach at
the pace of their understanding of my material which isn’t necessarily this
bead of our speech or the speed of our mouse clicks or how fast the slides can
go by I recently attended a lecture where in a half hour someone had 96
slides now to my way of thinking that’s really not about learning that’s the
kind of shock and awe campaign yes speak loudly we will hear you oh yeah okay
we’ll use the microphone oh that’s right for the webcast this gentleman’s
question I I might want to hear the evidence that that is the case I’m
thinking about a recently taught EKGs to first-year students and then in this
module based second-year students and most of the time was spent reminding
people they what they had previously learned so you know I mean there must be
a reason why in college we take a semester of chemistry and not an
occasional lecture in chemistry I I have to admit that I’m a skeptical that
someone could take a large body of scientific knowledge and learn that as
effectively episodically as they would without the constant reinforcement of
reminding what we were talking about yesterday well thank you for your
comment and I actually I suppose I did not bring the slide that shows you
exactly how coalesce these really are for instance when we’re in
musculoskeletal module it’s seven or eight weeks of learning a whole lot
about muscular anatomy and bone anatomy and those pieces come fairly fast and
furiously at the same time they’re also learning about bone physiology and
muscle physiology and bone and muscle energetics and metabolism do
you see what I mean so it’s not as separate as I must have somehow
communicated to you it’s not by weeks or years it’s really more that it’s not
necessarily all the same week because the pieces that go together for those
topics go together but the next week we pick up with quite a bit more anatomy in
genetics and so forth so there is a regionalization without necessarily a
total fragmentation if that helps you yes ma’am and I would be delighted to
stand here one day and tell you we have that evidence I think what we have is in
medical education a number of studies about problem-based learning that either
show no difference in their knowledge or some modest improvement in their
knowledge for regular forms of problem-based learning this is quite a
bit different from regular problem-based learning as I hope you see it’s quite a
bit modified and we’ve done what I think of as imported a lot of ideas from
general learning science about how to get this done from the general learning
science pieces of this have been tested and shown that they lead to better
knowledge but I don’t know that we’ve done this experiment where all these
pieces put together have been demonstrated to do better I had hoped to
randomize patient students to one campus or another but that got roundly rejected I’ll say had we been able to
randomize we might have been able to control for all those wonderful can
things we call confounding variables which means that their likelihood of
success in the traditional curriculum more traditional say Rick young in
Augusta versus our curriculum but nonetheless the measures of our
curriculum or of our students say they had the same entering MCAT school
the same entering GPAs the same a similar distribution of backgrounds as
those in Augusta and so there may be some comparability and I reckon we’ll
end up demonstrating that the knowledge measures are similar but I think you’ll
see that perhaps the skill measures will be different their ability to use
evidence in their everyday decision-making may be different but
even passing the board’s at a similar rate will be still an achievement of
this shows that it can work this way I have lots of people who are sure that it
can’t work this way primarily for the reason that it’s not the way they
learned and if that if we were if we that’s the kind of creationist view of
Education that we should do it the way they did it yeah question
yeah professor I studied at University of Missouri yes they use a problem-based
learning structure that’s I think very similar to yours where it’s each week is
based on a theme and there was some evidence collected there was a big bump
in the in the step one scores in the years following the adoption of that
curriculum I think there is some evidence that shows that this works I’d
also like to mention that this at Missouri this curriculum was created not
by MDS but by EDD Xand him phd’s in education who came in with different
biases hadn’t had a medical education and so we’re free of the biases that you
have somehow fortuitously escaped that everything has to be taught the way it
was in the 20th century well I appreciate you bringing that up I hope
everyone heard to two points in response and I think it connects we act there are
other stories right so we have one of our people that teaches with us came
from UTMB in Galveston and they had a similar experience when they went from
the traditional didactic to the problem oriented and problem-based approach with
small group of cases if you accept before after a
study evidence as useful they had for the several years before X and they had
for the several years after almost 1.3 X in terms of their board score so quite a
big jump and it was sustained it may interest you that I use some of these
same ideas in my prior hat my prior location when I overhauled the medicine
clerkship the internal medicine clerkship as a clerkship director at one
point and we went from four hours in a row of subspecialty lectures
can you imagine by the same person in the afternoon and did I leave out that
you had to go whether you’ve been uncalled the night before okay you can
imagine how lethal that was and our students scored on at that institution
as they entered Medical School at the fiftieth percentile and on step one at
the fiftieth percentile and no surprise on this shelf exam for the clerkship at
the 50th percentile but using many of these notions we overhauled the
clerkship used more active forms of learning did some of the same ideas and
for the next two years their shelf exam scores went to the 77th percentile I was
checking the shelf exam scores primarily because I was worried that it might go
down okay I didn’t really expect it to go up but it did go up and those data
are probably not going to be published for the following reason one of the
coolest things was we had no failures in that clerkship for academic reasons
whereas in prior years there had been several per year that couldn’t pass the
shelf exam nobody flunked for academic reasons and
it transferred their step ones there step two scores were higher than that
school had seen including one student who scored two points shy of the highest
possible score not happened before at their school and again no failures among
first time takers nobody really wants to own up to that my view of this
is that the evidence out there shows while incomplete and not necessarily
randomized evidence shows the same or better outcomes in terms of standardized
test measurements of knowledge but it is incomplete and I do look forward to
better more elaborate studies that control for more confounding in the
meantime if you knew something that would help even the weakest people in
the class succeed where they hadn’t succeeded before why wouldn’t we do it
particularly when you’re starting from scratch and you’re given the opportunity
the tabula rasa that may not be pronounced properly anyway that white
tablet to try and start from scratch and not have to reinvent it you have a
question sir yes so I’m aware of several though not in dentistry by the way I
should say there is a Center for evidence-based dentistry in the United
Kingdom and there is a movement worldwide for evidence-based dentistry
I’m not sure about dental school education doing this the whole piece
that we have here but I think there probably are I did some years back this
is going to be about four or five years ago now when I was at Oxford at the EBM
course there meet some people from one of the schools in the Netherlands in
dentistry and they were talking about how they had overhauled their curriculum
but I do not have enough details to be able to answer your question fully school you have to learn some technical
skills an examination and whatnot but you know well people have asked me how
I’ve succeeded in one in the things that I’ve done and usually my answer is
something like the combination of good luck opportunism
and sufficient naivete to believe that it might still be possible despite all
the wiser people who tell me it is not possible I can’t help but wonder I
wonder who’s tried it and I wonder how they’ve how it succeeded I suspect there
are schools out there trying some of these ideas I do think it’s true that
the each profession is likely to have different shapes of these ideas it’s
perhaps the difference between a method and a personal style you might have the
same method but with different shapes to fit the profession but I wonder how many
of these ideas there have to be you know why learn about the anatomy of tooth
enamel is it because it’s there or is it because you’re gonna have to make
decisions diagnostically and therapeutically about it and to the
extent that the decisions can help us organize what people learn so that it
would post that the knowledge is guiding that decision is brought to bear then I
think there’s a good chance it could affect but I just don’t know if enough
about dentistry to answer you you have a question or you’re okay yes so I think
one of the areas where the traditional curriculum really fails is teaching
kidney disease and you know physiology and yes I part of my job if directing a
kidney disease program is to try to improve outcomes in kidney disease in
the primary care setting and whenever I talked to physicians you know I sort of
say well one of the biggest obstacles is most physicians spend their entire
careers in recovery from the second year renal block and that always gets a
knowing laugh and I’m curious if you have any
anecdotal experience or more than that to comment on how this curriculum works
better at at the in kidney disease for as almost as a litmus test well thank
you for your question I’m not sure we have enough outcome data to tell you
about how well it works better does it work better I can tell you that
we’ve taken the same right well see I feel the same way often without
statistics in epidemiology like people hate it before I get to them why but
yeah so why why do they hate the kidney but the thing is a lot of people don’t
understand the enormous functional beauty of like the proximal tubules
cells they are amazing cells they do these amazing things and they’re
complicated and but they’re very function oriented right so there’s a
form kind of function beauty but it’s not always portrayed that way is it now
and I would say that the experience of many students is that renal physiology
cardiovascular physiology and pulmonary physiology are some of our most
difficult areas of physiology and the corresponding areas of path of
physiology and pathology are also quite difficult but we’ve done it we remain to
be seen how well we’ve done I was actually asked to give a non
nephrologist a kind of non expert view of managing patients with chronic kidney
disease what I think every doctor needs to know let me go back to my experience
as a clerkship director one of the big challenges I faced at that time was
overload curricular overload because I had these specialists saying that they
needed to know everything in kidney about same as angio proliferative and
memmer know proliferative and so forth and my third-year students couldn’t sort
acute kidney injury from chronic kidney disease and I was faced with the problem
that they were trying to teach the entire discipline of internal medicine
and nephrology in that one four-hour session it’s my only chance they would
say and I’ve got to get it in there well that’s not really going to happen
you can’t get it all in there I don’t know what all is but after the first
hour of four hours they have shut you down
if not sooner so as a consequence by trying to teach everything as far as I
could tell they learn close to nothing so I thought what if we built the other
way compared to learning nothing what if they learn one thing and then a
second thing and so forth so after hours of negotiation I finally pointed out
look for every hundred students at that Medical School who came 10 to 15 were
headed into careers in internal medicine let me say that another way 85 to 90
were not we needed to orient our clerkship to teach not internal medicine
for internists but what every doctor needed to know about internal medicine
even if you were a surgeon or pediatrician and if we did that it
wasn’t that hard wasn’t that long eventually they said here you know
you’re right acute kidney injury that’s pretty important chronic kidney disease
that’s also important maybe one or two other things and we focused it and again
the Shelf scores went up I was worried they might go down because we are
teaching fewer facts but although the number of facts floating in the air were
fewer they were a lot more coherent and I actually think that instead of the
standard five percent or less retention and transfer that most people get out of
most lectures most of the learning Sciences tell us that the retention and
transfer from active learning is more on the neighborhood of twenty five to fifty
percent now it’s still not a hundred I can’t tell you how many people I’ve
heard that say that but which are you putting your nickel on particularly if
it helps the people at the bottom of the class the people at the top they can
learn anything you could put them down in the center of armed conflict in
Kosovo or something and they’re still learning they’re like oh I didn’t know
any of this would you look at this but they’re gonna be okay what I’m much more
interested is the whole range of our learners making sure it’s
possible for all of them and when we did that in nephrology in internal medicine
the scores went up on those quizzes just as well
same thing here our students have learned renal physiology and
pathophysiology partly because this approach but now if you look at the
overview in many schools in the first year you have learning mostly lectures
about normal human structure and function and then in the second year
learning about human disease and then in the third year get to clinical Sciences
and what we’ve tried to introduce through the case based learning in our
approach is that in both years they’re learning pathophysiology in both years
but in this year they’re learning the normal by comparing and contrasting it
with the abnormal in the week they’re learning cardiac output they have a
patient with heart failure but it’s not primarily to learn the disease that
causes the heart failure primarily to learn the syndrome and they learn in the
first year what the renin-angiotensin-aldosterone system
normally does by comparing and contrasting it the normal to an abnormal
and why when a patient with heart failure lies down at night to their
lungs fill with fluid and they have trouble breathing
why doesn’t that happen to everyone in this room how do we stay out of heart
failure is the kind of and what it what is then normal but again by comparing
and contrasting it with the abnormal in the second year we remind them of the
pathophysiology and then focus on the diseases that cause it unless theme and
variation so in the first year the heart failure patient has a problem with
muscle pumping in the second year we introduced valvular disease with the
pressure or volume loads that caused it and so forth so we connect them
interstitial lung diseases we just recently did this ok there’s almost
Avogadro’s number of possible interstitial lung disease and they can’t
just do it by memory alone but we had a week that was organized around it ten or
eleven pathophysiologic categories with exemplars
each category and we then said and the crucial thing is the exposure so by
organizing and connecting through the coherent narrative pathophysiology which
includes again from genes to society so in effect they spend two years studying
pathophysiology connecting across all these sciences and why not include when
you say okay well you know that heart failure is associated with excess
activation of renin-angiotensin when we talk about its treatment this is where
they work they interfere with this activation or something like that and
then let’s it’s not too hard to say well how well do those treatments work see
how those pieces do fit together remains to be seen though I suspect like many
things it’ll be they’re roughly the same or better but I share your pain about
nephrology I’m not a nephrologist but because I’m a general internist who’s
unafraid to teach the I’m the cleanup teacher so I ended up teaching a fair
amount of the kidney pathophysiology I’m sure there were other questions and then
I ignored them so I apologize yes ma’am Hercule well I’m sorry that medicine lost you
but perhaps psychology as the better for it or whatever your field now is let me
say that I think the tectonic plates are moving in that direction even if I don’t
do it but I’m trying to play a role in my local situation first the tectonic
plates have you heard already about developmental the the Boyer excuse me
the Dreyfus and Dreyfus model a developmental model of competence has
people already talk to you through that well I’ll tell you what a two-minute
sidebar in that the idea is that professionals don’t come out of the can
fully formed right you aren’t born an excellent teacher or an excellent doctor
or an excellent scientist you have to grow to be that way and that growth is
not necessarily predetermined simply by going to school that growth has to do
with the development of particular competencies and if you take a
developmental stance ie you start somewhere novice you become expert what
are the milestones that you can begin to recognize as they move towards
competence and what are the experiences in coaching and feedback that you have
to get to reach that stage and if you take that general idea that is what a
lot of people mean the plus side of that is what people have called
competency-based education now this has
known for a while this is not necessarily new idea in fact the ACGME
the crediting body that accredits graduate education
some years back insisted that residency’s and you may have to live
with some of this in your work residences adopt a core competency model
for assessment of readiness to be certified and they identified six core
competencies let’s see if I can do it medical knowledge sure that sounds right
patient care sure we’re for that communication and interpersonal skills
okay yeah that’s good professionalism we’re all for that then to that a little
less clear practice based learning and improvement and systems based practice
but then when you look under the hood practice based learning and improvement
has a lot to do with what you and I might call evidence-based practice and
systems based practice has a lot to do with working deliberately effectively
with teams to provide health care and if you start looking at the Canadian system
they have a different set of competencies but in the last decade or
two now at the graduates level they have moved in the direction of
competency-based education well as you can imagine the magnetic force
gravitational force field of residency got the attention of a lot of medical
schools and surprise surprise a lot of medical schools are now going in the
direction of competency-based education and it may not surprise you that they’re
roughly the same six a year to back the Howard Hughes Medical Institute and
another group double AMT perhaps put together a study of is there such a
thing as competency-based education for undergraduates and they’ve released a
report there are two parts of it one is the science preparation and another is
the humanistic that has just recently been published and those are the
tectonic forces that will eventually get done what you’re
talk about what we’re doing on a local level is as we’re doing this in a
competency-based education way we’re also talking with the people right
across the street in the College of Ed and down the road in the biomedical
Health Sciences Institute to see if they’d like to align their education in
a way that prepares UGA is graduating seniors for success in our model I think
they’re waiting to see if our model succeeds but assuming it does along the
lines of what we talked about before assuming it does then I think they might
be willing to have a go at it because they were quite interested now remember
this is threatening to a lot of traditional departments because I’m in a
department of biochemistry or a Physiology or Anatomy cell biology we
meant wait a minute wait a minute you’re gonna have a course and it’s not going
to be called biochemistry or cell biology or anatomy and I’m have to teach
this but I won’t get credit so we have to find ways to administrative lease
all’ve the barriers to interdisciplinary
education what that sounds plausible that sounds like we might be able to get
it done but they won’t give it up for nothing I think what they need to give
it up for is more integrated undergraduate pre metal pre-med
preparation but that’s there’s a lot of persuasion left a head hard bargaining
I’m told is the way they say it here in Washington when they talk about all
these International Peace collaborations and so forth hard bargaining lies ahead
what other questions do you have yes my questions a mirror of the last question
yes how can we apply these principles to continuing medical education thank you
for asking let’s not forget this line back here most of what is done in the name of
continuing education as you know is a repeat of preclinical medical education
hour after hour of lecture the randomized trials show that there’s no
change in clinicians knowledge or their decision-making after the base on the
basis of a standardized lecture in fact it’s a very creationist view of medical
learning that you learn Anatomy once you learn something once and then after that
you don’t have to learn anything new you don’t have to unlearn so studies that
have shown effective things have showed that there are people that have figured
out ways to help people unlearn some old ideas and learn new ideas through more
active learning whether it’s workshops a thing called academic detailing so
instead of somebody telling you a drug product someone comes out and sells you
a new idea by showing you how it makes you better faster doctor or other
mechanisms to engage people in skill building there are methods of effective
ceme in fact there’s a movement among CME now called evidence-based continuing
education ie let’s use studies of effectiveness of these methods to guide
our efforts so there’s a another movement underway of people trying to
inform processes of continuing education what lags behind state approval the
certification there’s a lot of administrative processes that regulate C
and me that aren’t yet there we have another huge barrier and that one of the
major forms of financial support for continuing education is industry and I
hear startup ruckus but there’s lots of problems with that support and one of
the challenges is who’s gonna pay for continuing education okay and if you go
back to that water analogy if you say that we all pay if we don’t have current
best knowledge if we go and it’s 30-year old water filled with goo
and we use that for our living and eating we get sick same thing if our
knowledge is not current best knowledge we all in some way pay for it eventually
it kind of comes down to how are we going to share the expense of continuing
professional development for knowledge workers in healthcare I don’t know that
we have that sorted out but meanwhile we’re trying to help our students build
the skills because one of the essential skills is being able to recognize when
you don’t know something turn that hole in your head instead of hiding it you
know we all hide what’s cool to hide when we don’t know something right
you’re not sure this is true I can tell by the look on your face it’s alright
ask yourself when a teacher asks you a question what did you do if you don’t
know the answer you raise your hand you raise your hand you say I don’t know
well bless you but most people do not do that most people look down or they look
away try to become invisible very few people go away come on me I don’t know
this but boy am I ready to learn today it’s laughable because we learn from as
far down as kindergarten to hide when we don’t know and this is adaptive in
situations where knowing already is what’s valued and not knowing where
already is taken as a sign of weakness but it’s maladaptive in health care work
because if you don’t know and you hide that you don’t know who pays that price
your patients or your population we cannot afford that so what I want to
know is why don’t we get do I think we’re gonna maybe make evidence-based
practice possible oh yeah there’s some people working on that because when I
started people would laugh at me like I thought we should look up stuff to know
what to do for our patients and we’re gonna try and get it into the curricula
of medical schools oh yeah they’re people working on that we’re gonna
spread it to any field who wants to yeah they’re people working on that we’re
going to overcome the barriers to it they’re people working on that but each
in our own way you could take up this idea of if we’re gonna make
well-informed decisions about whatever my work is and we’re gonna teach our
young wings to make well-informed decisions after you’re gone how is that
going to happen each of us could make it happen in our own environment but it
does take I think a fair amount of willingness as Garrison Keillor says to
stand up and say what needs to be said even if you’re shy yes ma’am she’s gonna
have you repeat the question what role does electronic health records
play into your curriculum into our curriculum into the future okay I look
forward to the future day when it helps us right now if you’re in a clinical
environment in most situations in both elect electronic health records if you
want to look up something you have to sign out of that record because of all
the HIPAA and the patient confidentiality requirements that you
can’t actually have it open while you have other things open and you have to
go somewhere else or use some other device to get knowledge this seems to me
fixable no I’m not the programmer that’s going to make this happen
I don’t program at all but someone should be able to figure out how to have
knowledge resources woven right into the patient care flow resources so that when
you need help working up someone with hemoptysis or when you want to choose
was to use a resource about the adverse effects of a medicine you don’t have to
leave what you’re doing and go somewhere else and again that water analogy right
now when you want to go get information it’s the equivalent of you’ve got to
stop what you’re doing and go down and over here to the riverbank and take your
bucket and then go boil it yourself to get rid of the impurities and add the iodine and then wait a few
days and then you have knowledge that’s free Tobias why isn’t it like the tap
why don’t the electronic health records find a way to get knowledge resources in
in focused amounts at just the right level that represent current best
knowledge and not just the guidelines that they want you to follow but current
best knowledge that allows you to individualize care and personalize it we
don’t have that yet but it’s going to happen some somebody needs to figure
this out there was another hand up but I’ve moved around so I have lost track
of where it was okay well I’ll tell you what let’s skip ahead
to the end here’s them imitating me I take it as a compliment there they did
this to entertain themselves but it also entertains me I’ve already asked you
four questions this is another shot of our building I
want to thank you it’s gonna close this out dr. Richardson
you did exactly what we wanted you to do I think to a person we enjoyed the
presentation and there’s a good chance that we’ll remember some of what you’ve
said I want to thank all of you for participating and I Triple A and nci’s
division of cancer prevention worked with the Office of Disease Prevention to
sponsor this activity we’ll look forward to seeing you again

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *