Look at all these
beautiful faces.
Welcome to
Dreamforce again.
We are really
privileged this morning
to have two of the top
minds in health care
Let me first do a
brief introduction.
To my-- two to the
left, Dr. Ken Trauner,
Chief Innovation Officer
of Kaiser Permanente.
And I found out,
also recently, Ken
is a graduate of
Stanford here, undergrad,
and then went
on to Harvard
So definitely
very smart man.
Orthopedic surgeon
and a bunch
of other accomplishments,
I'm sure.
Thanks for
having me, Phil.
And then we have
Dr. Kris Lee.
Dr. Lee here is my wife's
boss boss boss boss.
My wife is a
Kaiser doctor.
And so it's a
real privilege.
Dr. Lee, the
official title
is associate executive
director of TPMG
But really, for those
who are the insider,
everyone knows
she's the boss.
So as we begin
this conversation,
we'd love to invite
both Kris and Ken here
to tell us a little
bit about TPMG
and Kaiser Permanente
in general.
But before that, I want
to just briefly share
We have been on a journey
in the last six months
working with TPMG on
a set of agentic POC,
using agentic technology
to help both providers
and potentially in
the future, patients
to get access to more
knowledgeable question--
question and answer
systems that are citeable,
that are trustable,
and so on.
And this is a real
interesting place for us
as a company to
choose health care
as a proof of
concept for some
of the agentive
technology.
Health care is a place
where getting an answer
You need to get an
answer that can actually
be verifiable
and trustworthy.
And so I love to invite
Kris and Ken to share
with us their
first impression
of the current state of
our PoC this morning.
But before that,
maybe, Kris,
can you maybe
spend a few minutes
with us telling
us what is TPMG?
What is Kaiser Permanente?
So Kaiser Permanente is
a very unique health care
ecosystem made up of both
the insurance company
and the medical
group, which actually
are the doctors
and the nurses
and the folks that are
delivering the care, all
in partnership
in one spot.
We also feature a
prepaid model of care.
So you pay your
premium, right, Phil?
You pay-- you're a
Kaiser member, right?
So you pay your
premium for the year.
No matter what
happens to you in--
for the rest of
the year, we're
just going to
take care of you.
We're not going
to bill you
for every single
procedure and every single
intervention or
visit that you might
So it's very
different from the fee
So you can see our
incentives are really
aligned with where we
think the health care
industry needs to be,
which is we make money.
I mean, I hate to talk
about making money
in health care,
but I have to have
a roof over my head, too.
We make money by
keeping people healthy,
which is the way the
entire industry, really,
honestly should
be aligned.
And that's what we do
at Kaiser Permanente.
Kris, you know, we
are in an age of AI.
Can you share with
us, with the audience,
what is the role of
AI, in your view?
And what are some of
the risks, potentially,
that you see that AI
has for health care?
So when we think
about health care
as an industry, we
can't get it 80% right.
You want me to get it all
the way right if I can.
And the way I see AI and
particularly the work
that we've been doing
is augmenting physicians
and clinicians to make
the best decisions
possible for our patients,
to help our patients get
the best care
that they can get,
and to be the healthiest
that they can be.
And the risks are
not getting it right.
We need it to be very
high level of confidence
in the information that
the tool is helping
and augmenting
the clinician,
the clinical
teams to do this.
Now, I keep saying
the word augmented,
and I think this
is very important.
We don't believe in
artificial intelligence
We believe in
augmented intelligence,
which is really
putting the patients
and our people at the
center of the focus
and augmenting the
care that they're
receiving or
delivering around that.
Did you want to
add anything, Ken?
No, AI has been used
pretty extensively,
as you know, with
machine learning models,
and we use it throughout
the organization.
The generative AI
is what we're really
talking about, which
is how are we--
how is that going to allow
us to accelerate what we
can do, the acceleration
of the development
But also I think Kris
is talking about it
in the context of
how we interface
with our patients,
which is where we
think it's most exciting.
And we'll get into it a
little more as we speak.
But it's quite
promising, But we
do have to be
acknowledging the risk,
It's very different
on the consumer,
and we hear all
the different other
It's very different
if the toppings
are wrong on your
pizza than getting
the prescription wrong
on your prescriptions,
So Ken, you are the
Chief Innovation Officer
Could you tell
us how do you--
what is your
day look like,
and how do you drive
innovation across TPMG?
Those are separate
questions.
And how do we
drive innovation?
Well, first of all,
what I want to point out
is that innovation,
and there's a mandate
that we need to
disrupt ourselves.
Health care is in crisis.
There's an imbalance
between the demand
and the supply
of physicians.
Our physicians
are overburdened,
which we'll end
up talking about.
So we need to
solve that problem.
But change is more than
just technology, right?
So one of the
things that we've
done over the last
couple of years
that maybe not
everyone is aware of
is we've reorganized our
leadership structure so
that our division
of research
and all of our
data assets,
our technology group
and all the programming,
our innovation group and
our business consulting
and our virtual
medicine is all
And technology so that
we can share resources
and move in a much
more expeditious way
in creating a real
advanced development
capability that lets
us very rapidly create
prototypes that we can
then start to test.
Because health care,
unlike other areas,
we have to
validate what we do
It's highly regulated
for a reason.
And we have to make
sure that what we do
is safe on so many
levels but also
accurate and equitable
and all the things that
come into the regulation.
One of the stories
that you shared with me
earlier in the year was as
an integrated health care
delivery system, you
are able to move also
very quickly when
new technology are
there's some
astounding number
on how quickly
you can deploy it.
Can you share that story?
So the first-- there's
a technology called
And I know many
of you, when
you go to the
doctor's office,
have all had the
experience where
you go to the
doctor's office,
and they're typing
away at a computer
and not really paying
attention-- or paying
attention to
you but really
trying to get their
documentation done.
And ambient technology
is technology
where it listens
to the conversation
between the doctor
and the patient,
and then it converts it
into a doctor's note.
So by the time you
leave the room,
the notes are already
completed for the doctor.
And while it may seem
like a very simple use
case for AI,
it's really quite
impactful to our doctors.
There's actually a concept
now called pajama time,
and pajama time refers to
the time your doctor is,
At the end of the
day, are spending
charting because
they haven't
had a chance to complete
their administrative work.
And it's crushing
to our doctors.
Burnout is an
enormous issue.
So this ambient
technology we introduced,
and actually Kris's
group introduced this
And it was
introduced-- we-- now,
our system has about
10,000 doctors and about
And this platform was
rolled out over four weeks
And after a
four-week period,
we had about 35%
adoption from scratch,
from a standing start,
with over 300,000 visits
that had been transcribed.
So if you take three
minutes per visit,
three to five minutes per
visit is over 1,000,000
minutes of time,
physician time,
saved in the first month
from a standing start.
Now, I don't know any
other health care system
in the country that has
the ability to scale
And that's just
the beginning
for many of the things
in the direction
Kris, do you want
to add to that?
The other cool
thing is that it
is saving the physicians
that time, both
within the clinic visit
and then when I go home
and I want to hang
out with my kids,
I mean, sometimes I want
to hang out with them.
At least, I can choose
to do that if I want to.
I don't have the pressure
of all of the charting
The other piece that's
so cool about it is
think about the
patient experience.
How many of you have gone
to your doctor's office,
and there's a third
person in the room, which
is the computer sitting in
front of the doctor's face
like this, and you're
on the other side?
Now, imagine I'm able
to remove that and just
look at you and talk
to you as a human being
and say, how can
I help you today?
So the member experience
is actually augmented
So it's really fantastic.
Now, Kris, I would love
to have you describe
I don't think-- for those
folks without family
members who are doctors,
they really have--
I don't know whether
people really
appreciate how
much technology
is a burden sometimes
when you need
when you need to go
home to do pajama time.
Can you share
with the audience
Doctor is the most
respected profession
Why are they burning out?
So with the-- this
started a while back
with the introduction
of the electronic health
record, which
was a blessing,
and sometimes
it's a curse.
I tell you, sometimes
it's a curse.
And we not only have
our 20 visits a day
sometimes more--
of our patients.
We also have
patients emailing us.
On average, our clinicians
in Northern California
receive 44 secure messages
from patients every day.
And what do you think
is in those messages?
They're like
clinic visits.
I have a runny nose,
and I have allergies.
And by the way, can
you help my dad?
Because I think
he has dementia,
and I need a refill
prescription of this
and-- all in one message.
So you can imagine
for clinicians
and sometimes,
hey, you guys,
I've had heart
attacks in my email
where a patient
writes in and says,
I think I'm having
indigestion.
I'm just going to sleep
this one off, Dr. Lee.
And I'm like looking
at this message at 5:30
in the morning because
I want to get some done
before I have my clinic
and then do my charting
So you can imagine the
immense psychological
pressure that's there
in-- and also in terms
of just the sheer volume.
And this is not a problem
just at Kaiser Permanente.
This is a problem in
the entire health care
Physicians are
burning out.
Training programs are
not filling the way
I'm looking at
Dr. Walker here.
In the ER, there
was a 25% vacancy
in training
positions for post
graduate medical
education.
Everybody, this
entire industry,
is facing a huge shortage
of doctors and nurses
and technicians and
all of the people
And so we must
change the way
that we deliver
this health care.
We cannot keep doing it
the same way that we have
We need to
embrace technology
to help us because,
again, very
So there's a concept
of practicing
to the top of
your license.
And as much as the
patients don't appreciate
when the doctor is in
front of the screen,
the doctors hate it also.
They went into medicine
to talk to their patients,
So the ability
to have time
to connect with
the patients
And that's been
removed or displaced
by low level
administrative work, which
is what we need to address
and where these tools can
Both of you, Kris and Ken,
tantalizingly described
a problem and
potentially how
AI or maybe technology
in general can help.
Can you maybe paint
a bigger picture?
I sense that in your head
there is something there
it's such a
crisis right now.
We need to have an
impact now, right?
It needs to be
done at scale.
And it has to be low risk
because, as you know,
these tools are new, and
they introduce things
It's very early on in
terms of regulation,
in terms of standards,
in terms of what--
how we need to create
responsible AI.
It's very early, but we
need to make a difference.
And to us, we
have a concept
that we call point of
contact resolution, which
is if a patient
has a problem--
and this is unique
to our system.
If a patient
has a problem,
how can we find
out what's going on
with that patient,
provide context
based on their
medical history,
identify what information
we need to gain,
and get them to the
resource that can solve it
at the first touch point?
That's what we're
going towards.
And then when you
break that down
into its component parts,
you start to realize, hey,
those are achievable baby
steps that actually can be
done with the
existing technology
and that we can allow us
to move towards a more
unified front door
for the patient
to get to the resources
they need right away.
And for-- imagine that you
go to the doctor's office.
And you walk into
the doctor's office,
and they already have all
the information they need,
and they already
know what's going on.
Who's gone to the
doctor's office where
you've told your
story four times,
and you walk into
the doctor's office,
and they start saying,
what's going on?
It's a very
different experience.
If the doctor walks in
and says, hey, Mr. Smith,
I understand your knee has
been hurting for three--
I'm really sorry
about that--
and is able to
immediately then spend
time in shared decision
making and connecting
with the patient,
which is restoring
the balance to the
practice of medicine
Yeah, no, if I
might, Phil, I also
think there is a
big patient safety
we're supremely
responsible for.
We can augment
patient safety
in this way using
AI, so getting you
to the right person,
place, or service
And this is
some of the work
we've been doing together
that is so very exciting.
It's really helping
in that realm as well.
Now, before we
open up questions
for the audience,
I just want
to say just how much
we in Salesforce, we
are feeling so privileged
to be working with one
of the top, if
not the top,
health care organizations
in the world.
how to apply AI to a
very important industry,
one last question for
me, maybe for both Ken
What is it that Salesforce
is working with you on?
Can you just,
at a high level,
describe what is
the proof of concept
Yeah, so I would be
happy to talk about it.
So we've been working
on some proto--
And by the way,
he complements us,
but Phil has an
exceptional team.
And we really appreciate
working together
with our rock star
team, small but mighty--
as we've been developing
a agentic AI tools.
And what's interesting is
we started with creating
And if you think
about it, as a system,
we standardize-- we
try to standardize
as much as possible
and practice
That's what we do
as a system that's
different than a lot of
others because of some
of the structures
that Kris alluded to.
So we have a large
body of information,
but it's quite fragmented
and often difficult
And so the
question is, can we
create a reference
engine or a tool which
is a front door in a
simple search bar that
allows our
providers, initially,
which is where we
started, to have
simple access in a
Google-like fashion
to unlimited information
that's curated, accurate,
Eventually, once
it's all validated,
we'll have the ability to
deploy that to patients.
Because if you think
about it, patients,
most of the time,
when they contact us,
they're not-- patients
have a challenge.
They either are looking
for information,
they're looking
for a treatment,
or they're looking
for reliable--
their answers to
their questions.
And so much of
what we have,
we have those
answers is just not
So how can we provide it?
So the first project, the
first prototype we did,
was a simple reference
engine with materials that
are widely available,
and we've launched it
And what's so interesting
is when we say agentic AI,
it's not just simple,
LLM, simple search query.
It actually allows you
to compare it to that.
But what you don't realize
with so much of the search
that we do is there's
an element of reasoning
to get the-- what people
want are the answers.
They don't want
access to something
And so a lot of the
searches-- and Phil
gave a nice
demo yesterday.
If you do a search,
for example, for what's
a pediatric dosage for a
certain allergy medicine
and you do a
search and say,
I have a six-year-old who
has allergies and needs--
what's the dosage for
whatever the medicine is?
I don't know any
of those things.
And if it sends
you a reference,
it gives you the material
and takes you to the site,
and it says, for patients
4 to 10 years old,
this is what
you use, you're
having to actually read
through these materials
The agentic AI in
its earliest stages
It'll look through
and say, OK, it's
It's between 4
and 10 years old,
and it gives you
the simple answer.
This patient
needs this dosage.
And so what you start
to realize as you start
to play with these tools
is how much of what
we do to get the
answers you want
And that's where
all this, you
start to realize the power
of some of these tools
as we start to make
a simple search,
simple answers, and this
is at its earliest phase.
The stitching together
of these agents
is so promising
on so many levels.
But, Kris, you
have comments?
One of the
challenges that we
face in creating
these things with you
is that we have
a lot of data.
Kaiser Permanente
has a lot of data,
and we have a lot of
longitudinal data.
We have more than 30
years of data around--
in our research groups,
with our technology
groups, from our patients.
And the value
of our data is
that it's different from
the fee-for-service world.
It's removing that
bias of like wait,
let me bring the
patients in as much
as I possibly
can because I'm
going to make more money.
I'm going to do this
scope and a scan
Our data set is unique
in that it's really
coming from this
value-based,
keep the patient
healthy perspective.
But I have not a data
lake, I have data puddles.
Here's some information,
here's some information,
And by being able to feed
all of that into the tool
the proof of concept that
we've been developing,
I'm starting to gather
all of that data
into my lake in an easily
accessible fashion,
both for physicians
for right now
and eventually, I
hope, for our patients.
I don't have to
remember, oh, go look
in that database,
or go to Clarity
I don't need to
do that anymore.
Now I have this wonderful
tool, so thank you.
Wow We are still very
much in our early phase
One of the things that
we very much would like,
next year when
we come back,
we would love to invite
both Kris and Ken
Hopefully we'll be able
to report our progress
Before we end,
we'd like to see
if there's any question
from the audience
that Ken and Kris can
potentially answer.
Yes, you discussed
ambient technologies.
What do they look like,
those ambient technologies
that will help
the provider
do those charts in a
more quick fashion?
So it's-- everything's
on the smartphone.
I bring my smartphone into
the room with the patient.
I consent every--
every patient consents.
Yes, we're going to
use this technology.
I click record, and I
put it down on the table.
And I just have my
patient interaction,
like, so, finish up
with all of that.
I'm not-- and I'm
not dictating.
This isn't like using
like a dictation
or a Dictaphone or any of
that type of technology.
I'm literally having
a conversation.
Phil, how's
your cat doing?
OK, that doesn't make
it into the note.
But, Phil, this
is what we're
going to do for your
allergies and your sinus
That makes it
into the note.
At the end of
the conversation
that I have
with my patient,
I don't have to
do anything else.
It brings the entire
curated note into the EHR.
My license is on the
line, right, anything
So I proofread it-- it's
pretty darn good now--
proofread it, and
I just say yes.
So I've literally done one
click, two clicks, three
Our doctors, on average,
are saving one hour a day.
And this was before
it was integrated
into the Electronic
Health Record.
We recently moved into
an integrated version,
which is now saving our
doctors even more time.
And the hour a day when we
looked at what time of day
was this going
into, it's not
coming out of the
direct patient care.
It's coming out
of pajama time.
So Phil's wife can now sit
down with Phil for dinner,
and then he can do
the dishes afterwards.
But that's what
it looks like.
That's what it looks like.
So I'm going to
cheat a little
because I used to work
for Bernard Tyson,
and I ran your Center
for Total Health.
And some of the
things that you've--
that I'm wondering,
because Kaiser Permanente
is the most expensive
insurance out there,
and I know as a
self-employed person
when I look at the
health exchange.
So how are you going
to promote these kinds
Because people-- you've
got a lot of competition,
and people need
to know that--
I just banged up
my knee, and I
don't know when you
implemented some of this.
But my PA, I walked
into his office
a couple of days ago,
and he knew everything.
So I don't know if you've
implemented it yet,
but you've got to
really figure out
how to promote this
so that members stay.
The fact that you've
got decades of families,
you've had an Electronic
Health Record since--
for 80 years at
least, some form.
So how are you
going to do that?
That's a really
great question.
I'll take that
one to start with.
First of all, I
don't know that we
We should be because I
feel like we're the best.
But-- sorry, I
had to go there.
But in terms of promotion,
so I'm publishing.
So if you look in
the literature,
so I'll use the ambient
AI scribe as an example.
When I was looking to
employ that technology
and I was
thinking, wow, this
I think this might be
something that-- we
might be on to something.
I went to the
literature, and there
was nothing, nothing in
the literature about how
And so, really, I
publish about a lot
I publish in very in
esteemed medical journals,
And we all should be
talking about this.
I think we all
need to be talking
So I go to a lots
of conferences,
some like these,
and really try
to engage as
many people, try
to engage the
regulators, try
to engage the
government, try
to engage other
health care
systems and other
health care providers.
Kaiser Permanente,
of course,
I'm always going to
promote what we do.
And what we do, I think is
deliver really excellent,
Yeah, so to add
to that, I think
in addition to
participating
in various communications,
we need to collect data
and actually not overstate
what we're able to do.
This is all very
early, and I
think it's really
important to be accurate
and to present the
data so that people
can see what's actually
true and what's not--
This is the
Gartner hype cycle
with regards to AI
and health care.
And you go out
and you hear
all the things that
people are saying
can be done in
health care.
There's a lot that could
be done, but some of this
is going to take
quite some time.
So I think we need to
be responsible about it.
Yeah, so before we go
to the next question,
I just want to say I'm
myself a Kaiser member.
And I can attest that
between the various health
care plans that
I have tried--
and I've tried
a lot of it--
Kaiser Permanente,
really highest quality
care that I can personally
find for my family.
There's a lady in
the front here.
I am really impressed
by the first touch.
As a mother for
40 years, this
is what I wanted for
myself and for my family,
the idea that I could
go in with a problem,
and the problem
is understood.
And you're gathering
information that's
relevant to me and
you're directing me
to the right
provider is huge.
My question for
you is, it seems
like the logical
next step is
to add scalable services
to that content.
Are you thinking about
that at all at this point?
What-- so you don't have
enough providers to--
And so you're
going to have
to use technology-driven
solutions.
And I'm wondering, what
are you focusing on there?
So when I'm thinking about
creating or employing
or deploying
technology, I really
have a couple of
buckets that I'm
One is physician and
provider burnout.
So decreasing the
clerical burden
and administrative
burden on folks
so that they can work to
the top of their license.
The second is
around patient
engagement, so
patient safety
and patient engagement
and bringing my--
inviting our
patients to come
into this space with us
and learn for themselves
and learn from
these tools as well.
What is the dose of
the nasal allergy
You don't need to talk
to me to know that.
I'm going to just give it
right out to the patients
That's where I
would like to go.
The third place I see
is around navigation.
And I think this is
a big health care
OK, she lives in Hawaii,
and she does not have.
She will not get
Kaiser Permanente.
Come on, let's get you
some Kaiser Permanente.
So she's with another
health care system
And so she gets-- she
has her insurance.
She pays a zillion
dollars for her insurance,
Here's a list of all the
doctors in our thing.
Go find yourself a doctor.
And she can't find the
doctors that she wants
She can't find a
primary care physician.
She can't find
those things.
So I think about
navigation also
for our patients but
also for the physicians.
You know, we have
so much data.
We have so many
references.
We have so many
algorithms and treatments
and different things that
we can do for our folks.
Make it easy to augment
my decision making process
and at the same time make
it easy for our patients
to navigate through this
vast health care system.
I think those
are the three
buckets that I'm really
thinking about right now.
So we have roughly
one minute left.
I want to just
pause, and I
want to tell you what
I'm looking at right now.
I have never
been in a setting
where we have a group of
audience with brightly
If you guys could
entertain me for just one
second, I would love
to take a picture
with the audience with you
guys in the background.
Challenges and successes.
Where are we headed next?
Gratitude for
everything you do.
Thank you for
visiting everybody.