♪ (intro music) ♪
(Jon) Hello, and welcome to Take 5 with The Joint Commission.
I'm Jon DePaolis, and I'll be your host for this episode.
Today, we're going to have an in-depth discussion
about the launch of the new survey technology:
Machine Learning for Survey Consistency.
This is an artificial intelligence program
that will help field staff
identify the best standard or element of performance
for a finding to be scored under during an accreditation survey
or certification review.
This is a complicated topic,
so I'm glad to be joined by Kin Lee,
The Joint Commission's Chief Information and Enterprise Security Officer.
(Kin) Well, thank you so much, Jon, happy to be here today.
(Jon) We are also being joined by Emily J. Wells
Project Director, Survey Management and Development
in Accreditation and Certification Operations.
Emily, thanks for coming on the show.
(Emily) Thanks so much for having me.
(Jon) Now, Kin, this has been a huge undertaking for you and your team.
Why did The Joint Commission decide it was time for machine learning
to become involved in the survey process?
(Kin) Yeah.
Over the last couple of years,
IT and the business, we have been formulating,
exploring a wide range of innovative technologies
to address business challenges.
We use RPI or a payoff matrix to help us decide
which opportunities we should pursue.
One of them was artificial intelligence and machine learning
for different business use cases.
But when we kind of discussed with the business executive leadership,
we identified that one of the pain points
was the inconsistency of assigning standards and EP
to a finding when surveyors or the field staff conducting surveys.
That was, as I said,
one of the pain points with the customers.
So we thought that there could be a solution
that we can deploy to
to mine many years of vetted survey data,
survey reports that have been done over many years
to help the field staff to accurately and consistently assign
correct standard and EP by using machine learning technologies.
The results show that not only the survey consistency has improved
but also produced time-saving productivity for our field staff.
So this machine learning technology
is part of the broader business digital transformation strategy
by leveraging data and analytics for other business use cases.
So we thought that's the right timing to explore this technology
and add value to our survey process and our customers.
(Jon) Emily, how much of the final accreditation decision
is going to come down to machine learning versus a field staff's judgment?
(Emily) That's a great question.
Machine learning is really a conduit to assist the field staff
as well, actually, as our Standards Interpretation Group,
with consistent scoring practices.
And so, our goal through this initiative
is really to leave the most accurate report
with our accredited organizations and to improve our overall consistency
in regards to where these issues are scored
both at that standard and element of performance level.
But of course, with any survey or review,
our field staff will use their judgment
and their clinical and engineering expertise,
really to make determinations about scoring
while using machine learning
as a really vital tool throughout that process.
So, they really do go hand in hand together.
(Jon) Kin, Emily touched on accuracy.
I am curious, how accurate does this machine learning program
appear to be?
(Kin) Yeah, we are very excited about this question, actually.
When we set out to develop this technology,
and it's a process of implementing the solution,
we set a goal of having the correct standard and EP
appearing on the top 10 search positions at least 75% of the time.
What that means is,
when a surveyor types in a keyword or phrase of findings,
the machine learning engine would provide possible match for those observations,
and we want to see the correct standard and EP
at least 75% of the time at the top 10 position.
After the testing
and real-life application of the technologies,
where the field staff are using that during surveys,
we are very happy to see the correct standard and EP
appearing on the top 5-- not top 10 only, top 5,
so it's positioned more than 80% of the time.
So it's better than our initial target.
I will also add that because of the nature of the machine learning technologies,
it has self-learning capabilities, continuous learning capabilities.
So, with additional data and knowledge by the field staff,
we expect the efficacy level of the solution
will continue to improve over time.
(Jon) Emily, from the customers perspective,
how will the survey experience change
as a result of this technology being implemented?
(Emily) Machine learning actually integrates
into our mobile survey technology quite seamlessly.
So when the customer is experiencing
an on-site accreditation survey or a review,
they really aren't going to see much change.
Now, that said, our goal for machine learning
would be for our customers to really experience
improved consistency in our scoring practices
between the field staff,
so that process that occurs during the survey or review,
as well as in our interpretation guidance.
So we would hope to see that our customers
see increase satisfaction with that improvement.
(Jon) Kin, I am curious, in your opinion,
what does the future of surveys look like
and how do you think the field can expect the technology
to continue to be integrated into their survey experience?
(Kin) Yeah, it's a very exciting time,
right now with the progression
of innovative technologies, emerging technologies.
We have much more opportunities to leverage those new capabilities
to make our survey process more advanced
and creating positive customer experience.
So what I would think is the future of the surveys
would be more of a hybrid model.
Today, the surveyor,
they would visit the organization on-site
and they would conduct the survey 100% on-site.
I think, going forward, there would be more of a hybrid model,
meaning some of the field staff will be on-site,
but they are able to leverage remote experts
to support the survey as necessary,
and some of the administrative type of work, such as document reviews,
could likely be done remotely,
and free up the time
for the field staff on-site to focus on high touch activities,
such as tracers.
To prepare for the hybrid model,
what we are evaluating is what technology we could leverage
to enable remote expert assist automated document processing,
mixed reality training,
just to name a few examples.
So we will continue to evaluate, explore,
and working, obviously, with our customers
and with our stakeholders, such as CMS,
to make sure that we are feeding the compliance requirements
and the confidential requirements and all the elements
so that we can continue to move this technology forward.
(Jon) Definitely sounds like this is an ongoing effort.
So maybe moving forward,
how are you and your team going to continue to improve
the new machine learning technology?
(Kin) Yeah.
The way we implement machine learning artificial intelligence technology
is not to do those use cases one by one,
but to look at building an AI knowledge backbone
that we can plug and play additional data sources or applications
to expand our capabilities in an edge-out fashion.
So we would continue to expand and leverage our data sources,
analytics, capabilities,
to drive insights and also predictive capabilities.
So those are all the elements that we are thinking about moving forward
in terms of progressing with this journey,
(Jon) This is very exciting information.
Emily, where can organizations go
if they have more questions about machine learning or the AI
which will be used on survey?
(Emily) I know that, through some of our efforts,
we have a few publications and information blog post
that has been sent out related to machine learning.
Of course, organizations can also reach out directly to myself,
and I believe Kin would say the same.
And we're really excited about this initiative
because I think that machine learning,
in closing, from my perspective, has really just opened up new potential.
We were really limited in the data that we had for field staff
in terms of scoring suggestions in the past.
So to be able to leverage
machine learning and artificial intelligence
the way we have now
really gives me a lot of hope, as well as a lot of excitement,
about other business processes
that we can use to improve consistency too.
And so, I think as we move forward with this initiative,
more will be released about other untapped potential that we have,
and we'll be using machine learning in the coming years as well.
(Jon) Emily, Kin, thank you both so much for coming on the show today.
(Kin) Thank you so much, Jon, for the opportunity today
to talk about this exciting technology, and like Emily said,
either one of us would be very happy to answer any questions.
We also have very capable technical and business teams behind the scene
who work very hard every day.
They can certainly help answer any question as well.
So thank you.
(Emily) Yes, thank you so much.
(Jon) And as always, thank you all for listening.
You can find more episodes of this program
on The Joint Commission's website at www.jointcommission.org.
This has been Take Five with The Joint Commission.
♪ (closing music) ♪