Trust in medical AI

My dissertation at Georgetown University, completed in 2024. Six preregistered studies, with more than 3,000 participants recruited, on why people trust a doctor's advice more than an AI's.

Main findings

I wear a ring that tracks my sleep and tells me how to improve it. Most days I ignore it. When a doctor at urgent care tells me the same thing, I listen. I wanted to know why, and whether it can change. So I ran six experiments with more than 3,000 people. Here is what they showed.

People trust a doctor more than an AI, for every kind of decision

In Study 2, 106 adults said how much they would trust a doctor, and how much they would trust a medical AI, for twelve decisions, from diagnosing strep throat to prescribing insulin. Of the 103 people who answered every question, 97 trusted the doctor more. Only 2 trusted the AI more.

Take strep throat. People rated their trust in a doctor at 8.4 out of 10 and in an AI at 3.9. Even for something as everyday as advice to stretch, the doctor won, 8.1 to 4.8.

97 of 103 trusted the doctor more than the AI

Each dot is one person in Study 2. Blue dots trusted the doctor more.

Doctors win, but the AI is not a zero

The AI still earned a middling amount of trust, about 4 out of 10 across the twelve decisions, against about 8 for the doctor.

So the question is how much, not whether. People weigh an AI's advice with more skepticism than a doctor's, even when the advice is the same.

Doctor AI

Average trust, twelve decisions (0 to 10)

8.1

4.1

Study 2, 106 adults.

People see a doctor as better at both kinds of thinking

Good doctors combine numbers with judgment. They read the lab result, and they also sense when something does not add up. In Study 1, 188 adults rated doctors higher than an AI on both.

People who rated a recommender higher on either skill also trusted it more. In Study 2, these ratings explained about half of the doctor's edge. The other half is still unexplained.

Doctor AI

Can use quantitative reasoning

7.2

5.2

Can use intuition

6.7

4.5

Study 1, ratings from 0 to 10.

Making an AI seem more human narrowed the gap, but only when people could compare

I gave the same health advice to everyone. In one version it came from a doctor named Jordan. In another it came from an AI called JO76N. In a third it came from an AI also named Jordan, who spoke like a person.

People who saw only one version did not trust the humanlike AI more. People who saw all three did, and it closed most of the distance to the doctor. Being able to compare changed how the AI was judged.

Seen alone (Study 3) Seen side by side (Study 5)

How much more people trusted the humanlike AI than the standard AI

0.1

0.5

Effect size (d): 0 is no difference, 0.5 is a medium one. Studies 3 to 5 included about 2,000 people.

Trust did not change what people did

In the last study, 560 people were recruited for 15 days of sleep logging, and 455 completed it. A doctor or an AI told them to sleep 7.5 to 9 hours and to spend less time on their phone in bed.

I found no difference in how closely people followed the advice, whoever gave it. Saying you trust a doctor more does not mean you act on the doctor's advice more. Still, the advice was low stakes, so this may not hold when the cost of acting is higher.

No gap

in following the advice from a doctor or an AI

Study 6, 15 days.

What this means. The gap in trust is large and steady, and it tracks how capable people think the advice-giver is. A small nudge can move it: describing an AI in humanlike terms was enough. That cuts both ways. It can help a good AI get a fair hearing, and it can make a weak one look better than it is. And because trust did not change behavior in my one test, I would measure what people do, not only what they say.

Deep dive: the evidence, study by study

Everything below shows the data behind these findings. Each chart has hover details, and the full analyses are on separate pages for Study 1, Study 2, Study 3, Study 4 and Study 5.

Plain-language guide to the terms

Trust: How willing someone is to follow the advice. In these studies, people rated this from 0 (not at all) to 10 (completely).
Quantitative reasoning: Deciding with numbers, data and statistics, like reading a lab result.
Intuition: Deciding with experience and judgment, like a doctor sensing that something is off when the tests look fine.
Effect size (Cohen's d): How big a difference is, on a scale that can be compared across studies. This site uses the Funder and Ozer (2019) labels for psychology: about 0.20 is small, 0.41 is medium, 0.63 is large, and 0.87 or more is very large.
95% interval: The range of values the true average probably falls in. A narrow range means a precise estimate; a range that includes zero means the difference may not be real.
p value: How surprising the result would be if there were no real difference. A p below .05 is the usual bar for calling a result reliable, and "p < .001" means chance is a very unlikely explanation.
Mixed model: A way to analyze data when each person gives many ratings, so that people who rate everything high or low do not distort the results.
Mediation: A test of whether one thing explains part of the link between two others. Here: does seeing a doctor as more competent explain part of why people trust the doctor more?
Preregistered: The plan for the study and its analysis was written and posted publicly before anyone looked at the data.

The question

Medical AI has shown diagnostic and prescriptive ability that rivals doctors. Yet when a doctor and an AI give people the same recommendation, people trust the doctor more. I wanted to know what drives that gap, and whether it can be closed.

The gap holds across medical domains

Study 2 asked 106 adults how much they trusted a doctor and an AI making the same medical judgment. Hover over a row or tab to it for the exact statistics.

AI trusted more Doctor trusted more

View as a table

Domain	Difference	95% CI	t	df	Cohen's d
Physical health	3.98	3.40 to 4.57	13.55	103	1.33
Mental health	3.91	3.30 to 4.53	12.67	103	1.24
Bodily injury	3.97	3.38 to 4.56	13.32	103	1.31
Acute sickness	3.93	3.33 to 4.54	12.85	104	1.25
Chronic sickness	4.00	3.40 to 4.60	13.2	104	1.29

Source: Leitao, M. R. (2024). Artificial Intelligence, Humanization, and Health Beliefs. Doctoral dissertation, Georgetown University. Study 2, Table 7. Preregistration and data and analysis are on OSF.

Who people trust, from doctors down to chatbots

In Study 1, 188 people rated how much they would trust each of eleven recommenders for medical decisions. Doctors and nurses sit at the top. Every AI sits below the human experts, near the level people gave a layperson. A doctor who uses AI lands in between.

Human
Doctor using AI
AI
Bars are 95% confidence intervals

View as a table

Recommender	Type	Mean	SD	95% CI	n
Doctor	Human	7.45	2.60	7.07 to 7.82	188
Nurse	Human	6.85	2.62	6.47 to 7.22	188
Doctor, remote visit	Human	6.14	2.80	5.73 to 6.54	187
Therapist	Human	6.03	2.85	5.62 to 6.44	188
Doctor using AI	Doctor using AI	5.65	3.07	5.20 to 6.09	187
Personal trainer	Human	5.32	2.71	4.92 to 5.71	187
AI, general	AI	4.69	3.06	4.24 to 5.13	185
AI diagnostic tool	AI	4.42	3.01	3.99 to 4.86	184
Layperson	Human	4.34	3.11	3.89 to 4.79	186
AI fitness coach	AI	4.30	3.07	3.86 to 4.75	187
AI chatbot	AI	4.25	3.12	3.80 to 4.70	186

Source: Study 1 of the dissertation (N = 188). Trust in each recommender, rated from 0 (not at all) to 10 (completely). Means with 95% confidence intervals, computed from the released study data. Hover over a row or tab to it for the details.

Making the AI humanlike helped only when people compared

I described one AI in a more humanlike way, with a name and human verbs (plus a voice from Study 4 on), and gave the same advice in every condition. Each row is a study. When each person saw one recommender, the humanlike AI did not gain significantly. When the same people judged all three side by side, it did.

Doctor versus standard AI
Humanlike AI versus standard AI
Hollow means not significant

View as a table

Study	Comparison	d	Test
Study 1	Doctor v standard AI	0.79	t(184) = 10.73, p < .001
Study 3	Doctor v standard AI	0.29	t(253.27) = 2.62, p = .009
Study 3	Humanlike v standard AI	0.07	t(477.11) = 0.76, p = .446
Study 4, between	Doctor v standard AI	0.31	t(264.44) = 2.73, p = .007
Study 4, between	Humanlike v standard AI	0.13	t(425.02) = 1.32, p = .188
Study 4, within	Doctor v standard AI	0.41	t(503) = 9.28, p < .001
Study 4, within	Humanlike v standard AI	0.36	t(503) = 7.98, p < .001
Study 5, within	Doctor v standard AI	0.50	t(726) = 13.44, p < .001
Study 5, within	Humanlike v standard AI	0.46	t(726) = 12.52, p < .001

Source: Computed from the released data for Studies 3, 4 and 5 (analysis/study3-full-results.py, study4-full-results.py, study5-full-results.py) and Study 1 as before. Trust in Studies 1, 3 and 4; combined trust and belief in Study 5. Hover over a row or tab to it for the test statistics.

How I measured it

I collected trust ratings and a behavioral measure called weight on advice, which records how far people move their own judgment toward a recommendation. Every study was preregistered.

What this means for product teams

If a health AI is trusted less than a clinician, the gap persisted across every medical domain I tested, so adjusting the content alone may not close it. Test how people perceive the decision-maker, and measure what they do, not only what they say they trust.

Doctors beat AI on trust, intuition and quantitative reasoning

Study 1 asked 185 people to rate a doctor and an AI on three things: how much they trust each to make health recommendations, and how capable each is of using quantitative reasoning and of using intuition. The doctor came out ahead on all three. The largest gap is trust (d = 0.79).

Doctor
AI
Bars are 95% confidence intervals

Source: Study 1 (N = 188), full sample. Means and 95% confidence intervals of each rating, doctor and AI, on a 0 to 10 scale. The statistics match the dissertation (Hypotheses 6 and 7 and the exploratory test). Hover over a row or tab to it for the details.

Chapter 2: Is the gap real, and what is behind it?

The first chapter of the experiments asks a plain question. When a doctor and a medical AI are judged on the same health decision, do people trust the doctor more, and is that true for every kind of decision? Both studies take the view of an ordinary patient, using broad medical decisions rather than one specific scenario.

The chapter also tests a first explanation. Clinical expertise blends quantitative reasoning (data and statistics) with intuition (experience and judgment), and an AI is usually thought to have only the first. If people trust doctors more because they credit them with both, ratings of these two abilities should track trust.

Study 1 (188 adults) rated fifteen decisions and eleven recommenders, from a doctor to an AI fitness coach. Study 2 (106 adults) replicated the gap on twelve concrete decisions, rated a doctor and an AI side by side, and added warmth, competence and mind perception. Together they set up Chapters 3 and 4, where I try to close the gap by making an AI seem more human.

Filter the data and rerun the results

The cleaned Study 1 data and the analysis code are public, as part of my commitment to open science. Choose who to include and the chart and tests below update on the spot, using the same tests as the paper. With no filters, the table reproduces the dissertation. Nothing here identifies a participant: I removed IP addresses, locations, timestamps and IDs, left out sensitive demographics, grouped the rest into broad categories, and shuffled the participant numbers.

Loading the data.

Pick more than one option in a row to include both. Choices in different rows combine, so Woman and 60+ means women aged 60 and over. Results appear only when at least 10 people match.

Doctor
AI
Bars are 95% confidence intervals

The tests, rerun for this group

Hypothesis	Means	Difference	t (df)	p	d

How this group rated each recommender

Rating Trust Quantitative reasoning Intuition

Compare with

Each bar counts people who gave that rating. Hover over, or tab to, a bar for the count. The strip shows how many people rated the first option higher, the same, or lower.

Download the data (CSV) Codebook (CSV) Analysis code (R)

Paired t tests on the released data. Decision items are on a 0 to 100 scale, recommender ratings on 0 to 10. The paper's hypothesis numbers are used; the exploratory test (E) was not preregistered. How the file was cleaned is in the project README. The data, code, preregistration and original analysis notebook are all linked from the full Study 1 page.

Every table and figure from Study 1

Descriptives, reliability, factor analysis, all eight tests, the eleven-recommender maps and the mixed models, rebuilt from the public data. Each figure has hover details and a table view.

Open Study 1 · every table and figure · humans versus AI · dig deeper

Every table and figure from Study 2

Study 2's 106 people rated trust in a doctor and in an AI for twelve decisions, and the two on warmth, competence, intuition and quantitative reasoning. The page rebuilds the tests, the mixed models and the mediation analysis from the public file, with hover details and table views. The de-identified data, codebook, R script, original notebook and preregistration are linked from the Study 2 page; the study is also on OSF (preregistration, data and analysis).

Open Study 2 · every table and figure · dig deeper

Every table and figure from Studies 3, 4 and 5

Each study has an overview, a page with every table and figure rebuilt from the released data, and a page with a figure map and Bayes factors. The de-identified data, codebooks, R scripts, original analysis notebooks and OSF preregistrations are linked from each overview. Studies 3 to 5 are also on OSF (Study 3, Study 4, Study 5).

Study 3 (609 adults) · Study 4 (559 adults) · Study 5 (854 adults)

Study 3 in full · Study 4 in full · Study 5 in full

Dig deeper: Studies 3 to 6

Studies 1 to 5 have their own pages, built from the released data. Study 6 does not have raw data here yet, so for that study this section does what the numbers in the dissertation allow. It lists where each figure and table sits, and it runs Bayes factors on the comparisons the dissertation reports.

Where each dissertation figure and table stands

In the dissertation	What it shows	On this site	Notes
Figures 13, 14, 17, 22, 28, 29	Vignette images, conceptual models and the sleep infographic	In the dissertation and on OSF	Materials, not data; not redrawn here
Figures 15 and 16	Study 3: mean trust and belief by recommender, with 95% intervals	Study 3, Figures 2, 3 and 6
Figures 18 to 21	Study 4: trust and belief, between people and side by side	Study 4, Figures 2 to 12
Figures 23 and 24	Study 5: trust and belief by recommender, with and without a voice	Study 5, Figures 4 and 8 to 10
Figures 25 to 27	Study 5: mediation through mind perception	Study 5, Figures 15 to 18
Figures 30 to 33	Study 6: adherence to the sleep and phone-in-bed advice across days, and mediation	Waiting on the Study 6 data
Tables 14 to 19	Studies 3 and 4: regression and mixed-model tables	Study 3 and Study 4 pages	Age is in four bands in the public files, so age coefficients differ in scale
Tables 20 to 22	Study 5: ANOVA and mixed-model tables	Study 5, Tables 5 and 12 to 14
Tables 23 and 24	Study 6: adherence tables	Waiting on the Study 6 data
All t tests in the text	Planned and follow-up comparisons in Studies 3 to 6	Bayes factors below	Studies 3 to 5 from the raw data; Study 6 from the reported t, d and degrees of freedom

When the test found no difference

A p value can only say whether a difference was detected. When it was not, it cannot tell you whether the difference is truly absent or whether the study was too small to see it. A Bayes factor can. It asks how many times more likely the data are under one story than under the other: there is no difference, or there is a difference of a plausible size. If the data are 10 times more likely when there is no difference, that is a Bayes factor of 10 for no difference.

The second story needs a guess about "a plausible size," so I ran it three ways: a small, a medium and a large effect, using the benchmarks Funder and Ozer (2019) proposed for psychology. The guess is a Cauchy prior, a bell-shaped curve centered on zero with long tails, so smaller effects are most likely and larger ones are not ruled out. Each row of a chart shows all three. When the three markers sit close together, the conclusion does not depend on the guess. When they spread out, it does.

Effect	Correlation (r)	Cohen's d	What it means in daily life
Very small	.05	0.10	Hard to see in one person, but it can add up over many people or many years
Small	.10	0.20	Noticeable only when you look across many cases
Medium	.20	0.41	A typical finding in published psychology, and a real difference in everyday life
Large	.30	0.63	Rare in psychology, and clearly visible
Very large	.40 or more	0.87 or more	Uncommon, and often an overestimate

These benchmarks are more realistic than the older 0.2, 0.5 and 0.8 rule of thumb, which describes effects most psychology studies never reach. This site labels every effect size with them. For ratings from the same person, d is the mean difference divided by the spread of the differences, and I convert it to r the way you would for two equal groups, so the labels are a guide.

Times more likely	How to describe the evidence
1 to 3	Anecdotal: barely worth mentioning
3 to 10	Moderate
10 to 30	Strong
30 to 100	Very strong
Over 100	Extreme

The same words apply in both directions. Data that are 12 times more likely under no difference are strong evidence for no difference, and data 12 times more likely under a difference are strong evidence for a difference. Near 1, the data cannot tell the two stories apart.

Small, medium and large effect (the labelled dot is medium)
Data favor no difference
Data favor a difference

View as a table

Test	If the effect is small	If medium	If large
Trust: humanlike AI versus standard AI	2.5 times for no difference	4.5 times for no difference	6.7 times for no difference
Trust: doctor versus standard AI	4.9 times for a difference	4.3 times for a difference	3.4 times for a difference
Trust: humanlike AI versus doctor	1.7 times for a difference	1.3 times for a difference	about equally likely for no difference
Belief: humanlike AI versus standard AI	3.1 times for no difference	5.6 times for no difference	8.5 times for no difference
Belief: doctor versus standard AI	2.9 times for a difference	2.4 times for a difference	1.8 times for a difference
Belief: humanlike AI versus doctor	1.9 times for a difference	1.4 times for a difference	about equally likely for a difference
Trust: humanlike AI versus standard AI	1.6 times for no difference	2.5 times for no difference	3.6 times for no difference
Trust: doctor versus standard AI	6.2 times for a difference	5.7 times for a difference	4.6 times for a difference
Trust: humanlike AI versus doctor	about equally likely for a difference	1.4 times for no difference	2 times for no difference
Belief: humanlike AI versus standard AI	about equally likely for no difference	1.7 times for no difference	2.5 times for no difference
Belief: doctor versus standard AI	4.1 times for a difference	3.6 times for a difference	2.8 times for a difference
Belief: humanlike AI versus doctor	1.7 times for no difference	2.6 times for no difference	3.8 times for no difference
Trust: humanlike AI versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Trust: doctor versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Trust: humanlike AI versus doctor	1.3 times for a difference	1.3 times for no difference	2 times for no difference
Belief: humanlike AI versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Belief: doctor versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Belief: humanlike AI versus doctor	2.1 times for no difference	4 times for no difference	6 times for no difference
Doctor: voiced versus written	2.4 times for no difference	4.3 times for no difference	6.5 times for no difference
Humanlike AI: voiced versus written	2.9 times for a difference	2 times for a difference	1.4 times for a difference
Standard AI: voiced versus written	1.7 times for a difference	about equally likely for a difference	1.2 times for no difference
Humanlike AI versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Doctor versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Humanlike AI versus doctor	118 times for a difference	75 times for a difference	52 times for a difference
Voice, across all recommenders	2.6 times for no difference	4.6 times for no difference	6.8 times for no difference
Voice effect for the standard AI versus the humanlike AI	2.6 times for no difference	4.6 times for no difference	6.9 times for no difference
Voice effect for the doctor versus the humanlike AI	2.5 times for no difference	4.4 times for no difference	6.5 times for no difference
Doctor versus humanlike AI after adjusting for mind, warmth, competence	2.2 times for no difference	3.8 times for no difference	5.7 times for no difference
Mind perception as a mediator: doctor versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Mind perception as a mediator: humanlike AI versus standard AI	over a billion times for a difference	over a billion times for a difference	over a billion times for a difference
Mind perception as a mediator: humanlike AI versus doctor	5.7 times for no difference	12 times for no difference	18 times for no difference
Sleep: change after the advice, AI versus doctor	4.4 times for no difference	8.5 times for no difference	13 times for no difference
Sleep: AI versus doctor, overall	5.3 times for no difference	10 times for no difference	16 times for no difference
Phone in bed: change after the advice, AI versus doctor	2.3 times for no difference	4.3 times for no difference	6.5 times for no difference
Phone in bed: AI versus doctor, overall	4.6 times for no difference	9 times for no difference	14 times for no difference
Sleep: mind perception as a mediator	14 times for no difference	28 times for no difference	42 times for no difference
Phone in bed: mind perception as a mediator	14 times for no difference	29 times for no difference	44 times for no difference

Humanizing the AI, seen alone, did not help. In Study 3 the data are 4.5 times more likely under no difference between the humanlike and the standard AI for trust and 5.7 times more likely for belief, moderate evidence. Study 4 adds a voice and the picture softens to 2.5 and 1.7 times, which leans toward no effect without settling it.

The two voice effects are suggestive, not firm. In Study 5 a voice raised trust in the humanlike AI (p = .017) and lowered it for the standard AI (p = .036), but the data are only 2 times more likely under a difference for the first, and about equally likely either way for the second. That is close to a coin flip. A doctor's recommendation was not changed by a voice (the data are 4.4 times more likely under no change), and voice overall made no difference (4.6 times more likely under none).

The humanlike AI and the doctor are hard to separate between people, and the doctor is ahead side by side. Seen alone (Studies 3 and 4), the evidence about humanlike AI versus doctor is anecdotal either way. Seen side by side in Study 4 the gap is small and the evidence stays anecdotal. In Study 5, with more people, the doctor was rated above the humanlike AI, and the data are 75 times more likely under a difference, very strong evidence for a small gap. Once warmth, competence and mind perception are accounted for, the data favor no remaining gap (3.8 times more likely).

The recommender did not change what people did. In Study 6 the change in sleep and in phone use after the advice was the same for the AI and the doctor. The data are 8.6 times more likely under no difference for sleep and 4.3 times more likely for phone use in bed, moderate evidence. This is a stronger statement than the dissertation could make from the p values alone. The mediation results (28 times more likely and 29 times more likely, under no indirect effect) come from a bell-shaped approximation for indirect effects, so treat them as a guide.

Bayes factors for Studies 3 to 5 come from the raw data. For Study 6 they come from the reported statistics: the effective sample size of a two-group test is recovered as (t / d)², and coefficients use the estimate and the width of the reported interval. The dissertation prints some values with rounding, so the Study 6 Bayes factors are accurate to roughly the first digit.

How many people does a finding need?

Power is the chance a study detects a real effect of a given size. Slide the effect size or pick one of the Study 1 results to see how many people it takes, and how well Study 1's 188 people could detect it. This is the same reason the filters above stop at 10 people: small groups can only detect large differences, so a real but modest difference often looks like nothing.

Effect size (d) 0.41 Significance level

Chance of detecting it 80% 90%

Use a Study 1 result

Chosen effect
Small (0.20), medium (0.41), large (0.63)
Study 1 sample size

View as a table

Power for a two-sided paired t test, the test used for every comparison in Study 1, computed in your browser from the noncentral t distribution and checked against scipy. This is a sensitivity view: it shows what size of effect the study could detect, not how likely the results already seen were. The H labels use the dissertation's numbering.

Main findings

People trust a doctor more than an AI, for every kind of decision

Doctors win, but the AI is not a zero

People see a doctor as better at both kinds of thinking

Making an AI seem more human narrowed the gap, but only when people could compare

Trust did not change what people did

Deep dive: the evidence, study by study

The question

The gap holds across medical domains

Who people trust, from doctors down to chatbots

Making the AI humanlike helped only when people compared

How I measured it

What this means for product teams

Doctors beat AI on trust, intuition and quantitative reasoning

Filter the data and rerun the results

The tests, rerun for this group

How this group rated each recommender

Every table and figure from Study 1

Every table and figure from Study 2

Every table and figure from Studies 3, 4 and 5

Dig deeper: Studies 3 to 6

Where each dissertation figure and table stands

When the test found no difference

How many people does a finding need?

More charts for this page