Study 1 · Trust in medical AI

Study 1: every table and figure

Each analysis from the dissertation, rebuilt from the released data, with a figure beside every table. Hover over any mark for the details.

Deep dive: every table and figure

The rest of this page rebuilds each analysis from the public data. Hover over any mark for the details, and open the guide below if a term is unfamiliar.

Plain-language guide to the terms
Trust
How willing someone is to follow the advice. In these studies, people rated this from 0 (not at all) to 10 (completely).
Quantitative reasoning
Deciding with numbers, data and statistics, like reading a lab result.
Intuition
Deciding with experience and judgment, like a doctor sensing that something is off when the tests look fine.
Effect size (Cohen's d)
How big a difference is, on a scale that can be compared across studies. This site uses the Funder and Ozer (2019) labels for psychology: about 0.20 is small, 0.41 is medium, 0.63 is large, and 0.87 or more is very large.
95% interval
The range of values the true average probably falls in. A narrow range means a precise estimate; a range that includes zero means the difference may not be real.
p value
How surprising the result would be if there were no real difference. A p below .05 is the usual bar for calling a result reliable, and "p < .001" means chance is a very unlikely explanation.
Mixed model
A way to analyze data when each person gives many ratings, so that people who rate everything high or low do not distort the results.
Mediation
A test of whether one thing explains part of the link between two others. Here: does seeing a doctor as more competent explain part of why people trust the doctor more?
Preregistered
The plan for the study and its analysis was written and posted publicly before anyone looked at the data.

How to read this page. The seven preregistered tests (H1 to H7) and one exploratory test (E) match the dissertation. Everything else was run in my exploratory analysis notebook and is recomputed here from the public file. The public file groups demographics into broad categories, so the sample table is coarser than the one in the dissertation.

Who took part

The study was preregistered with a target of 186 people recruited through CloudResearch's Prime Panels. After removing preview responses and anyone who did not consent, 188 remain.

Table 1. The people in the released file

What to look for. The share in each group. Ages are spread fairly evenly across the four bands, and women are the larger group.

Group People Share
Age: 18-295529.3%
Age: 30-445127.1%
Age: 45-594222.3%
Age: 60+3920.7%
Gender: Woman11360.1%
Gender: Man7439.4%
Race: White12264.9%
Race: Black4423.4%
Race: Another or multiple1910.1%
Education: No college degree11661.7%
Education: College degree or more7238.3%
Political views: Moderate7439.4%
Political views: Liberal5830.9%
Political views: Conservative5629.8%
Experience with AI: None or a little11963.3%
Experience with AI: Moderate or more6936.7%

Shares are of all 188 people. Left blank in the file: age (1), gender (1), race (3). Gender had only woman and man responses, so smaller groups do not appear. Race is White or Black only when that is the only group chosen; every other answer, including choosing more than one, is combined because no other group reached 30 people.

Table 1 as a figure

Each bar is the share of the people in that group. Longer means more of the sample. Hover a bar for the head count.

0%20%40%60%80%100% Age 18-29 (n = 55) 30-44 (n = 51) 45-59 (n = 42) 60+ (n = 39) Gender Woman (n = 113) Man (n = 74) Race White (n = 122) Black (n = 44) Another or multiple (n = 19) Education No college degree (n = 116) College degree or more (n = 72) Political views Moderate (n = 74) Liberal (n = 58) Conservative (n = 56) Experience with AI None or a little (n = 119) Moderate or more (n = 69) Share of people (%)

Fifteen medical decisions

For each decision, people slid from 0 (a professional needs only intuition and personal experience) to 100 (needs only quantitative reasoning). Every decision fell above the middle except general wellbeing advice, which sat just below it.

Table 2. Quantitative reasoning needed, by decision (0 to 100)

What to look for. Every decision sits above the middle of the 0 to 100 scale except general wellbeing advice, which sits just below it.

Measure n Mean SD Median Min Max Skew Kurtosis SE
Diagnose diabetes18665.0425.7669.160.00100.00−0.60−0.341.89
Prescribe insulin and a diet change18664.7227.8569.680.00100.00−0.63−0.562.04
Recommend knee surgery18664.3126.1365.840.00100.00−0.50−0.521.92
Diagnose a broken bone18763.6325.6463.260.00100.00−0.41−0.371.88
Prescribe antibiotics18662.3527.0062.970.00100.00−0.36−0.801.98
Prescribe pain medication18660.7225.4758.000.00100.00−0.24−0.521.87
Medical decisions in general18660.0724.1358.120.00100.00−0.32−0.321.77
Prescribe antidepressants and therapy18659.5826.0260.480.00100.00−0.27−0.811.91
Diagnose strep throat18659.0826.2857.300.00100.00−0.20−0.771.93
Diagnose depression18557.6325.4756.890.00100.00−0.23−0.621.87
Recommend stretching18754.4924.7753.060.00100.00−0.24−0.451.81
Recommend a strict diet18652.2726.8351.700.00100.00−0.10−0.861.97
Recommend vitamin C and zinc18650.7327.3450.750.00100.00−0.03−1.012.00
Recommend meditation18650.5227.7951.140.00100.00−0.03−0.922.04
Recommend general wellbeing habits18649.0326.7449.410.00100.000.10−0.841.96

Sorted from most to least quantitative reasoning needed. Skew and kurtosis are the estimates psych::describe reports.

Figure 1. Every decision, every person

Each row is one of the fifteen decisions. Every dot is one person's answer on a 0 to 100 slider, from needs only intuition (0) to needs only quantitative reasoning (100). The curve shows where answers pile up.

Diagnose diabetes 65.0 Prescribe insulin and a diet change 64.7 Recommend knee surgery 64.3 Diagnose a broken bone 63.6 Prescribe antibiotics 62.4 Prescribe pain medication 60.7 Medical decisions in general 60.1 Prescribe antidepressants and therapy 59.6 Diagnose strep throat 59.1 Diagnose depression 57.6 Recommend stretching 54.5 Recommend a strict diet 52.3 Recommend vitamin C and zinc 50.7 Recommend meditation 50.5 Recommend general wellbeing habits 49.0 020406080100 Needs intuition (0) versus needs quantitative reasoning (100)

Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.

View as a table
MeasurenMean95% interval
Diagnose diabetes18665.061.3 to 68.7
Prescribe insulin and a diet change18664.760.8 to 68.6
Recommend knee surgery18664.360.6 to 68.1
Diagnose a broken bone18763.659.8 to 67.2
Prescribe antibiotics18662.458.6 to 66.1
Prescribe pain medication18660.757.0 to 64.4
Medical decisions in general18660.156.5 to 63.5
Prescribe antidepressants and therapy18659.655.7 to 63.4
Diagnose strep throat18659.155.3 to 62.6
Diagnose depression18557.654.2 to 61.1
Recommend stretching18754.551.0 to 58.1
Recommend a strict diet18652.348.5 to 56.2
Recommend vitamin C and zinc18650.746.8 to 54.6
Recommend meditation18650.546.6 to 54.4
Recommend general wellbeing habits18649.045.1 to 52.9
Table 3. The decision groups used in the tests (0 to 100)

What to look for. The same ratings averaged into the groups used later. Compare the means across rows; preventative decisions need the least quantitative reasoning.

Measure n Mean SD Median Min Max Skew Kurtosis SE
Diagnostic (4 decisions)18561.4118.9162.570.00100.00−0.450.491.39
Prescriptive (4 decisions)18661.8420.2163.060.00100.00−0.550.451.48
Preventative (4 decisions)18651.9820.4452.172.15100.00−0.12−0.361.50
Physical health (9 decisions)18659.2516.9760.704.63100.00−0.470.801.24
Mental health (3 decisions)18555.8419.7855.500.00100.00−0.060.151.45
Bodily injury (3 decisions)18659.6917.6559.081.71100.00−0.050.421.29
Acute sickness (3 decisions)18657.3920.4658.760.00100.00−0.440.181.50
Chronic sickness (3 decisions)18660.6820.7762.540.00100.00−0.610.261.52

Each group is the average of its decisions. A person missing any one decision in a group is missing for that group, so n runs from 185 to 186.

Figure 2. By kind of health problem

The same slider ratings, grouped by kind of health problem. Each dot is one person; the curve shows where answers cluster.

Chronic sickness 60.7 Bodily injury 59.7 Physical health 59.3 Acute sickness 57.4 Mental health 55.8 020406080100 Intuition (0) versus quantitative reasoning (100)

Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.

View as a table
MeasurenMean95% interval
Chronic sickness18660.757.6 to 63.5
Bodily injury18659.757.3 to 62.3
Physical health18659.356.8 to 61.6
Acute sickness18657.454.4 to 60.3
Mental health18555.853.2 to 58.6

Figure 3. By stage of care

The same slider ratings, grouped by stage of care: preventing, diagnosing or prescribing.

Prescriptive 61.8 Diagnostic 61.4 Preventative 52.0 020406080100 Intuition (0) versus quantitative reasoning (100)

Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.

View as a table
MeasurenMean95% interval
Prescriptive18661.858.8 to 64.6
Diagnostic18561.458.8 to 64.0
Preventative18652.049.0 to 54.7

Do the decision ratings hang together?

Before averaging decisions into groups, I checked how closely the items in each group move together. Alpha runs from 0 to 1; higher means people answered the items in a group alike. The bodily injury group is the weakest (alpha .47), so results that rest on it deserve more caution.

Table 4. Reliability of each group of decisions

What to look for. Alpha near or above .80 means the items in a group can safely be averaged. The bodily injury group is the weakest, so lean on it less.

Group Items Cronbach’s alpha Average item correlation
Diagnostic decisions4.71.38
Prescriptive decisions4.76.44
Preventative decisions4.76.45
Physical health decisions9.82.34
Mental health decisions3.61.34
Bodily injury decisions3.47.23
Acute sickness decisions3.64.37
Chronic sickness decisions3.67.40

Table 4 as a figure

Each dot is the alpha for one group of decisions: how alike people answered the items inside it. The dashed lines mark .70, .80 and .90; above .70 is usually taken as acceptable and above .80 as good.

.70.80.90 0.500.600.700.800.901.00 Diagnostic decisions (4 items) Prescriptive decisions (4 items) Preventative decisions (4 items) Physical health decisions (9 items) Mental health decisions (3 items) Bodily injury decisions (3 items) Acute sickness decisions (3 items) Chronic sickness decisions (3 items) Cronbach's alpha
Item correlations behind each alpha
Diagnostic decisions
Item 1 2 3 4
1. Diagnose a broken bone.265.521.520
2. Diagnose depression.265.170.281
3. Diagnose strep throat.521.170.548
4. Diagnose diabetes.520.281.548
Prescriptive decisions
Item 1 2 3 4
1. Prescribe pain medication.475.429.401
2. Prescribe antidepressants and therapy.475.423.365
3. Prescribe antibiotics.429.423.529
4. Prescribe insulin and a diet change.401.365.529
Preventative decisions
Item 1 2 3 4
1. Recommend stretching.466.408.429
2. Recommend meditation.466.473.489
3. Recommend vitamin C and zinc.408.473.418
4. Recommend a strict diet.429.489.418
Physical health decisions
Item 1 2 3 4 5 6 7 8 9
1. Diagnose a broken bone.521.520.125.240.250.416.580.477
2. Diagnose strep throat.521.548.184.157.271.331.706.460
3. Diagnose diabetes.520.548.019.103.226.455.542.650
4. Recommend stretching.125.184.019.408.429.151.084.107
5. Recommend vitamin C and zinc.240.157.103.408.418.227.253.226
6. Recommend a strict diet.250.271.226.429.418.265.274.321
7. Prescribe pain medication.416.331.455.151.227.265.429.401
8. Prescribe antibiotics.580.706.542.084.253.274.429.529
9. Prescribe insulin and a diet change.477.460.650.107.226.321.401.529
Mental health decisions
Item 1 2 3
1. Diagnose depression.219.588
2. Recommend meditation.219.229
3. Prescribe antidepressants and therapy.588.229
Bodily injury decisions
Item 1 2 3
1. Diagnose a broken bone.125.416
2. Recommend stretching.125.151
3. Prescribe pain medication.416.151
Acute sickness decisions
Item 1 2 3
1. Diagnose strep throat.157.706
2. Recommend vitamin C and zinc.157.253
3. Prescribe antibiotics.706.253
Chronic sickness decisions
Item 1 2 3
1. Diagnose diabetes.226.650
2. Recommend a strict diet.226.321
3. Prescribe insulin and a diet change.650.321

Figure 4. How many factors?

A scree plot lists the strength of each possible factor in the twelve decisions that make up the groups (185 people answered all twelve). Three factors stand above 1, the usual cut-off, and a fourth falls short.

Eigenvalue of 1 123456789101112 012345 Number of factors Eigenvalue
View as a table
FactorEigenvalue
14.55
21.99
31.19
40.68
50.64
60.60
70.59
80.48
90.46
100.31
110.28
120.24

Table 5. A three-factor solution

Three factors, allowed to correlate, sort the twelve decisions by what they ask of a clinician. The names are my reading of the pattern, not part of the analysis. Pain medication for a knee injury is the one decision that splits across factors.

Decision Factor 1Diagnosing and treating physical illnessFactor 2Preventing illness and promoting healthFactor 3Treating mental health Shared
Prescribe antibiotics 0.82 0.01 −0.02 0.67
Diagnose strep throat 0.81 0.05 −0.12 0.67
Diagnose diabetes 0.69 −0.10 0.19 0.53
Prescribe insulin and a diet change 0.67 0.02 0.06 0.45
Diagnose a broken bone 0.62 0.03 0.13 0.41
Prescribe pain medication 0.35 0.11 0.32 0.23
Recommend meditation −0.13 0.73 0.12 0.56
Recommend stretching 0.03 0.70 −0.13 0.51
Recommend a strict diet 0.21 0.61 −0.02 0.42
Recommend vitamin C and zinc 0.05 0.59 0.12 0.37
Prescribe antidepressants and therapy 0.06 −0.03 0.92 0.85
Diagnose depression −0.07 0.19 0.61 0.41
Variance explained 24%15%12% 51%

Factors correlate 0.24 (1 and 2), 0.47 (1 and 3) and 0.23 (2 and 3). Bold loadings are 0.30 or larger.

Three factors, minimum residual extraction and oblimin rotation, on the 12 decisions that form the composites (everyone with all 12 answers). The notebook asked for minchi extraction, which gives the same answer when every correlation rests on the same number of people. Loadings are the pattern matrix.

Figure 4b. The factor diagram

The same solution as a diagram, as the notebook’s factor diagram drew it. Each decision points to the factor it loads on most (solid line); dashed lines are smaller loadings of .30 or more. The curves give the correlations between factors.

.24 .23 .47 .82.81.69.67.62.35.32.73.70.61.59.92.61 Prescribe antibiotics Diagnose strep throat Diagnose diabetes Prescribe insulin and a diet change Diagnose a broken bone Prescribe pain medication Recommend meditation Recommend stretching Recommend a strict diet Recommend vitamin C and zinc Prescribe antidepressants and therapy Diagnose depression
Diagnosing and treating physical illness
Preventing illness and promoting health
Treating mental health

The tests

The preregistered plan was a series of paired t tests, since each person answered every item. H1 to H5 compare kinds of decision; H6 and H7 compare a doctor with an AI. The AI here is the general medical AI. Test E asks the same of quantitative reasoning and was not preregistered.

Table 6. Paired t tests

What to look for. The difference column and p. H1 shows no gap, H2 and H3 show a clear gap, and H6, H7 and E show the doctor rated well above the AI.

Test Comparison Means (SD) Difference [95% CI] t (df) p d [95% CI]
H1Diagnostic vs prescriptive61.41 (18.91), 61.79 (20.25)−0.37 [-2.16, 1.41]−0.41 (184).680−0.03 [-0.17, 0.11]
H2Diagnostic vs preventative61.41 (18.91), 51.90 (20.47)9.51 [6.15, 12.87]5.58 (184)< .0010.41 [0.26, 0.56]
H3Prescriptive vs preventative61.84 (20.21), 51.98 (20.44)9.86 [6.42, 13.30]5.66 (185)< .0010.42 [0.26, 0.56]
H4Physical vs mental health59.21 (17.01), 55.84 (19.78)3.37 [0.80, 5.94]2.59 (184).0100.19 [0.04, 0.34]
H5Bodily injury vs acute sickness59.69 (17.65), 57.39 (20.46)2.31 [-0.10, 4.72]1.89 (185).0610.14 [-0.01, 0.28]
H6Doctor vs AI: intuition6.73 (2.93), 4.51 (3.04)2.22 [1.67, 2.78]7.92 (187)< .0010.58 [0.42, 0.73]
H7Doctor vs AI: trust7.50 (2.55), 4.69 (3.06)2.81 [2.29, 3.33]10.73 (184)< .0010.79 [0.62, 0.95]
EDoctor vs AI: quantitative reasoning (exploratory)7.22 (2.95), 5.20 (3.18)2.02 [1.50, 2.54]7.63 (187)< .0010.56 [0.40, 0.71]

H1 to H5 compare how much quantitative reasoning decisions need (0 to 100); H6, H7 and E compare ratings of a doctor and an AI (0 to 10). A positive difference means the first item was rated higher. Tests are two-sided at alpha = .05, the criterion in the preregistration.

Figure 5. Person as effect size

An average can hide how many people actually differ. For each comparison, this counts people who rated the first item higher, the same, or lower. Even for the largest gap, trust in a doctor over an AI, 14% of people trusted the AI more and 19% rated the two the same. The notebook counted this for trust (doctor versus AI, and layperson versus AI); I ran the same count for every test.

H1. Diagnostic vs prescriptive 48% 50% H2. Diagnostic vs preventative 66% 32% H3. Prescriptive vs preventative 65% 33% H4. Physical vs mental health 56% 43% H5. Bodily injury vs acute sickness 55% 43% H6. Intuition: doctor vs AI 59% 23% H7. Trust: doctor vs AI 67% 14% E. Quantitative reasoning: doctor vs AI 59% 20% X1. Trust: layperson vs AI 36% 39% X2. Intuition: layperson vs AI 39% 31% X3. AI: quantitative reasoning vs intuition 41% 32% X4. Trust: doctor vs doctor using AI 56% 21%
  • First item rated higher
  • Rated the same
  • Second item rated higher
  • Doctor is blue, AI is coral, wherever they appear
Table 7. Follow-up comparisons from the notebook

What to look for. Which comparisons show a clear gap and which do not: look at whether the interval for the difference stays away from zero.

Test Comparison Means (SD) Difference [95% CI] t (df) p d [95% CI]
X1Trust: layperson versus AI4.34 (3.10), 4.67 (3.06)−0.33 [-0.75, 0.10]−1.51 (183).132−0.11 [-0.26, 0.03]
X2Intuition: layperson versus AI4.92 (3.16), 4.51 (3.04)0.41 [-0.01, 0.84]1.93 (187).0550.14 [0.00, 0.28]
X3AI: quantitative reasoning versus intuition5.20 (3.18), 4.51 (3.04)0.69 [0.24, 1.14]3.05 (187).0030.22 [0.08, 0.37]
X4Trust: doctor versus doctor using AI7.49 (2.55), 5.65 (3.07)1.84 [1.34, 2.34]7.20 (186)< .0010.53 [0.37, 0.68]

Exploratory comparisons the notebook ran after the preregistered tests: a layperson against the AI, the AI's quantitative reasoning against its intuition, and a doctor against a doctor using AI.

Eleven recommenders

Each person rated all eleven recommenders on three things, each from 0 to 10: how much they trust it to make health recommendations, how capable it is of quantitative reasoning, and how capable it is of intuition. A doctor using AI is treated as AI in the models later on, as in the notebook.

Table 8. Ratings of each recommender, mean (SD)

What to look for. Read down the trust column. Doctors and nurses sit near the top, the AI recommenders sit below the human experts, and a doctor using AI sits in between.

Recommender Trust Quantitative reasoning Intuition n (trust)
Doctor7.45 (2.60)7.22 (2.95)6.73 (2.93)188
Nurse6.85 (2.62)6.44 (2.76)6.54 (2.70)188
Doctor, remote visit6.14 (2.80)5.95 (2.84)6.05 (2.90)187
Therapist6.03 (2.85)5.90 (2.78)6.23 (2.81)188
Doctor using AI5.65 (3.07)5.78 (2.96)5.16 (3.15)187
Personal trainer5.32 (2.71)5.05 (2.72)5.40 (2.95)187
AI, general4.69 (3.06)5.20 (3.18)4.51 (3.04)185
AI diagnostic tool4.42 (3.01)5.19 (3.14)4.32 (3.01)184
Layperson4.34 (3.11)4.37 (2.97)4.92 (3.16)186
AI fitness coach4.30 (3.07)4.81 (3.06)4.04 (2.99)187
AI chatbot4.25 (3.12)4.85 (3.09)4.16 (3.09)186

Sorted from most to least trusted.

Full descriptives for each rating
Trust (0 to 10)
Measure n Mean SD Median Min Max Skew Kurtosis SE
Doctor1887.452.608.000.0010.00−0.78−0.520.19
Nurse1886.852.628.000.0010.00−0.55−0.730.19
Doctor, remote visit1876.142.806.000.0010.00−0.41−0.630.21
Therapist1886.032.856.000.0010.00−0.46−0.530.21
Doctor using AI1875.653.076.000.0010.00−0.33−0.940.22
Personal trainer1875.322.715.000.0010.00−0.04−0.720.20
AI, general1854.693.064.000.0010.000.19−0.980.23
AI diagnostic tool1844.423.014.500.0010.000.23−1.000.22
Layperson1864.343.114.000.0010.000.24−1.020.23
AI fitness coach1874.303.074.000.0010.000.24−1.020.22
AI chatbot1864.253.124.000.0010.000.25−1.140.23
Quantitative reasoning (0 to 10)
Measure n Mean SD Median Min Max Skew Kurtosis SE
Doctor1887.222.958.000.0010.00−0.78−0.580.22
Nurse1886.442.767.000.0010.00−0.33−0.900.20
Doctor, remote visit1885.952.846.000.0010.00−0.27−0.920.21
Therapist1885.902.786.000.0010.00−0.18−0.840.20
Doctor using AI1885.782.966.000.0010.00−0.24−0.900.22
Personal trainer1885.052.725.000.0010.000.11−0.810.20
AI, general1885.203.185.000.0010.000.02−1.220.23
AI diagnostic tool1885.193.145.000.0010.000.03−1.060.23
Layperson1884.372.974.000.0010.000.19−0.950.22
AI fitness coach1874.813.065.000.0010.000.06−1.070.22
AI chatbot1884.853.095.000.0010.000.03−1.050.23
Intuition (0 to 10)
Measure n Mean SD Median Min Max Skew Kurtosis SE
Doctor1886.732.937.000.0010.00−0.54−0.820.21
Nurse1886.542.706.500.0010.00−0.40−0.590.20
Doctor, remote visit1876.052.906.000.0010.00−0.30−0.870.21
Therapist1876.232.816.000.0010.00−0.33−0.670.21
Doctor using AI1885.163.155.000.0010.00−0.04−1.120.23
Personal trainer1885.402.955.000.0010.00−0.03−0.940.22
AI, general1884.513.044.000.0010.000.15−0.970.22
AI diagnostic tool1874.323.014.000.0010.000.26−0.890.22
Layperson1884.923.165.000.0010.00−0.04−1.180.23
AI fitness coach1884.042.994.000.0010.000.32−0.870.22
AI chatbot1884.163.094.000.0010.000.27−0.980.23

Figure 6. Trust in each recommender, every person

Each row is a recommender. Every dot is one person's trust rating on a 0 to 10 scale, and the curve shows where ratings cluster. Higher means more trust.

Doctor 7.5 Nurse 6.8 Doctor, remote visit 6.1 Therapist 6.0 Doctor using AI 5.7 Personal trainer 5.3 AI, general 4.7 AI diagnostic tool 4.4 Layperson 4.3 AI fitness coach 4.3 AI chatbot 4.3 0246810 Trust, from 0 (not at all) to 10 (completely)

Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.

View as a table
MeasurenMean95% interval
Doctor1887.57.1 to 7.8
Nurse1886.86.5 to 7.2
Doctor, remote visit1876.15.8 to 6.5
Therapist1886.05.6 to 6.4
Doctor using AI1875.75.2 to 6.1
Personal trainer1875.34.9 to 5.7
AI, general1854.74.3 to 5.1
AI diagnostic tool1844.44.0 to 4.8
Layperson1864.33.9 to 4.8
AI fitness coach1874.33.9 to 4.8
AI chatbot1864.33.8 to 4.7

Figure 7. Where each recommender sits

Doctors sit highest on both kinds of capability. Every AI is seen as less capable of intuition than of quantitative reasoning, and sits below the doctor on both. A doctor using AI lands between the doctor and the AIs.

Equally capable of both Doctor Nurse Doctor, remote visit Therapist Personal trainer Layperson Doctor using AI AI, general AI diagnostic tool AI chatbot AI fitness coach 34567 34567 Capable of intuition (0 to 10) Capable of quantitative reasoning (0 to 10)
  • Human
  • Doctor using AI
  • AI
  • Bars are 95% confidence intervals
View as a table
RecommenderTypeIntuitionQuantitative reasoningn
DoctorHuman6.73 ± 0.427.22 ± 0.42188
NurseHuman6.54 ± 0.396.44 ± 0.39188
Doctor, remote visitHuman6.05 ± 0.425.95 ± 0.41188
TherapistHuman6.23 ± 0.405.90 ± 0.40188
Personal trainerHuman5.40 ± 0.425.05 ± 0.39188
LaypersonHuman4.92 ± 0.454.37 ± 0.43188
Doctor using AIDoctor using AI5.16 ± 0.455.78 ± 0.42188
AI, generalAI4.51 ± 0.445.20 ± 0.45188
AI diagnostic toolAI4.32 ± 0.435.19 ± 0.45188
AI chatbotAI4.16 ± 0.444.85 ± 0.44188
AI fitness coachAI4.04 ± 0.434.81 ± 0.44187

What goes with trust

Trust tracks both kinds of perceived capability. These analyses are exploratory, and they show what moves together, not what causes what. First, one recommender at a time.

Table 9. Trust predicted by intuition and quantitative reasoning, for three recommenders

What to look for. The β column. Both intuition and quantitative reasoning go with trust for all three recommenders; for the doctor quantitative reasoning is stronger, and for the AI intuition is.

Recommender and predictor b SE t p β
Doctor: Intercept2.800.397.21< .001
Doctor: Intuition0.260.064.12< .001.30
Doctor: Quantitative reasoning0.400.066.32< .001.45
AI, general: Intercept0.850.332.60.010
AI, general: Intuition0.490.068.20< .001.49
AI, general: Quantitative reasoning0.310.065.36< .001.32
Layperson: Intercept0.580.321.83.069
Layperson: Intuition0.450.076.83< .001.45
Layperson: Quantitative reasoning0.360.075.17< .001.34

Table 9 as a figure

Each dot is how strongly a rating goes with trust, in standardized units (β), with a 95% interval. Dashed lines mark a small, medium and large effect on the Funder and Ozer scale. Intervals that stay right of zero mean the rating reliably goes with trust.

smallmediumlarge 0.000.100.200.300.400.500.600.700.80 Doctor Intuition Quantitative reasoning AI, general Intuition Quantitative reasoning Layperson Intuition Quantitative reasoning Standardized effect on trust (β)
Table 10. How well each model fits

What to look for. R² is the share of the variation in trust the two ratings explain. About half, so the ratings matter but are not the whole story.

Recommender n R² Adjusted R² F (df) p
Doctor188.47.4782.57 (2, 185)< .001
AI, general185.51.5093.72 (2, 182)< .001
Layperson186.51.5094.02 (2, 183)< .001

Table 10 as a figure

Each bar is the share of the variation in trust that the two ratings explain (R²). A bar at 50% means the ratings account for half of why people trusted one recommender more than another.

0%20%40%60%80%100% Doctor AI, general Layperson Variation in trust explained (R², %)
Table 11. Correlations among the three ratings

What to look for. Every correlation is positive: people who rate a recommender high on one thing tend to rate it high on the others.

Recommender and pair r p n
Doctor: Trust and Quantitative reasoning.650< .001188
Doctor: Trust and Intuition.598< .001188
Doctor: Quantitative reasoning and Intuition.667< .001188
AI, general: Trust and Quantitative reasoning.571< .001185
AI, general: Trust and Intuition.655< .001185
AI, general: Quantitative reasoning and Intuition.503< .001185
Layperson: Trust and Quantitative reasoning.617< .001186
Layperson: Trust and Intuition.659< .001186
Layperson: Quantitative reasoning and Intuition.615< .001186

Table 11 as a figure

Each square is the correlation between two ratings for one recommender. Darker blue means the two move together more strongly.

Doctor TrustQuantitativereasoningIntuition Trust .65 .60 Quantitative reasoning .65 .67 Intuition .60 .67
AI, general TrustQuantitativereasoningIntuition Trust .57 .66 Quantitative reasoning .57 .50 Intuition .66 .50
Layperson TrustQuantitativereasoningIntuition Trust .62 .66 Quantitative reasoning .62 .61 Intuition .66 .61

Mixed models across all eleven recommenders

Each person rated eleven recommenders, so their ratings are not independent. These models give every person their own baseline level of trust. The AI recommenders are the four AI tools plus a doctor using AI, as in the notebook.

Table 12. The models side by side

What to look for. Marginal R² jumps from .05 to .52 once the two ratings enter. A lower AIC means a better balance of fit and simplicity.

Model Marginal R² Conditional R² Person share AIC
1. AI versus human only.05.44.419691
2. Quantitative reasoning and intuition.52.64.258640
3. Add AI versus human.51.64.268610
4. Let the slopes differ for AI.51.64.268594
5. Between and within people.52.64.258642
6. Between and within people, plus AI.53.65.268609
7. AI recommenders only.43.68.443855
8. Human recommenders only.51.64.254708

Random intercept for each person, restricted maximum likelihood, 188 people with 11 ratings each. Tests are Wald z tests. Standardized betas are b times the SD of the predictor over the SD of trust. Model comparisons refit by maximum likelihood. The notebook's last model (3D) uses grand-mean variables the notebook never creates, so it is not reproduced here. A doctor using AI counts as AI, as in the notebook.

Table 12 as a figure. Each bar is the share of the variation in trust a model explains: first by its predictors alone, then with each person's own baseline added. Longer is a better fit.

0%20%40%60%80%100% Predictors alone (marginal R²) 1. AI versus human only 2. Quantitative reasoning and intuition 3. Add AI versus human 4. Let the slopes differ for AI 5. Between and within people 6. Between and within people, plus AI 7. AI recommenders only 8. Human recommenders only Predictors plus each person (conditional R²) 1. AI versus human only 2. Quantitative reasoning and intuition 3. Add AI versus human 4. Let the slopes differ for AI 5. Between and within people 6. Between and within people, plus AI 7. AI recommenders only 8. Human recommenders only Variation in trust explained (%)
Coefficients for each model
Model 1. AI versus human only
Predictor b SE 95% CI z p β
Intercept6.020.165.71 to 6.3338.20< .001
AI recommender (vs human)−1.370.10−1.58 to -1.17−13.31< .001−.22

2,051 ratings from 188 people. Person share .41.

Model 2. Quantitative reasoning and intuition
Predictor b SE 95% CI z p β
Intercept1.030.140.76 to 1.317.48< .001
Quantitative reasoning0.370.020.34 to 0.4119.63< .001.37
Intuition0.440.020.40 to 0.4723.92< .001.44

2,051 ratings from 188 people. Person share .25.

Model 3. Add AI versus human
Predictor b SE 95% CI z p β
Intercept1.420.151.12 to 1.729.23< .001
Quantitative reasoning0.370.020.34 to 0.4119.79< .001.37
Intuition0.410.020.37 to 0.4421.48< .001.41
AI recommender (vs human)−0.490.09−0.66 to -0.32−5.71< .001−.08

2,051 ratings from 188 people. Person share .26.

Model 4. Let the slopes differ for AI
Predictor b SE 95% CI z p β
Intercept1.450.171.11 to 1.798.38< .001
Quantitative reasoning0.440.030.39 to 0.4917.70< .001.43
AI recommender (vs human)−0.430.19−0.79 to -0.06−2.26.024−.07
Quantitative reasoning × AI−0.140.04−0.21 to -0.07−4.10< .001
Intuition0.330.030.28 to 0.3913.00< .001.34
Intuition × AI0.140.040.07 to 0.203.89< .001

2,051 ratings from 188 people. Person share .26.

Model 5. Between and within people
Predictor b SE 95% CI z p β
Intercept0.920.260.41 to 1.423.56< .001
Quantitative reasoning, person average0.310.070.18 to 0.434.66< .001.21
Quantitative reasoning, vs own average0.380.020.34 to 0.4118.63< .001.27
Intuition, person average0.530.070.40 to 0.668.06< .001.36
Intuition, vs own average0.430.020.39 to 0.4722.29< .001.32

2,051 ratings from 188 people. Person share .25.

Model 6. Between and within people, plus AI
Predictor b SE 95% CI z p β
Intercept1.150.260.64 to 1.664.40< .001
Quantitative reasoning, person average0.310.070.18 to 0.434.65< .001.21
Quantitative reasoning, vs own average0.370.020.33 to 0.4118.67< .001.26
Intuition, person average0.530.070.40 to 0.668.02< .001.36
Intuition, vs own average0.390.020.35 to 0.4319.59< .001.29
AI recommender (vs human)−0.510.09−0.68 to -0.34−5.90< .001−.08

2,051 ratings from 188 people. Person share .26.

Model 7. AI recommenders only
Predictor b SE 95% CI z p β
Intercept1.360.200.97 to 1.756.84< .001
Quantitative reasoning0.270.030.21 to 0.339.25< .001.27
Intuition0.430.030.37 to 0.4914.98< .001.43

929 ratings from 187 people. Person share .44.

Model 8. Human recommenders only
Predictor b SE 95% CI z p β
Intercept1.310.180.96 to 1.667.39< .001
Quantitative reasoning0.450.030.40 to 0.4917.40< .001.45
Intuition0.350.030.30 to 0.4113.19< .001.35

1,122 ratings from 188 people. Person share .25.

Table 13. Does adding a term improve the fit?

What to look for. Every p is below .001, so each added term improved the model.

Comparison χ² df p
Model 2 to Model 3: add AI versus human32.321< .001
Model 3 to Model 4: let the slopes differ for AI20.212< .001
Model 5 to Model 6: add AI versus human34.551< .001

Likelihood ratio tests, refit by maximum likelihood.

Table 13 as a figure. Each bar is how much a model improves when a term is added (χ²). Longer means a bigger improvement; with one degree of freedom, anything over 3.8 is past the usual cutoff.

p = .05 0.010.020.030.040.0 Model 2 to Model 3: add AI versus human Model 3 to Model 4: let the slopes differ for AI Model 5 to Model 6: add AI versus human Improvement in fit (χ²)

Figure 8. What lifts trust differs for AI and for humans

For human recommenders, perceived quantitative reasoning is the stronger partner of trust (b = 0.45 against 0.35 for intuition). For AI, it is intuition (b = 0.43 against 0.27). The interaction model agrees: the two slopes differ by recommender type (Model 4: both interaction terms, p < .001).

Human recommenders Quantitative reasoning: +0.45 per point Intuition: +0.35 per point 0246810 Rating of the recommender (0 to 10) AI recommenders Quantitative reasoning: +0.27 per point Intuition: +0.43 per point 0246810 Rating of the recommender (0 to 10) 0246810 Predicted trust (0 to 10)
  • Quantitative reasoning
  • Intuition
  • The other rating is held at its average. Bands are 95% confidence intervals.

Predicted trust from the AI-only and human-only models, with the other rating held at its average. Bands are 95% confidence intervals for the fixed effects.

Table 14. Correlations between people and within people

What to look for. Compare the two columns. Between-people values reflect who the person is; within-people values reflect which recommender is being rated.

Recommenders and pair Between people, r (p) Within people, r (p)
All recommenders: Trust and Intuition.776 (< .001).562 (< .001)
All recommenders: Trust and Quantitative reasoning.726 (< .001).519 (< .001)
All recommenders: Intuition and Quantitative reasoning.769 (< .001).384 (< .001)
AI recommenders: Trust and Intuition.781 (< .001).391 (< .001)
AI recommenders: Trust and Quantitative reasoning.663 (< .001).283 (< .001)
AI recommenders: Intuition and Quantitative reasoning.614 (< .001).224 (< .001)
Human recommenders: Trust and Intuition.711 (< .001).526 (< .001)
Human recommenders: Trust and Quantitative reasoning.725 (< .001).592 (< .001)
Human recommenders: Intuition and Quantitative reasoning.744 (< .001).451 (< .001)

Between-person correlations use each person's average over the recommenders. Within-person correlations use each rating's distance from that person's own average, pooled across people.

Table 14 as a figure, for all recommenders together. The left grid is between people: do people who rate recommenders high on one thing rate them high on the other? The right grid is within people: for one person, do recommenders rated higher on one thing also get rated higher on the other?

Between people TrustIntuitionQuantitativereasoning Trust .78 .73 Intuition .78 .77 Quantitative reasoning .73 .77
Within people TrustIntuitionQuantitativereasoning Trust .56 .52 Intuition .56 .38 Quantitative reasoning .52 .38