Study 1: every table and figure
Each analysis from the dissertation, rebuilt from the released data, with a figure beside every table. Hover over any mark for the details.
Deep dive: every table and figure
The rest of this page rebuilds each analysis from the public data. Hover over any mark for the details, and open the guide below if a term is unfamiliar.
Plain-language guide to the terms
- Trust
- How willing someone is to follow the advice. In these studies, people rated this from 0 (not at all) to 10 (completely).
- Quantitative reasoning
- Deciding with numbers, data and statistics, like reading a lab result.
- Intuition
- Deciding with experience and judgment, like a doctor sensing that something is off when the tests look fine.
- Effect size (Cohen's d)
- How big a difference is, on a scale that can be compared across studies. This site uses the Funder and Ozer (2019) labels for psychology: about 0.20 is small, 0.41 is medium, 0.63 is large, and 0.87 or more is very large.
- 95% interval
- The range of values the true average probably falls in. A narrow range means a precise estimate; a range that includes zero means the difference may not be real.
- p value
- How surprising the result would be if there were no real difference. A p below .05 is the usual bar for calling a result reliable, and "p < .001" means chance is a very unlikely explanation.
- Mixed model
- A way to analyze data when each person gives many ratings, so that people who rate everything high or low do not distort the results.
- Mediation
- A test of whether one thing explains part of the link between two others. Here: does seeing a doctor as more competent explain part of why people trust the doctor more?
- Preregistered
- The plan for the study and its analysis was written and posted publicly before anyone looked at the data.
How to read this page. The seven preregistered tests (H1 to H7) and one exploratory test (E) match the dissertation. Everything else was run in my exploratory analysis notebook and is recomputed here from the public file. The public file groups demographics into broad categories, so the sample table is coarser than the one in the dissertation.
Who took part
The study was preregistered with a target of 186 people recruited through CloudResearch's Prime Panels. After removing preview responses and anyone who did not consent, 188 remain.
What to look for. The share in each group. Ages are spread fairly evenly across the four bands, and women are the larger group.
| Group | People | Share |
|---|---|---|
| Age: 18-29 | 55 | 29.3% |
| Age: 30-44 | 51 | 27.1% |
| Age: 45-59 | 42 | 22.3% |
| Age: 60+ | 39 | 20.7% |
| Gender: Woman | 113 | 60.1% |
| Gender: Man | 74 | 39.4% |
| Race: White | 122 | 64.9% |
| Race: Black | 44 | 23.4% |
| Race: Another or multiple | 19 | 10.1% |
| Education: No college degree | 116 | 61.7% |
| Education: College degree or more | 72 | 38.3% |
| Political views: Moderate | 74 | 39.4% |
| Political views: Liberal | 58 | 30.9% |
| Political views: Conservative | 56 | 29.8% |
| Experience with AI: None or a little | 119 | 63.3% |
| Experience with AI: Moderate or more | 69 | 36.7% |
Shares are of all 188 people. Left blank in the file: age (1), gender (1), race (3). Gender had only woman and man responses, so smaller groups do not appear. Race is White or Black only when that is the only group chosen; every other answer, including choosing more than one, is combined because no other group reached 30 people.
Table 1 as a figure
Each bar is the share of the people in that group. Longer means more of the sample. Hover a bar for the head count.
Fifteen medical decisions
For each decision, people slid from 0 (a professional needs only intuition and personal experience) to 100 (needs only quantitative reasoning). Every decision fell above the middle except general wellbeing advice, which sat just below it.
What to look for. Every decision sits above the middle of the 0 to 100 scale except general wellbeing advice, which sits just below it.
| Measure | n | Mean | SD | Median | Min | Max | Skew | Kurtosis | SE |
|---|---|---|---|---|---|---|---|---|---|
| Diagnose diabetes | 186 | 65.04 | 25.76 | 69.16 | 0.00 | 100.00 | −0.60 | −0.34 | 1.89 |
| Prescribe insulin and a diet change | 186 | 64.72 | 27.85 | 69.68 | 0.00 | 100.00 | −0.63 | −0.56 | 2.04 |
| Recommend knee surgery | 186 | 64.31 | 26.13 | 65.84 | 0.00 | 100.00 | −0.50 | −0.52 | 1.92 |
| Diagnose a broken bone | 187 | 63.63 | 25.64 | 63.26 | 0.00 | 100.00 | −0.41 | −0.37 | 1.88 |
| Prescribe antibiotics | 186 | 62.35 | 27.00 | 62.97 | 0.00 | 100.00 | −0.36 | −0.80 | 1.98 |
| Prescribe pain medication | 186 | 60.72 | 25.47 | 58.00 | 0.00 | 100.00 | −0.24 | −0.52 | 1.87 |
| Medical decisions in general | 186 | 60.07 | 24.13 | 58.12 | 0.00 | 100.00 | −0.32 | −0.32 | 1.77 |
| Prescribe antidepressants and therapy | 186 | 59.58 | 26.02 | 60.48 | 0.00 | 100.00 | −0.27 | −0.81 | 1.91 |
| Diagnose strep throat | 186 | 59.08 | 26.28 | 57.30 | 0.00 | 100.00 | −0.20 | −0.77 | 1.93 |
| Diagnose depression | 185 | 57.63 | 25.47 | 56.89 | 0.00 | 100.00 | −0.23 | −0.62 | 1.87 |
| Recommend stretching | 187 | 54.49 | 24.77 | 53.06 | 0.00 | 100.00 | −0.24 | −0.45 | 1.81 |
| Recommend a strict diet | 186 | 52.27 | 26.83 | 51.70 | 0.00 | 100.00 | −0.10 | −0.86 | 1.97 |
| Recommend vitamin C and zinc | 186 | 50.73 | 27.34 | 50.75 | 0.00 | 100.00 | −0.03 | −1.01 | 2.00 |
| Recommend meditation | 186 | 50.52 | 27.79 | 51.14 | 0.00 | 100.00 | −0.03 | −0.92 | 2.04 |
| Recommend general wellbeing habits | 186 | 49.03 | 26.74 | 49.41 | 0.00 | 100.00 | 0.10 | −0.84 | 1.96 |
Sorted from most to least quantitative reasoning needed. Skew and kurtosis are the estimates psych::describe reports.
Figure 1. Every decision, every person
Each row is one of the fifteen decisions. Every dot is one person's answer on a 0 to 100 slider, from needs only intuition (0) to needs only quantitative reasoning (100). The curve shows where answers pile up.
Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.
View as a table
| Measure | n | Mean | 95% interval |
|---|---|---|---|
| Diagnose diabetes | 186 | 65.0 | 61.3 to 68.7 |
| Prescribe insulin and a diet change | 186 | 64.7 | 60.8 to 68.6 |
| Recommend knee surgery | 186 | 64.3 | 60.6 to 68.1 |
| Diagnose a broken bone | 187 | 63.6 | 59.8 to 67.2 |
| Prescribe antibiotics | 186 | 62.4 | 58.6 to 66.1 |
| Prescribe pain medication | 186 | 60.7 | 57.0 to 64.4 |
| Medical decisions in general | 186 | 60.1 | 56.5 to 63.5 |
| Prescribe antidepressants and therapy | 186 | 59.6 | 55.7 to 63.4 |
| Diagnose strep throat | 186 | 59.1 | 55.3 to 62.6 |
| Diagnose depression | 185 | 57.6 | 54.2 to 61.1 |
| Recommend stretching | 187 | 54.5 | 51.0 to 58.1 |
| Recommend a strict diet | 186 | 52.3 | 48.5 to 56.2 |
| Recommend vitamin C and zinc | 186 | 50.7 | 46.8 to 54.6 |
| Recommend meditation | 186 | 50.5 | 46.6 to 54.4 |
| Recommend general wellbeing habits | 186 | 49.0 | 45.1 to 52.9 |
What to look for. The same ratings averaged into the groups used later. Compare the means across rows; preventative decisions need the least quantitative reasoning.
| Measure | n | Mean | SD | Median | Min | Max | Skew | Kurtosis | SE |
|---|---|---|---|---|---|---|---|---|---|
| Diagnostic (4 decisions) | 185 | 61.41 | 18.91 | 62.57 | 0.00 | 100.00 | −0.45 | 0.49 | 1.39 |
| Prescriptive (4 decisions) | 186 | 61.84 | 20.21 | 63.06 | 0.00 | 100.00 | −0.55 | 0.45 | 1.48 |
| Preventative (4 decisions) | 186 | 51.98 | 20.44 | 52.17 | 2.15 | 100.00 | −0.12 | −0.36 | 1.50 |
| Physical health (9 decisions) | 186 | 59.25 | 16.97 | 60.70 | 4.63 | 100.00 | −0.47 | 0.80 | 1.24 |
| Mental health (3 decisions) | 185 | 55.84 | 19.78 | 55.50 | 0.00 | 100.00 | −0.06 | 0.15 | 1.45 |
| Bodily injury (3 decisions) | 186 | 59.69 | 17.65 | 59.08 | 1.71 | 100.00 | −0.05 | 0.42 | 1.29 |
| Acute sickness (3 decisions) | 186 | 57.39 | 20.46 | 58.76 | 0.00 | 100.00 | −0.44 | 0.18 | 1.50 |
| Chronic sickness (3 decisions) | 186 | 60.68 | 20.77 | 62.54 | 0.00 | 100.00 | −0.61 | 0.26 | 1.52 |
Each group is the average of its decisions. A person missing any one decision in a group is missing for that group, so n runs from 185 to 186.
Figure 2. By kind of health problem
The same slider ratings, grouped by kind of health problem. Each dot is one person; the curve shows where answers cluster.
Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.
View as a table
| Measure | n | Mean | 95% interval |
|---|---|---|---|
| Chronic sickness | 186 | 60.7 | 57.6 to 63.5 |
| Bodily injury | 186 | 59.7 | 57.3 to 62.3 |
| Physical health | 186 | 59.3 | 56.8 to 61.6 |
| Acute sickness | 186 | 57.4 | 54.4 to 60.3 |
| Mental health | 185 | 55.8 | 53.2 to 58.6 |
Figure 3. By stage of care
The same slider ratings, grouped by stage of care: preventing, diagnosing or prescribing.
Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.
View as a table
| Measure | n | Mean | 95% interval |
|---|---|---|---|
| Prescriptive | 186 | 61.8 | 58.8 to 64.6 |
| Diagnostic | 185 | 61.4 | 58.8 to 64.0 |
| Preventative | 186 | 52.0 | 49.0 to 54.7 |
Do the decision ratings hang together?
Before averaging decisions into groups, I checked how closely the items in each group move together. Alpha runs from 0 to 1; higher means people answered the items in a group alike. The bodily injury group is the weakest (alpha .47), so results that rest on it deserve more caution.
What to look for. Alpha near or above .80 means the items in a group can safely be averaged. The bodily injury group is the weakest, so lean on it less.
| Group | Items | Cronbach’s alpha | Average item correlation |
|---|---|---|---|
| Diagnostic decisions | 4 | .71 | .38 |
| Prescriptive decisions | 4 | .76 | .44 |
| Preventative decisions | 4 | .76 | .45 |
| Physical health decisions | 9 | .82 | .34 |
| Mental health decisions | 3 | .61 | .34 |
| Bodily injury decisions | 3 | .47 | .23 |
| Acute sickness decisions | 3 | .64 | .37 |
| Chronic sickness decisions | 3 | .67 | .40 |
Table 4 as a figure
Each dot is the alpha for one group of decisions: how alike people answered the items inside it. The dashed lines mark .70, .80 and .90; above .70 is usually taken as acceptable and above .80 as good.
Item correlations behind each alpha
| Item | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| 1. Diagnose a broken bone | .265 | .521 | .520 | |
| 2. Diagnose depression | .265 | .170 | .281 | |
| 3. Diagnose strep throat | .521 | .170 | .548 | |
| 4. Diagnose diabetes | .520 | .281 | .548 |
| Item | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| 1. Prescribe pain medication | .475 | .429 | .401 | |
| 2. Prescribe antidepressants and therapy | .475 | .423 | .365 | |
| 3. Prescribe antibiotics | .429 | .423 | .529 | |
| 4. Prescribe insulin and a diet change | .401 | .365 | .529 |
| Item | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| 1. Recommend stretching | .466 | .408 | .429 | |
| 2. Recommend meditation | .466 | .473 | .489 | |
| 3. Recommend vitamin C and zinc | .408 | .473 | .418 | |
| 4. Recommend a strict diet | .429 | .489 | .418 |
| Item | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|
| 1. Diagnose a broken bone | .521 | .520 | .125 | .240 | .250 | .416 | .580 | .477 | |
| 2. Diagnose strep throat | .521 | .548 | .184 | .157 | .271 | .331 | .706 | .460 | |
| 3. Diagnose diabetes | .520 | .548 | .019 | .103 | .226 | .455 | .542 | .650 | |
| 4. Recommend stretching | .125 | .184 | .019 | .408 | .429 | .151 | .084 | .107 | |
| 5. Recommend vitamin C and zinc | .240 | .157 | .103 | .408 | .418 | .227 | .253 | .226 | |
| 6. Recommend a strict diet | .250 | .271 | .226 | .429 | .418 | .265 | .274 | .321 | |
| 7. Prescribe pain medication | .416 | .331 | .455 | .151 | .227 | .265 | .429 | .401 | |
| 8. Prescribe antibiotics | .580 | .706 | .542 | .084 | .253 | .274 | .429 | .529 | |
| 9. Prescribe insulin and a diet change | .477 | .460 | .650 | .107 | .226 | .321 | .401 | .529 |
| Item | 1 | 2 | 3 |
|---|---|---|---|
| 1. Diagnose depression | .219 | .588 | |
| 2. Recommend meditation | .219 | .229 | |
| 3. Prescribe antidepressants and therapy | .588 | .229 |
| Item | 1 | 2 | 3 |
|---|---|---|---|
| 1. Diagnose a broken bone | .125 | .416 | |
| 2. Recommend stretching | .125 | .151 | |
| 3. Prescribe pain medication | .416 | .151 |
| Item | 1 | 2 | 3 |
|---|---|---|---|
| 1. Diagnose strep throat | .157 | .706 | |
| 2. Recommend vitamin C and zinc | .157 | .253 | |
| 3. Prescribe antibiotics | .706 | .253 |
| Item | 1 | 2 | 3 |
|---|---|---|---|
| 1. Diagnose diabetes | .226 | .650 | |
| 2. Recommend a strict diet | .226 | .321 | |
| 3. Prescribe insulin and a diet change | .650 | .321 |
Figure 4. How many factors?
A scree plot lists the strength of each possible factor in the twelve decisions that make up the groups (185 people answered all twelve). Three factors stand above 1, the usual cut-off, and a fourth falls short.
View as a table
| Factor | Eigenvalue |
|---|---|
| 1 | 4.55 |
| 2 | 1.99 |
| 3 | 1.19 |
| 4 | 0.68 |
| 5 | 0.64 |
| 6 | 0.60 |
| 7 | 0.59 |
| 8 | 0.48 |
| 9 | 0.46 |
| 10 | 0.31 |
| 11 | 0.28 |
| 12 | 0.24 |
Table 5. A three-factor solution
Three factors, allowed to correlate, sort the twelve decisions by what they ask of a clinician. The names are my reading of the pattern, not part of the analysis. Pain medication for a knee injury is the one decision that splits across factors.
| Decision | Factor 1Diagnosing and treating physical illness | Factor 2Preventing illness and promoting health | Factor 3Treating mental health | Shared |
|---|---|---|---|---|
| Prescribe antibiotics | 0.82 | 0.01 | −0.02 | 0.67 |
| Diagnose strep throat | 0.81 | 0.05 | −0.12 | 0.67 |
| Diagnose diabetes | 0.69 | −0.10 | 0.19 | 0.53 |
| Prescribe insulin and a diet change | 0.67 | 0.02 | 0.06 | 0.45 |
| Diagnose a broken bone | 0.62 | 0.03 | 0.13 | 0.41 |
| Prescribe pain medication | 0.35 | 0.11 | 0.32 | 0.23 |
| Recommend meditation | −0.13 | 0.73 | 0.12 | 0.56 |
| Recommend stretching | 0.03 | 0.70 | −0.13 | 0.51 |
| Recommend a strict diet | 0.21 | 0.61 | −0.02 | 0.42 |
| Recommend vitamin C and zinc | 0.05 | 0.59 | 0.12 | 0.37 |
| Prescribe antidepressants and therapy | 0.06 | −0.03 | 0.92 | 0.85 |
| Diagnose depression | −0.07 | 0.19 | 0.61 | 0.41 |
| Variance explained | 24% | 15% | 12% | 51% |
Factors correlate 0.24 (1 and 2), 0.47 (1 and 3) and 0.23 (2 and 3). Bold loadings are 0.30 or larger.
Three factors, minimum residual extraction and oblimin rotation, on the 12 decisions that form the composites (everyone with all 12 answers). The notebook asked for minchi extraction, which gives the same answer when every correlation rests on the same number of people. Loadings are the pattern matrix.
Figure 4b. The factor diagram
The same solution as a diagram, as the notebook’s factor diagram drew it. Each decision points to the factor it loads on most (solid line); dashed lines are smaller loadings of .30 or more. The curves give the correlations between factors.
The tests
The preregistered plan was a series of paired t tests, since each person answered every item. H1 to H5 compare kinds of decision; H6 and H7 compare a doctor with an AI. The AI here is the general medical AI. Test E asks the same of quantitative reasoning and was not preregistered.
What to look for. The difference column and p. H1 shows no gap, H2 and H3 show a clear gap, and H6, H7 and E show the doctor rated well above the AI.
| Test | Comparison | Means (SD) | Difference [95% CI] | t (df) | p | d [95% CI] |
|---|---|---|---|---|---|---|
| H1 | Diagnostic vs prescriptive | 61.41 (18.91), 61.79 (20.25) | −0.37 [-2.16, 1.41] | −0.41 (184) | .680 | −0.03 [-0.17, 0.11] |
| H2 | Diagnostic vs preventative | 61.41 (18.91), 51.90 (20.47) | 9.51 [6.15, 12.87] | 5.58 (184) | < .001 | 0.41 [0.26, 0.56] |
| H3 | Prescriptive vs preventative | 61.84 (20.21), 51.98 (20.44) | 9.86 [6.42, 13.30] | 5.66 (185) | < .001 | 0.42 [0.26, 0.56] |
| H4 | Physical vs mental health | 59.21 (17.01), 55.84 (19.78) | 3.37 [0.80, 5.94] | 2.59 (184) | .010 | 0.19 [0.04, 0.34] |
| H5 | Bodily injury vs acute sickness | 59.69 (17.65), 57.39 (20.46) | 2.31 [-0.10, 4.72] | 1.89 (185) | .061 | 0.14 [-0.01, 0.28] |
| H6 | Doctor vs AI: intuition | 6.73 (2.93), 4.51 (3.04) | 2.22 [1.67, 2.78] | 7.92 (187) | < .001 | 0.58 [0.42, 0.73] |
| H7 | Doctor vs AI: trust | 7.50 (2.55), 4.69 (3.06) | 2.81 [2.29, 3.33] | 10.73 (184) | < .001 | 0.79 [0.62, 0.95] |
| E | Doctor vs AI: quantitative reasoning (exploratory) | 7.22 (2.95), 5.20 (3.18) | 2.02 [1.50, 2.54] | 7.63 (187) | < .001 | 0.56 [0.40, 0.71] |
H1 to H5 compare how much quantitative reasoning decisions need (0 to 100); H6, H7 and E compare ratings of a doctor and an AI (0 to 10). A positive difference means the first item was rated higher. Tests are two-sided at alpha = .05, the criterion in the preregistration.
Figure 5. Person as effect size
An average can hide how many people actually differ. For each comparison, this counts people who rated the first item higher, the same, or lower. Even for the largest gap, trust in a doctor over an AI, 14% of people trusted the AI more and 19% rated the two the same. The notebook counted this for trust (doctor versus AI, and layperson versus AI); I ran the same count for every test.
- First item rated higher
- Rated the same
- Second item rated higher
- Doctor is blue, AI is coral, wherever they appear
What to look for. Which comparisons show a clear gap and which do not: look at whether the interval for the difference stays away from zero.
| Test | Comparison | Means (SD) | Difference [95% CI] | t (df) | p | d [95% CI] |
|---|---|---|---|---|---|---|
| X1 | Trust: layperson versus AI | 4.34 (3.10), 4.67 (3.06) | −0.33 [-0.75, 0.10] | −1.51 (183) | .132 | −0.11 [-0.26, 0.03] |
| X2 | Intuition: layperson versus AI | 4.92 (3.16), 4.51 (3.04) | 0.41 [-0.01, 0.84] | 1.93 (187) | .055 | 0.14 [0.00, 0.28] |
| X3 | AI: quantitative reasoning versus intuition | 5.20 (3.18), 4.51 (3.04) | 0.69 [0.24, 1.14] | 3.05 (187) | .003 | 0.22 [0.08, 0.37] |
| X4 | Trust: doctor versus doctor using AI | 7.49 (2.55), 5.65 (3.07) | 1.84 [1.34, 2.34] | 7.20 (186) | < .001 | 0.53 [0.37, 0.68] |
Exploratory comparisons the notebook ran after the preregistered tests: a layperson against the AI, the AI's quantitative reasoning against its intuition, and a doctor against a doctor using AI.
Eleven recommenders
Each person rated all eleven recommenders on three things, each from 0 to 10: how much they trust it to make health recommendations, how capable it is of quantitative reasoning, and how capable it is of intuition. A doctor using AI is treated as AI in the models later on, as in the notebook.
What to look for. Read down the trust column. Doctors and nurses sit near the top, the AI recommenders sit below the human experts, and a doctor using AI sits in between.
| Recommender | Trust | Quantitative reasoning | Intuition | n (trust) |
|---|---|---|---|---|
| Doctor | 7.45 (2.60) | 7.22 (2.95) | 6.73 (2.93) | 188 |
| Nurse | 6.85 (2.62) | 6.44 (2.76) | 6.54 (2.70) | 188 |
| Doctor, remote visit | 6.14 (2.80) | 5.95 (2.84) | 6.05 (2.90) | 187 |
| Therapist | 6.03 (2.85) | 5.90 (2.78) | 6.23 (2.81) | 188 |
| Doctor using AI | 5.65 (3.07) | 5.78 (2.96) | 5.16 (3.15) | 187 |
| Personal trainer | 5.32 (2.71) | 5.05 (2.72) | 5.40 (2.95) | 187 |
| AI, general | 4.69 (3.06) | 5.20 (3.18) | 4.51 (3.04) | 185 |
| AI diagnostic tool | 4.42 (3.01) | 5.19 (3.14) | 4.32 (3.01) | 184 |
| Layperson | 4.34 (3.11) | 4.37 (2.97) | 4.92 (3.16) | 186 |
| AI fitness coach | 4.30 (3.07) | 4.81 (3.06) | 4.04 (2.99) | 187 |
| AI chatbot | 4.25 (3.12) | 4.85 (3.09) | 4.16 (3.09) | 186 |
Sorted from most to least trusted.
Full descriptives for each rating
| Measure | n | Mean | SD | Median | Min | Max | Skew | Kurtosis | SE |
|---|---|---|---|---|---|---|---|---|---|
| Doctor | 188 | 7.45 | 2.60 | 8.00 | 0.00 | 10.00 | −0.78 | −0.52 | 0.19 |
| Nurse | 188 | 6.85 | 2.62 | 8.00 | 0.00 | 10.00 | −0.55 | −0.73 | 0.19 |
| Doctor, remote visit | 187 | 6.14 | 2.80 | 6.00 | 0.00 | 10.00 | −0.41 | −0.63 | 0.21 |
| Therapist | 188 | 6.03 | 2.85 | 6.00 | 0.00 | 10.00 | −0.46 | −0.53 | 0.21 |
| Doctor using AI | 187 | 5.65 | 3.07 | 6.00 | 0.00 | 10.00 | −0.33 | −0.94 | 0.22 |
| Personal trainer | 187 | 5.32 | 2.71 | 5.00 | 0.00 | 10.00 | −0.04 | −0.72 | 0.20 |
| AI, general | 185 | 4.69 | 3.06 | 4.00 | 0.00 | 10.00 | 0.19 | −0.98 | 0.23 |
| AI diagnostic tool | 184 | 4.42 | 3.01 | 4.50 | 0.00 | 10.00 | 0.23 | −1.00 | 0.22 |
| Layperson | 186 | 4.34 | 3.11 | 4.00 | 0.00 | 10.00 | 0.24 | −1.02 | 0.23 |
| AI fitness coach | 187 | 4.30 | 3.07 | 4.00 | 0.00 | 10.00 | 0.24 | −1.02 | 0.22 |
| AI chatbot | 186 | 4.25 | 3.12 | 4.00 | 0.00 | 10.00 | 0.25 | −1.14 | 0.23 |
| Measure | n | Mean | SD | Median | Min | Max | Skew | Kurtosis | SE |
|---|---|---|---|---|---|---|---|---|---|
| Doctor | 188 | 7.22 | 2.95 | 8.00 | 0.00 | 10.00 | −0.78 | −0.58 | 0.22 |
| Nurse | 188 | 6.44 | 2.76 | 7.00 | 0.00 | 10.00 | −0.33 | −0.90 | 0.20 |
| Doctor, remote visit | 188 | 5.95 | 2.84 | 6.00 | 0.00 | 10.00 | −0.27 | −0.92 | 0.21 |
| Therapist | 188 | 5.90 | 2.78 | 6.00 | 0.00 | 10.00 | −0.18 | −0.84 | 0.20 |
| Doctor using AI | 188 | 5.78 | 2.96 | 6.00 | 0.00 | 10.00 | −0.24 | −0.90 | 0.22 |
| Personal trainer | 188 | 5.05 | 2.72 | 5.00 | 0.00 | 10.00 | 0.11 | −0.81 | 0.20 |
| AI, general | 188 | 5.20 | 3.18 | 5.00 | 0.00 | 10.00 | 0.02 | −1.22 | 0.23 |
| AI diagnostic tool | 188 | 5.19 | 3.14 | 5.00 | 0.00 | 10.00 | 0.03 | −1.06 | 0.23 |
| Layperson | 188 | 4.37 | 2.97 | 4.00 | 0.00 | 10.00 | 0.19 | −0.95 | 0.22 |
| AI fitness coach | 187 | 4.81 | 3.06 | 5.00 | 0.00 | 10.00 | 0.06 | −1.07 | 0.22 |
| AI chatbot | 188 | 4.85 | 3.09 | 5.00 | 0.00 | 10.00 | 0.03 | −1.05 | 0.23 |
| Measure | n | Mean | SD | Median | Min | Max | Skew | Kurtosis | SE |
|---|---|---|---|---|---|---|---|---|---|
| Doctor | 188 | 6.73 | 2.93 | 7.00 | 0.00 | 10.00 | −0.54 | −0.82 | 0.21 |
| Nurse | 188 | 6.54 | 2.70 | 6.50 | 0.00 | 10.00 | −0.40 | −0.59 | 0.20 |
| Doctor, remote visit | 187 | 6.05 | 2.90 | 6.00 | 0.00 | 10.00 | −0.30 | −0.87 | 0.21 |
| Therapist | 187 | 6.23 | 2.81 | 6.00 | 0.00 | 10.00 | −0.33 | −0.67 | 0.21 |
| Doctor using AI | 188 | 5.16 | 3.15 | 5.00 | 0.00 | 10.00 | −0.04 | −1.12 | 0.23 |
| Personal trainer | 188 | 5.40 | 2.95 | 5.00 | 0.00 | 10.00 | −0.03 | −0.94 | 0.22 |
| AI, general | 188 | 4.51 | 3.04 | 4.00 | 0.00 | 10.00 | 0.15 | −0.97 | 0.22 |
| AI diagnostic tool | 187 | 4.32 | 3.01 | 4.00 | 0.00 | 10.00 | 0.26 | −0.89 | 0.22 |
| Layperson | 188 | 4.92 | 3.16 | 5.00 | 0.00 | 10.00 | −0.04 | −1.18 | 0.23 |
| AI fitness coach | 188 | 4.04 | 2.99 | 4.00 | 0.00 | 10.00 | 0.32 | −0.87 | 0.22 |
| AI chatbot | 188 | 4.16 | 3.09 | 4.00 | 0.00 | 10.00 | 0.27 | −0.98 | 0.23 |
Figure 6. Trust in each recommender, every person
Each row is a recommender. Every dot is one person's trust rating on a 0 to 10 scale, and the curve shows where ratings cluster. Higher means more trust.
Outline: how answers are spread (smoothed). Dots: one person each. Diamond and bar: the average and its bootstrapped 95% interval.
View as a table
| Measure | n | Mean | 95% interval |
|---|---|---|---|
| Doctor | 188 | 7.5 | 7.1 to 7.8 |
| Nurse | 188 | 6.8 | 6.5 to 7.2 |
| Doctor, remote visit | 187 | 6.1 | 5.8 to 6.5 |
| Therapist | 188 | 6.0 | 5.6 to 6.4 |
| Doctor using AI | 187 | 5.7 | 5.2 to 6.1 |
| Personal trainer | 187 | 5.3 | 4.9 to 5.7 |
| AI, general | 185 | 4.7 | 4.3 to 5.1 |
| AI diagnostic tool | 184 | 4.4 | 4.0 to 4.8 |
| Layperson | 186 | 4.3 | 3.9 to 4.8 |
| AI fitness coach | 187 | 4.3 | 3.9 to 4.8 |
| AI chatbot | 186 | 4.3 | 3.8 to 4.7 |
Figure 7. Where each recommender sits
Doctors sit highest on both kinds of capability. Every AI is seen as less capable of intuition than of quantitative reasoning, and sits below the doctor on both. A doctor using AI lands between the doctor and the AIs.
- Human
- Doctor using AI
- AI
- Bars are 95% confidence intervals
View as a table
| Recommender | Type | Intuition | Quantitative reasoning | n |
|---|---|---|---|---|
| Doctor | Human | 6.73 ± 0.42 | 7.22 ± 0.42 | 188 |
| Nurse | Human | 6.54 ± 0.39 | 6.44 ± 0.39 | 188 |
| Doctor, remote visit | Human | 6.05 ± 0.42 | 5.95 ± 0.41 | 188 |
| Therapist | Human | 6.23 ± 0.40 | 5.90 ± 0.40 | 188 |
| Personal trainer | Human | 5.40 ± 0.42 | 5.05 ± 0.39 | 188 |
| Layperson | Human | 4.92 ± 0.45 | 4.37 ± 0.43 | 188 |
| Doctor using AI | Doctor using AI | 5.16 ± 0.45 | 5.78 ± 0.42 | 188 |
| AI, general | AI | 4.51 ± 0.44 | 5.20 ± 0.45 | 188 |
| AI diagnostic tool | AI | 4.32 ± 0.43 | 5.19 ± 0.45 | 188 |
| AI chatbot | AI | 4.16 ± 0.44 | 4.85 ± 0.44 | 188 |
| AI fitness coach | AI | 4.04 ± 0.43 | 4.81 ± 0.44 | 187 |
What goes with trust
Trust tracks both kinds of perceived capability. These analyses are exploratory, and they show what moves together, not what causes what. First, one recommender at a time.
What to look for. The β column. Both intuition and quantitative reasoning go with trust for all three recommenders; for the doctor quantitative reasoning is stronger, and for the AI intuition is.
| Recommender and predictor | b | SE | t | p | β |
|---|---|---|---|---|---|
| Doctor: Intercept | 2.80 | 0.39 | 7.21 | < .001 | |
| Doctor: Intuition | 0.26 | 0.06 | 4.12 | < .001 | .30 |
| Doctor: Quantitative reasoning | 0.40 | 0.06 | 6.32 | < .001 | .45 |
| AI, general: Intercept | 0.85 | 0.33 | 2.60 | .010 | |
| AI, general: Intuition | 0.49 | 0.06 | 8.20 | < .001 | .49 |
| AI, general: Quantitative reasoning | 0.31 | 0.06 | 5.36 | < .001 | .32 |
| Layperson: Intercept | 0.58 | 0.32 | 1.83 | .069 | |
| Layperson: Intuition | 0.45 | 0.07 | 6.83 | < .001 | .45 |
| Layperson: Quantitative reasoning | 0.36 | 0.07 | 5.17 | < .001 | .34 |
Table 9 as a figure
Each dot is how strongly a rating goes with trust, in standardized units (β), with a 95% interval. Dashed lines mark a small, medium and large effect on the Funder and Ozer scale. Intervals that stay right of zero mean the rating reliably goes with trust.
What to look for. R² is the share of the variation in trust the two ratings explain. About half, so the ratings matter but are not the whole story.
| Recommender | n | R² | Adjusted R² | F (df) | p |
|---|---|---|---|---|---|
| Doctor | 188 | .47 | .47 | 82.57 (2, 185) | < .001 |
| AI, general | 185 | .51 | .50 | 93.72 (2, 182) | < .001 |
| Layperson | 186 | .51 | .50 | 94.02 (2, 183) | < .001 |
Table 10 as a figure
Each bar is the share of the variation in trust that the two ratings explain (R²). A bar at 50% means the ratings account for half of why people trusted one recommender more than another.
What to look for. Every correlation is positive: people who rate a recommender high on one thing tend to rate it high on the others.
| Recommender and pair | r | p | n |
|---|---|---|---|
| Doctor: Trust and Quantitative reasoning | .650 | < .001 | 188 |
| Doctor: Trust and Intuition | .598 | < .001 | 188 |
| Doctor: Quantitative reasoning and Intuition | .667 | < .001 | 188 |
| AI, general: Trust and Quantitative reasoning | .571 | < .001 | 185 |
| AI, general: Trust and Intuition | .655 | < .001 | 185 |
| AI, general: Quantitative reasoning and Intuition | .503 | < .001 | 185 |
| Layperson: Trust and Quantitative reasoning | .617 | < .001 | 186 |
| Layperson: Trust and Intuition | .659 | < .001 | 186 |
| Layperson: Quantitative reasoning and Intuition | .615 | < .001 | 186 |
Table 11 as a figure
Each square is the correlation between two ratings for one recommender. Darker blue means the two move together more strongly.
Mixed models across all eleven recommenders
Each person rated eleven recommenders, so their ratings are not independent. These models give every person their own baseline level of trust. The AI recommenders are the four AI tools plus a doctor using AI, as in the notebook.
What to look for. Marginal R² jumps from .05 to .52 once the two ratings enter. A lower AIC means a better balance of fit and simplicity.
| Model | Marginal R² | Conditional R² | Person share | AIC |
|---|---|---|---|---|
| 1. AI versus human only | .05 | .44 | .41 | 9691 |
| 2. Quantitative reasoning and intuition | .52 | .64 | .25 | 8640 |
| 3. Add AI versus human | .51 | .64 | .26 | 8610 |
| 4. Let the slopes differ for AI | .51 | .64 | .26 | 8594 |
| 5. Between and within people | .52 | .64 | .25 | 8642 |
| 6. Between and within people, plus AI | .53 | .65 | .26 | 8609 |
| 7. AI recommenders only | .43 | .68 | .44 | 3855 |
| 8. Human recommenders only | .51 | .64 | .25 | 4708 |
Random intercept for each person, restricted maximum likelihood, 188 people with 11 ratings each. Tests are Wald z tests. Standardized betas are b times the SD of the predictor over the SD of trust. Model comparisons refit by maximum likelihood. The notebook's last model (3D) uses grand-mean variables the notebook never creates, so it is not reproduced here. A doctor using AI counts as AI, as in the notebook.
Table 12 as a figure. Each bar is the share of the variation in trust a model explains: first by its predictors alone, then with each person's own baseline added. Longer is a better fit.
Coefficients for each model
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 6.02 | 0.16 | 5.71 to 6.33 | 38.20 | < .001 | |
| AI recommender (vs human) | −1.37 | 0.10 | −1.58 to -1.17 | −13.31 | < .001 | −.22 |
2,051 ratings from 188 people. Person share .41.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 1.03 | 0.14 | 0.76 to 1.31 | 7.48 | < .001 | |
| Quantitative reasoning | 0.37 | 0.02 | 0.34 to 0.41 | 19.63 | < .001 | .37 |
| Intuition | 0.44 | 0.02 | 0.40 to 0.47 | 23.92 | < .001 | .44 |
2,051 ratings from 188 people. Person share .25.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 1.42 | 0.15 | 1.12 to 1.72 | 9.23 | < .001 | |
| Quantitative reasoning | 0.37 | 0.02 | 0.34 to 0.41 | 19.79 | < .001 | .37 |
| Intuition | 0.41 | 0.02 | 0.37 to 0.44 | 21.48 | < .001 | .41 |
| AI recommender (vs human) | −0.49 | 0.09 | −0.66 to -0.32 | −5.71 | < .001 | −.08 |
2,051 ratings from 188 people. Person share .26.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 1.45 | 0.17 | 1.11 to 1.79 | 8.38 | < .001 | |
| Quantitative reasoning | 0.44 | 0.03 | 0.39 to 0.49 | 17.70 | < .001 | .43 |
| AI recommender (vs human) | −0.43 | 0.19 | −0.79 to -0.06 | −2.26 | .024 | −.07 |
| Quantitative reasoning × AI | −0.14 | 0.04 | −0.21 to -0.07 | −4.10 | < .001 | |
| Intuition | 0.33 | 0.03 | 0.28 to 0.39 | 13.00 | < .001 | .34 |
| Intuition × AI | 0.14 | 0.04 | 0.07 to 0.20 | 3.89 | < .001 |
2,051 ratings from 188 people. Person share .26.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 0.92 | 0.26 | 0.41 to 1.42 | 3.56 | < .001 | |
| Quantitative reasoning, person average | 0.31 | 0.07 | 0.18 to 0.43 | 4.66 | < .001 | .21 |
| Quantitative reasoning, vs own average | 0.38 | 0.02 | 0.34 to 0.41 | 18.63 | < .001 | .27 |
| Intuition, person average | 0.53 | 0.07 | 0.40 to 0.66 | 8.06 | < .001 | .36 |
| Intuition, vs own average | 0.43 | 0.02 | 0.39 to 0.47 | 22.29 | < .001 | .32 |
2,051 ratings from 188 people. Person share .25.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 1.15 | 0.26 | 0.64 to 1.66 | 4.40 | < .001 | |
| Quantitative reasoning, person average | 0.31 | 0.07 | 0.18 to 0.43 | 4.65 | < .001 | .21 |
| Quantitative reasoning, vs own average | 0.37 | 0.02 | 0.33 to 0.41 | 18.67 | < .001 | .26 |
| Intuition, person average | 0.53 | 0.07 | 0.40 to 0.66 | 8.02 | < .001 | .36 |
| Intuition, vs own average | 0.39 | 0.02 | 0.35 to 0.43 | 19.59 | < .001 | .29 |
| AI recommender (vs human) | −0.51 | 0.09 | −0.68 to -0.34 | −5.90 | < .001 | −.08 |
2,051 ratings from 188 people. Person share .26.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 1.36 | 0.20 | 0.97 to 1.75 | 6.84 | < .001 | |
| Quantitative reasoning | 0.27 | 0.03 | 0.21 to 0.33 | 9.25 | < .001 | .27 |
| Intuition | 0.43 | 0.03 | 0.37 to 0.49 | 14.98 | < .001 | .43 |
929 ratings from 187 people. Person share .44.
| Predictor | b | SE | 95% CI | z | p | β |
|---|---|---|---|---|---|---|
| Intercept | 1.31 | 0.18 | 0.96 to 1.66 | 7.39 | < .001 | |
| Quantitative reasoning | 0.45 | 0.03 | 0.40 to 0.49 | 17.40 | < .001 | .45 |
| Intuition | 0.35 | 0.03 | 0.30 to 0.41 | 13.19 | < .001 | .35 |
1,122 ratings from 188 people. Person share .25.
What to look for. Every p is below .001, so each added term improved the model.
| Comparison | χ² | df | p |
|---|---|---|---|
| Model 2 to Model 3: add AI versus human | 32.32 | 1 | < .001 |
| Model 3 to Model 4: let the slopes differ for AI | 20.21 | 2 | < .001 |
| Model 5 to Model 6: add AI versus human | 34.55 | 1 | < .001 |
Likelihood ratio tests, refit by maximum likelihood.
Table 13 as a figure. Each bar is how much a model improves when a term is added (χ²). Longer means a bigger improvement; with one degree of freedom, anything over 3.8 is past the usual cutoff.
Figure 8. What lifts trust differs for AI and for humans
For human recommenders, perceived quantitative reasoning is the stronger partner of trust (b = 0.45 against 0.35 for intuition). For AI, it is intuition (b = 0.43 against 0.27). The interaction model agrees: the two slopes differ by recommender type (Model 4: both interaction terms, p < .001).
- Quantitative reasoning
- Intuition
- The other rating is held at its average. Bands are 95% confidence intervals.
Predicted trust from the AI-only and human-only models, with the other rating held at its average. Bands are 95% confidence intervals for the fixed effects.
What to look for. Compare the two columns. Between-people values reflect who the person is; within-people values reflect which recommender is being rated.
| Recommenders and pair | Between people, r (p) | Within people, r (p) |
|---|---|---|
| All recommenders: Trust and Intuition | .776 (< .001) | .562 (< .001) |
| All recommenders: Trust and Quantitative reasoning | .726 (< .001) | .519 (< .001) |
| All recommenders: Intuition and Quantitative reasoning | .769 (< .001) | .384 (< .001) |
| AI recommenders: Trust and Intuition | .781 (< .001) | .391 (< .001) |
| AI recommenders: Trust and Quantitative reasoning | .663 (< .001) | .283 (< .001) |
| AI recommenders: Intuition and Quantitative reasoning | .614 (< .001) | .224 (< .001) |
| Human recommenders: Trust and Intuition | .711 (< .001) | .526 (< .001) |
| Human recommenders: Trust and Quantitative reasoning | .725 (< .001) | .592 (< .001) |
| Human recommenders: Intuition and Quantitative reasoning | .744 (< .001) | .451 (< .001) |
Between-person correlations use each person's average over the recommenders. Within-person correlations use each rating's distance from that person's own average, pooled across people.
Table 14 as a figure, for all recommenders together. The left grid is between people: do people who rate recommenders high on one thing rate them high on the other? The right grid is within people: for one person, do recommenders rated higher on one thing also get rated higher on the other?