Study 2

The gap is robust. People trusted a doctor more than an AI on every one of the twelve decisions, in every kind of health problem and at every stage of care, and the result survives correction for twelve tests. It is large (d from 0.88 to 1.30), and smallest for the preventative decisions, such as recommending stretching or vitamins. Together with Study 1, this is a double replication, which is why the later studies could take the gap as given and try to change it. It does not mean people give an AI no trust: the average for the AI was 4.1 on the 0 to 10 scale, not zero. They give it a more skeptical reading than they give a doctor.

What a decision needs does not explain the gap. The gap varied from decision to decision, but not in step with how much quantitative reasoning people said the decision needed. So whatever makes some decisions feel better suited to an AI, it is something other than the quantitative-versus-intuition trade-off I measured. That measure is a single slider that treats the two as opposites, and a future version should rate them separately.

How people see the recommender explains part of it. Doctors were rated higher than the AI on quantitative reasoning, intuition, warmth, competence and every mind-perception item. Quantitative reasoning and intuition each carried a significant share of the gap, even together, whereas warmth did not once competence was in the model. Neither explains the whole gap: about half of it remained after the mediators were included (2.08 of 3.90 points). Still, it gives a framework for how people judge a recommender, and so for predicting and perhaps shifting their trust. If an AI is to earn more trust, it will do so by being seen as capable of both kinds of reasoning, not by changing what it recommends.

Cautions. The qualities people rated are strongly correlated, so some of what looks like separate mechanisms may be a general positive impression of the recommender; each still carries a unique share in the model, which argues against that being the whole story. The decisions were abstract, so people supplied their own picture of the situation. The sample is 106 online-panel adults with an older average age (63 in the dissertation), and the results are about stated trust, not behavior. Chapters 3 and 4 take up the first and last of these.

• study2-ratings.csv, the de-identified ratings
• study2-codebook.csv, what every column means
• study2-analysis.R, the main tests in R, from the public file
• study2-original-analysis.Rmd, my original notebook, run on the raw export
• Preregistration (PDF), written before data collection
• study2-full-results.py, the code behind every table and figure on this page
• study2-deidentify.py, how the raw export became the public file

Everything above was computed in Python from the released file. Where the notebook used an R package or Mplus I used the closest equivalent: descriptives and reliability follow psych, the mixed models follow lme4 with a random intercept and a random slope for the decision's need for quantitative reasoning, and the mediation models are within-person regressions with a bootstrap over people. The main results match the dissertation: for example the doctor effect of 1.68 and the weights on intuition (.14), quantitative reasoning (.27), warmth (.14) and competence (.58).

Four differences are worth knowing. People's ratings here are on the scales they saw, whereas the notebook kept some of the survey's stored codes, so means and intercepts differ while differences, slopes and correlations match. Composites are missing when any item is missing. Age is in bands. And the mediation tests are close to, not identical to, the Mplus output. I have not run the R code on the released file to compare line by line.

The data, codebook and preregistration are shared under CC BY 4.0; the code under the MIT license. Please do not use the file to try to identify anyone. Please cite the dissertation: Leitao, M. R. (2024). Artificial Intelligence, Humanization, and Health Beliefs. Georgetown University.