← Can Konuk Dissertation Précis
PhD Dissertation · Précis

Causal Explanations and Continuous Computation

A précis of my doctoral dissertation, completed at Institut Jean Nicod (École Normale Supérieure) under the supervision of Salvador Mascarenhas.

Author
Can Konuk · Stanford University
Defended
December 2025, Université Paris Cité
Full text
Dissertation PDF

Introduction

This dissertation investigates human causal understanding, that is, people's ability to represent and reason over the causal relations inherent to a system. Specifically, it examines the relation between our category of cause, on which rests our knowledge of what events count as causes for what outcomes, and the graded notions of causal strength and responsibility that underlie our intuition that the contribution made by a certain event is large or small. This requires a theory of the representations we use to encode causal systems, and a theory of how these representations are computed over.

Causal explanations provide the empirical counterpart to such a theory. When we explain why something happened, we externalize aspects of our causal representations, revealing not just what we know about a causal system but how that knowledge is mentally organized. Consider a forest fire sparked by lightning. The fire required both lightning and oxygen; without either, no combustion could occur. Yet we unhesitatingly cite the lightning as the cause, never the oxygen. This asymmetry is what philosophers and cognitive scientists call causal selection. Importantly, the causal facts do not force this choice upon us, as the fire depended on both factors equally; rather, it reflects how our cognitive machinery represents and weighs causes. This makes causal selection judgments a window into the representational and computational mechanisms by which we mentally apprehend causal systems.

In particular, they highlight two dimensions of our causal understanding. On the one hand, our representations encode structure: discrete commitments about what causes what. Causation runs from lightning and oxygen to fire, not the reverse. On the other hand, our representations support graded distinctions: lightning matters more than oxygen. This duality of discrete structure coexisting with continuous gradation is a special case of what Smolensky (1986) called the "Structure/Statistics Dilemma." Cognition appears both "hard" (rule-governed, honoring discrete logical constraints) and "soft" (context-sensitive, encoding information in continuous quantities that admit degrees). Traditional cognitive architectures tend to privilege one aspect at the expense of the other.

The dominant framework for modeling causal relations, Structural Causal Models (Pearl, 2009), addresses the "hard" side decisively. SCMs encode causal systems as directed acyclic graphs paired with structural equations, grounding the distinction between seeing and doing. Structural equations are non-parametric: they implement discrete, rule-like relations. Whatever gradedness there is enters through exogenous variables — distributions encoding how common or rare each factor is. SCMs provide the formal machinery for counterfactual simulation models of causal selection (Icard et al., 2017; Quillien & Lucas, 2023; Gerstenberg et al., 2021), in which the responsibility of a cause \(C\) for an effect \(E\) is a function of how well \(C\) covaries with \(E\) across counterfactual worlds imagined by a subject.

My dissertation highlights the advantages of such a counterfactual simulation view, whose principle I endorse. But I also argue that the choice of SCMs as the machinery underlying counterfactual simulations is too limiting. Specifically, I argue that many of the patterns of responsibility judgments currently explained via structural models would be better understood by assuming different, neural representations of the same causal relations, representations that:

  1. possess more structural details than SCMs,
  2. break down that structure in terms of parametric functions with continuous-valued weights at the local level of pairwise connections between nodes.

This raises a classic worry: connectionist models threaten underdetermination (Massaro, 1988; Fodor & Pylyshyn, 1988): infinitely many architectures can compute any given input-output function. My dissertation proposes to circumvent this problem via an intermediate level of idealization — theories from formal semantics (Groenendijk, 2008; Link, 1983), mental model theory (Johnson-Laird, 1983; Khemlani & Johnson-Laird, 2013; Byrne, 2005), and logic programming (Robinson, 1965; Lloyd, 1984) that go beyond classical representations while remaining symbolically constrained.

Part I presents the first systematic experiments on plural causal judgments — "E happened because of A and B." The findings reveal that causal intuitions are sensitive to the structural details in our representation of causal rules that SCMs do not capture. I show how these patterns can be accounted for in terms of operations over neural representations of the same causal systems.

Part II turns to how causal selection explanations guide the acquisition of causal knowledge. I present a new rule-induction paradigm to test inference from explanations, and propose to understand the role of explanations as attentional instructions: "E happened because of A" tells a learner to pay more attention to "A" as they are trying to learn from examples. I model this using attention masks that bias gradient-based learning procedures toward rules that assign A greater importance.

Part IFrom representations to explanations

Causal selection and plural causes

Prior work has established that causal selection depends on normality: abnormal causes receive different treatment than routine ones (Hitchcock & Knobe, 2009; Icard et al., 2017). Crucially, the direction of this effect depends on causal structure. Conjunctive structures show abnormal inflation (the rare cause receives credit), while disjunctive structures show abnormal deflation (the common cause receives credit). These patterns can be captured by counterfactual simulation models of causal judgments, which say that causes receive credit to the extent that the outcome covaries with them across alternative scenarios.

Counterfactual simulation theoryStructural causal modelABCDEE := (A ∧ B) ∨ (C ∧ D)Exogenous distributionsUA ∼ Bern(0.7)UB ∼ Bern(0.1)UC ∼ Bern(0.2)UD ∼ Bern(0.9)sampleActual worldOutcomeABCD1W0anchoringSampled worldsABCD1W1ABCD1W2ABCD0W3ABCD0W4A ∧ BCovariationA cause receives credit to the extent that E covaries with it across sampled counterfactuals.ABA∧BEW01111W10101W21001W31000W41000Deflationary compute match rates for A and B separately, then aggregate (e.g., average or sum).Holistic compute the match rate for A∧B directly — the conjunction is treated as a single unit whose covariation with E determines causal strength.
Counterfactual simulation theoryStructural causal modelABCDEE := (A ∧ B) ∨ (C ∧ D)ExogenousdistributionsUA ∼ Bern(0.7)UB ∼ Bern(0.1)UC ∼ Bern(0.2)UD ∼ Bern(0.9)sampleActual worldOutcomeABCD1W0anchoringSampled worldsABCD1W1ABCD1W2ABCD0W3ABCD0W4A ∧ BCovariationA cause receives credit to the extent that E covaries with it across sampled counterfactuals.ABA∧BEW01111W10101W21001W31000W41000Deflationary compute match rates for A and B separately, then aggregate (e.g., average or sum).Holistic compute the match rate for A∧B directly — the conjunction is treated as a single unit whose covariation with E determines causal strength.
Fig. 1Counterfactual simulation models of causal judgment (Icard et al., 2017; Quillien & Lucas, 2023). Causal credit is computed in two stages. First, sampling alternatives: the reasoner imagines alternative worlds that might have been. The sampling is biased towards worlds that are similar to the actual world and events that have a higher prior probability of occurrence. Formally, each variable’s sampling probability is a mixture \(P_{\text{sample}}(X{=}1) = s \cdot x_{\text{actual}} + (1-s) \cdot P(X)\), where the free parameter \(s\) controls the balance between the actual world and the prior. Second, covariation: causes receive credit to the extent that the outcome covaries with them across alternative scenarios imagined by the subject.

These effects have been successfully modeled within the framework of SCMs. But the central claim of this dissertation is that SCMs lack the structural resources to capture all patterns in causal selection. My dissertation makes this shortcoming evident through the first systematic experiments on plural causal judgments — statements like "\(E\) happened because of \(A\) and \(B\)" (Konuk et al., under review).

Such judgments have received little systematic study, perhaps because of a tempting deflationary hypothesis: that the strength of a plural simply aggregates the independent singular judgments about each variable. Under this view, plural selection reduces to singular selection, requiring no new machinery.

Habituation round: three urns A, B and C with Draw buttons; the player's score and the rule, 2 points required to win, 1 colored ball = 1 point
(a) Habituation
Round to be judged: John drew colored balls from urns A, B and C, scored 3 points and won
(b) Round to be judged
Querying judgments: urns A and C circled, with a 1 to 9 agreement scale for the statement “John won because he drew colored balls from boxes A and C”
(c) Querying judgments
Fig. 2Experiment 1 design. Our first experiment was designed to rule out the deflationary account and establish that plural causes are a genuine phenomenon. (a) Habituation rounds introduce the urn game. (b) Participants then see the round to be judged. (c) They rate a singular or plural causal statement, such as “John won because he drew colored balls from boxes A and C.”

Our first experiment (presented at CogSci 2023; Konuk et al., 2023) was meant to rule out this deflationary account and establish that plural causes are a genuine phenomenon. It provided evidence that people evaluate plural causes as bona fide candidates for causal selection, entities whose counterfactual profile is apprehended directly rather than recomposed from individual constituents.

With plural causes established as a genuine phenomenon, a second experiment tested whether counterfactual theories could account for them within the framework of structural models. I designed a game with an explicitly disjunctive rule: winning required either drawing from urns \(A\) and \(B\) together, or from urns \(C\) and \(D\) together — formally:

$$ \text{WIN} := (A \land B) \lor (C \land D) $$
John drew a colored ball from all four urns, two yellow and two purple, and won
(a) Overdetermined positive
John drew a white ball from one urn and colored balls from the other three, and won
(b) Triple positive
John drew a white ball from all four urns and lost
(c) Overdetermined negative
John drew a colored ball from only one urn and white balls from the other three, and lost
(d) Triple negative
Fig. 3Experiment 2 conditions. The experiment included positive-outcome conditions where one or both routes to winning were satisfied, and negative-outcome conditions. To win, a player needed two purple balls (urns \(A\) and \(B\)) or two yellow balls (urns \(C\) and \(D\)). (a) Overdetermined positive: a colored ball from every urn. (b) Triple positive: a white ball from one urn and a colored ball from the other three. (c) Overdetermined negative: a white ball from every urn. (d) Triple negative: a colored ball from only one urn.

A clear pattern emerged: participants strongly preferred "same-side" pairs — plurals whose members sit on the same route (\(A\)&\(B\) or \(C\)&\(D\)) — over "cross-side" pairs that mix variables from different routes (\(A\)&\(C\) or \(B\)&\(D\)). Same-side ratings substantially exceeded counterfactual model predictions. This provides evidence that people group causes in ways that track disjunctive structure, beyond what counterfactual dependence profiles alone can explain.

An even more striking pattern emerged for negative outcomes. Classical counterfactual theories systematically predict abnormal deflation in this setup: the urns \(B\) and \(C\) that contain the higher number of white balls should be rated highest. But participants showed the opposite — preference for urns \(A\) and \(D\) instead. Moreover, the same-side versus cross-side distinction that dominated winning rounds disappeared entirely in losing rounds.

Experiment 2 results for wins: mean ratings and model predictions (CESM, NSM) by urn set, with same-side pairs rated highest
Wins
Experiment 2 results for losses: mean ratings and model predictions (CESM, NSM) by urn set, with no same-side preference
Losses
Fig. 4Experiment 2 results. Same-side ratings exceeded counterfactual model predictions for wins. For losses, participants showed abnormal inflation (opposite of prediction) and the same-side structure collapsed.

These results are deeply puzzling as they cannot be reconciled with existing counterfactual theories. I proposed an account that was able to capture these results remarkably well while staying within the framework of counterfactual simulation models. It assumes that as people score various candidate causes relative to how well they correlate with the (losing) outcome across counterfactuals, their target for what counts as a loss is not the classical negation of the winning conditions, but rather a "stronger" negation:

$$ \text{LOSE}_{\text{strong}} := \neg A \land \neg B \land \neg C \land \neg D $$

This hypothesis was inspired by the phenomenon of homogeneity in the natural language semantics of plural terms. Consider the sentence "The boys didn't leave." Classically, negating "The boys left" should yield "At least one boy didn't leave" — but speakers typically interpret it as "None of the boys left." Negation applies uniformly to all members of the plural, yielding a stronger reading than classical logic predicts. A model incorporating this assumption dramatically improves fit.

A new computational model of causal selection

I take the experiments above to illustrate a more general point: causal judgments depend not just on the ground truth rule underlying a causal system, but on how it is mentally represented. A central theoretical commitment of this dissertation is that we can capture this internal structure using neural architectures to model the cognitive machinery that underlies the production of causal judgments.

I say that the simulation processes by which we generate counterfactuals and elaborate causal explanations unfold over neural networks with richer structure than Structural Causal Models typically encode. In these neural models, intermediate layers track which combinations of variables form coherent pathways to an outcome.

Three representations W W W W W W E H1 H2 A θ = 0.8 B θ = −2.2 C θ = −1.4 D θ = 2.2 route 1 route 2 Internal representation neural network with continuous weights and activation functions E A B C D SCM DAG External world E := (A ∧ B) ∨ (C ∧ D) A B C D E ← A, B E ← C, D E logic program observes internally represented as
Three representations E A B C D SCM DAG External world E := (A ∧ B) ∨ (C ∧ D) W W W W W W E H1 H2 A θ = 0.8 B θ = −2.2 C θ = −1.4 D θ = 2.2 route 1 route 2 Internal representation neural network with continuous weights and activation functions A B C D E ← A, B E ← C, D E logic program observes internally represented as
Fig. 5How an observer internally represents a causal system. Top right: the external world with four urns. The SCM DAG shows all causes connecting directly to \(E\) with no internal structure. Left: the observer's neural representation, derived via the CILP algorithm, creates one hidden node per clause in the logic program. Each hidden node (\(H_1\), \(H_2\)) computes the conjunction of its inputs; the output node computes their disjunction.

The two disjuncts in \((A \land B) \lor (C \land D)\) translate into two distinct hidden nodes (\(H_1\) and \(H_2\)), each tracking whether its corresponding "route" to the outcome is satisfied. The routes that subjects track in their causal judgments thus correspond to separable subnetworks in the model.

One might worry that this proposal faces a classic problem for connectionist approaches. For any rule like the one above, infinitely many networks could compute the same truth function, including architectures quite different from the one I propose. What principled reason do we have to prefer one over another?

I argue that this underdeterminacy problem can be circumvented by appealing to an intermediate level of representation: that of symbolic theories of computation. I turn to the framework of logic programming. The central idea is that representing a logical relation requires executing a procedure that can derive \(E\) from \(A, B, C, D\). The class of general logic programs represents the simplest class of programs that can represent such a rule. It consists of sets of Horn clauses, each specifying how to derive a conclusion from premises:

$$\begin{aligned} E &\leftarrow A, B \\ E &\leftarrow C, D \end{aligned}$$

Each clause represents one "route" to establishing \(E\). This modular structure is independently supported by theories of mental representation. Both mental model theory and inquisitive semantics propose that people represent disjunctions as collections of distinct alternatives rather than flat Boolean formulas.

Logic programs also illuminate the asymmetry between positive and negative outcomes. Proving \(E\) is existential: find one clause whose body is satisfied. Proving \(\sim E\) (negation-as-failure) is universal: verify that every route to \(E\) fails. When cashed out in the machinery of neural models, this asymmetry gives us a way to explain both the collapse of same-side structure and the reversal from abnormal deflation to abnormal inflation for losses.

To cash out these symbolic constraints in neural terms, I appeal to the CILP neuro-symbolic translation algorithm (Garcez et al., 2002), which systematically compiles logic programs into three-layer neural networks with one hidden node per clause. The architecture is no longer arbitrary: it is tethered to the clause structure of the source program.

Counterfactual sampling and weight updates

But specifying the architecture is only half the story. My dissertation also proposes a theory of the processes by which these representations are computed over. The core idea is that people stress-test their internal models: they simulate counterfactual scenarios, and these simulations incrementally adjust connection weights. This weight adjustment models the cognitive work of pondering each cause's contribution; the accumulated weights then ground the final importance judgment.

This troubleshooting process unfolds in three stages. First, starting from the observed actual world, subjects explore neighboring counterfactual states via an MCMC process following the mutation sampler model of Davis & Rehder (2020). At each step, they consider flipping one input variable's value. Transition probabilities depend on two factors: the normality of the event and whether the proposed change would alter hidden node activations from the previous steps.

MCMC counterfactual sampling processOnOffCarries signalthe walkS0actual worldS1S2 = S1S3EH1H2ABCDEH1H2ABCDEH1H2ABCDEH1H2ABCDflip Aacceptflip Drejectflip Cacceptno changeE changesthe first step, magnifiedcurrent stateproposed stateEH1H2ABCDEH1H2ABCDproposeflip AuniformA′ ≠ H1misalignedacceptance probabilityα = min(1, eΔ),Δ = λprior(A)+λcoh(A, H1)normalityalignmentmisalignment with H1 adds cost, so lowers αu < α?acceptmove to proposedrejectstay in current
MCMC counterfactual sampling processOnOffCarries signalthe walkS0actualS1EH1H2ABCDEH1H2ABCDflip Aacceptflip DrejectS2 = S1S3EH1H2ABCDEH1H2ABCDflip Cacceptno changeE changesone step, magnified: S0 → S1current stateproposed stateEH1H2ABCDEH1H2ABCDproposeflip AuniformA′ ≠ H1: misalignedacceptance probabilityα = min(1, eΔ)Δ = λprior(A)+λcoh(A, H1)normalityalignmentmisalignment with H1 adds cost, so lowers αu < α?acceptmove to proposedrejectstay in current
Fig. 6Counterfactual sampling via MCMC. Starting from the observed actual world, subjects explore neighboring counterfactual states. Transition probabilities depend on the normality of the event and whether the proposed change would alter hidden node activations.

In each sampled state, subjects update their internal models' weights using Layer-wise Feedback Propagation (LFP; Weber et al., 2025). The key insight is that connections strengthen when pre- and postsynaptic signals align, and weaken when they oppose. This produces abnormal deflation at the hidden-to-output layer (normal routes gain credit) and abnormal inflation at the input-to-hidden layer (the absent input that deactivated a hidden node gains credit).

Weight update mechanismOn (+1)Off (−1)Carries signalStrengthenedWeakenedsample 1 (from fig. 6)(a) hidden → outputEH1H2↓ penalized↑ rewardedaH1 = −1aH2 = +1rE = 1, zE > 0abnormal deflationSince zE > 0, alignment requirespositive activation.H2 (+1) aligns ⇒ wH2→E strengthenedH1 (−1) misaligns ⇒ wH1→E weakenedThe inactive hidden node loses credit.(b) input → hiddenH1AB↑ rewarded↓ penalizedaA = −1aB = +1rH1, zH1 < 0abnormal inflationSince zH1 < 0, alignment now requiresnegative activation.A (−1) aligns ⇒ wA→H1 strengthenedB (+1) misaligns ⇒ wB→H1 weakenedThe absent input gainsdisproportionate credit.rH1 propagateslfp update ruleδwij = η · |wij| · aizj · rjaipresynaptic activationzjpostsynaptic pre-activationrjreward at node jkeythe sign of δwij depends onwhether sign(ai) = sign(zj):aligned ⇒ strengthened,misaligned ⇒ weakened(a)(b)EH1H2ABCD−1+1+1+1rE = 1
Weight update mechanismOn (+1)Off (−1)Carries signalStrengthenedWeakenedsample 1 (from fig. 6)(a) hidden → outputabnormal deflationSince zE > 0, alignment requirespositive activation.H2 (+1) aligns ⇒ wH2→E strengthenedH1 (−1) misaligns ⇒ wH1→E weakenedThe inactive hidden node loses credit.EH1H2↓ penalized↑ rewardedaH1 = −1aH2 = +1rE = 1, zE > 0(b) input → hiddenH1AB↑ rewarded↓ penalizedaA = −1aB = +1rH1, zH1 < 0abnormal inflationSince zH1 < 0, alignment now requiresnegative activation.A (−1) aligns ⇒ wA→H1 strengthenedB (+1) misaligns ⇒ wB→H1 weakenedThe absent input gainsdisproportionate credit.rH1 propagateslfp update ruleδwij = η · |wij| · aizj · rjaipresynaptic activationzjpostsynaptic pre-activationrjreward at node jkeythe sign of δwij depends onwhether sign(ai) = sign(zj):aligned ⇒ strengthened,misaligned ⇒ weakened(a)(b)EH1H2ABCD−1+1+1+1rE = 1
Fig. 7Weight update mechanism. Connections strengthen when pre- and postsynaptic signals align, and weaken when they oppose. This produces different patterns at different layers of the network.

This mechanism captures the asymmetry between positive and negative outcomes observed in Experiment 2: for wins, sampling concentrates on the satisfied route, so hidden-layer deflation dominates and same-side structure emerges; for losses, all routes must be checked, input-layer inflation takes over, and surprising failures gain credit.

After sampling, causal importance is computed via Layer-wise Relevance Propagation (LRP; Bach et al., 2015), which distributes credit backward through the updated weights. The final importance score divides summed relevance by path complexity:

$$ \kappa(C, O) = \frac{\sum_{c \in C} R_c}{\mathcal{C}(C, O)} $$

where \(\mathcal{C}(C, O)\) counts edge-disjoint active routes. This parsimony term explains the same-side preference: causes operating through a single route score higher than those spreading across multiple routes.

Model fit: computational model against mean causal ratings in four conditions Computational model Mean responses model parameters η = 1.15, γ = 0.58 (cumulative update timing) overdetermined negative 1 3 5 7 9 A B C D A B A C A D B C B D C D A C D B C D causation causes overdetermined positive 1 3 5 7 9 A B C D A B A C B C C D A B C A B D triple 0 1 3 5 7 9 A B D A B A D B D A B D causation causes triple 1 1 3 5 7 9 A B D A B A D B D A B D
Model fit: computational model against mean causal ratings in four conditions Computational model Mean responses model parameters η = 1.15, γ = 0.58 (cumulative update timing) overdetermined negative 1 3 5 7 9 A B C D A B A C A D B C B D C D A C D B C D causation causes overdetermined positive 1 3 5 7 9 A B C D A B A C B C C D A B C A B D causation causes triple 0 1 3 5 7 9 A B D A B A D B D A B D causation causes triple 1 1 3 5 7 9 A B D A B A D B D A B D causation causes
Fig. 8LRP and model fit. LRP distributes credit backward through the network. The model provides excellent fit to the experimental data.

The coherence of these stages is crucial: hidden nodes influence which counterfactuals are sampled, weight updates accumulate evidence about causal pathways, and relevance propagation reads off the result.

Part IIFrom explanations to learning

The inference-from-explanation puzzle

Part II turns to the converse question: how do causal explanations guide the acquisition of causal knowledge? An explanation like "E happened because of C" deliberately omits most of the causal picture, singling out one variable from among many that might matter. Yet such sparse signals seem remarkably effective: everyday explanations help us learn the structure of unfamiliar causal systems. How can mentioning a single cause guide inference about a rule that may involve several variables in complex combinations?

Experimental evidence

I co-developed a new paradigm to study this phenomenon (presented at CogSci 2024; Navarre et al., 2024), using the same urn-sampling setup as Part I.

Four successive draws, each with an explanation such as “Here you won because you drew a colored ball from urn C”, in the any-cause and the causal-selection conditions, beside the full history of ten observations
(i) Draws with explanations, and the observation history
A column of new urn combinations whose outcome, win or loss, the participant predicts
(iii) Outcomes to predict
The four urns A to D and the Draw sample button that produces each observation
(ii) Urns and sampling
Table of the four kinds of observation with their any-cause and causal-selection explanations
(iv) All observations and explanations
Fig. 9Experimental paradigm for studying inference from explanations. Participants observe outcomes and (in some conditions) receive causal selection explanations, then infer the underlying rule. (i) A sequence of draws with their explanations, as given in the any-cause and the causal-selection conditions, next to the full history of observations once all ten samples are drawn. (ii) The urns participants sample from, and the sampling process. (iii) The table of draws whose outcomes participants are asked to predict (not all 16 shown). (iv) The four distinct observations and the explanations given for each.

The experiment yielded three key findings: (i) participants who received causal selection explanations (CS) inferred the correct rule significantly more often than those who saw observations alone (OBS); (ii) participants who received explanations citing any causally relevant variable (AC) performed worse than OBS, despite receiving additional information; and (iii) participants in the CS and OBS conditions showed a striking preference for the simple rule W := C, even though one CS explanation ("You lost because of C and D") should have ruled it out.

Number of participants whose predictions matched each rule. two of A, C, D: OBS 1, CS 0, AC 4; A and C: OBS 11, CS 2, AC 23; always win: OBS 12, CS 4, AC 5; C: OBS 35, CS 43, AC 8; C or D: OBS 4, CS 17, AC 0; other rules: OBS 40, CS 31, AC 58. 0 20 40 60 participants Observations only (OBS) Causal selection (CS) Any cause (AC) two of A, C, D 1 0 4 A ∧ C 11 2 23 always win 12 4 5 C 35 43 8 C ∨ D 4 17 0 other 40 31 58
Number of participants whose predictions matched each rule. two of A, C, D: OBS 1, CS 0, AC 4; A and C: OBS 11, CS 2, AC 23; always win: OBS 12, CS 4, AC 5; C: OBS 35, CS 43, AC 8; C or D: OBS 4, CS 17, AC 0; other rules: OBS 40, CS 31, AC 58. 0 20 40 60 participants Observations only (OBS) Causal selection (CS) Any cause (AC) two of A, C, D 1 0 4 A ∧ C 11 2 23 always win 12 4 5 C 35 43 8 C ∨ D 4 17 0 other 40 31 58
Fig. 10Rules inferred by participants. Number of participants in each condition whose predictions for the 16 possible draws matched each rule (103 in OBS, 97 in CS, 98 in AC). The simple rule W := C was very popular in both the OBS and CS conditions, although not in the AC condition.

Two accounts offer competing explanations for these patterns. The first is reverse-engineering: the learner treats explanations as Gricean signals, reasoning "If the explainer chose to say 'because of C,' they must believe C is the decisive factor — so I should favor hypotheses where C plays the key role." Formally, the learner infers the explainer's beliefs by asking which hypotheses would lead a rational speaker to produce that explanation.

Reverse-engineering account observation ω A B C D W present absent won learner’s hypothesis testing hypothesis would say W := C “C” W := A ∧ C “A & C” W := ¬B “¬B” W := (A ∧ D) ∨ C “C” “You won because of C” explanation ξ Reverse-engineering account “You won because of C” explanation ξ observation ω A B C D W present absent learner’s hypothesis testing hypothesis would say W := C “C” W := A ∧ C “A & C” W := ¬B “¬B” W := (A ∧ D) ∨ C “C”

Algorithm (reverse-engineering)

For each hypothesis \(h_i \in \mathcal{H}\):

1.Assume \(h_i\) is ground truth

2.Simulate explanation \(\xi_i\) you would give for \(\omega\)

3.Check if \(\xi_i\) matches observed \(\xi\)

Return hypotheses where \(\xi_i = \xi\)

Bayesian variant

Compute likelihood from causal scores:

$$P(\xi \mid h_i, \omega) = \frac{\kappa(\xi, \omega, h_i)^2}{\sum_{e \in \mathbf{E}} \kappa(e, \omega, h_i)^2}$$

where \(\kappa\) is causal responsibility. Update posterior:

$$P(h_i \mid \xi, \omega) \propto P(\xi \mid h_i, \omega) \cdot P(h_i)$$
Fig. 11The reverse-engineering account. Learners treat explanations as Gricean signals and infer which hypotheses would lead a rational speaker to produce that explanation. Left: the learner observes an outcome (a win, with balls A = on, B = off, C = on, D = off) and receives an explanation (“You won because of C”). Right, inside the learner’s head: the learner iterates through candidate hypotheses. For each hypothesis, they run their own counterfactual simulation process (as in Part I) to generate the explanation they would produce if they believed that hypothesis to be the true rule. If the learner assumed W := C were true, counterfactual simulation would yield “Because of C” (matching \(\xi\), marked ✓); if they assumed W := A ∧ C, it would yield “Because of A & C” (not matching, marked ×). Bottom left: the discrete algorithm retains only hypotheses that generate matching explanations. Bottom right: the Bayesian variant computes a likelihood term based on causal responsibility scores \(\kappa\), allowing graded belief updates.

I propose an alternative, attention-based account. Rather than inferring what the explainer believes, the learner treats explanations as instructions to attend to certain variables during learning. Mentioned variables are amplified; others are attenuated. More gradient signal then flows through attended inputs, biasing the learner toward rules that assign those variables greater importance.

Attention-based learning: the explanation acts as an attention mask that amplifies the mentioned input C; gradient updates flow more strongly through the amplified pathway observation ω A B C D W present absent won learner’s attention-based learning ∇ strong ∇ weak ∇ W H1 H2 A ×e−α B ×e−α D ×e−α C ×e+α amplified “You won because of C” explanation ξ Attention-based learning: the explanation acts as an attention mask that amplifies the mentioned input C; gradient updates flow more strongly through the amplified pathway “You won because of C” explanation ξ observation ω A B C D W present absent learner’s attention-based learning ∇ strong ∇ weak ∇ W H1 H2 A ×e−α B ×e−α D ×e−α C ×e+α amplified

Algorithm (attention-based learning)

Given observation \(\omega\) and explanation \(\xi\):

1.Parse \(\xi\) to extract highlighted variables

2.Construct attention mask \(\alpha(\xi)\)

3.Amplify inputs in \(\xi\) by \(e^{+\alpha}\), attenuate others by \(e^{-\alpha}\)

4.Run standard gradient descent on masked inputs

Result: weights on C-pathways receive stronger updates

Attention mask formulation

For input \(x_i\) and explanation highlighting variable \(c\):

$$\tilde{x}_i = x_i \cdot \begin{cases} e^{+\alpha} & \text{if } i = c \\ e^{-\alpha} & \text{otherwise} \end{cases}$$

Gradient update: \(\Delta w \propto \tilde{x} \cdot \nabla_{\text{loss}}\)

Key insight: no hypothesis enumeration required

Fig. 12The attention-based model of learning from explanations. Left: the learner observes the same outcome and explanation as in the previous figure. Right, inside the learner’s head: rather than enumerating hypotheses, the learner applies an attention mask \(\alpha(\xi)\) that amplifies the input corresponding to the mentioned variable (C, shown larger with a vermilion border) and attenuates others (shown smaller). During backpropagation, gradient updates (red dashed arrows) flow more strongly through the amplified pathway, causing weights on those connections to update more. Bottom left: the algorithm processes explanations by modulating input activations before learning. Bottom right: the mathematical formulation shows how attention scaling biases gradient updates. This mechanism organically inflects the gradient-based updates that learning would produce naturally, pushing convergence toward rules that assign the mentioned variable greater importance.

Three considerations favor the attention-based account.

First, reverse-engineering cannot explain the W := C preference: if a learner hears "You lost because of C and D," they should infer at a minimum that the explainer believes both C and D matter, ruling out W := C entirely. Yet that rule remained participants' most popular response. Attention explains this: about half the CS explanations mentioned C alone, so attention to C accumulated across trials, enabling incremental convergence despite one contradicting explanation.

Second, reverse-engineering struggles with the AC decrement: any parameter settings that make W := C the model's top prediction also predict that AC should outperform OBS, the opposite of what we observed. Attention can accommodate the decrement: attending to less diagnostic variables may divert gradient signal from critical ones, hindering learning.

Third, attention has a decisive advantage in computational tractability. Reverse-engineering requires enumerating the hypothesis space and running counterfactual simulations for each candidate, scaling poorly as variables increase. Attention sidesteps this by integrating inference from explanations directly into the learning process by which people infer rules from examples.

Model comparison
Fig. 13Model comparison. Left: experimental results across conditions. Right: the attention-based model captures patterns that the reverse-engineering account cannot explain.

Conclusion

This dissertation proposes a new theory of the internal representations that support human causal understanding. It bridges the gap between symbolic theories of causal reasoning and subsymbolic neural computation to offer a unified account of how we produce and evaluate explanations, and how we learn from the explanations of others.

Part I presents the first systematic experiments on plural causal judgments, revealing that people track representational structure — routes, groupings, surprising failures — that standard formal models do not encode. I account for these patterns using neural architectures that model the internal machinery of human causal understanding.

Part II proposes that causal explanations guide learning by directing attention during gradient-based learning. A common thread connects both: graded causal intuition emerges from continuous weight parameters in our internal representations.

This common thread weaves together insights from psychology, AI, linguistics, and philosophy into a coherent research program — one that grounds human causal cognition in neural computation and illuminates how explanations transfer causal knowledge.