- Variable present
- Absent
1 The question
Explanations change what people learn, but not always for the better. In an earlier model of inference from explanation (Navarre, Konuk, Bramley and Mascarenhas, 2024), inspired by the Rational Speech Act framework, a listener reverse-engineers the explainer’s causal beliefs from what they chose to say, then updates their own beliefs about the causal system by Bayesian inference. That model predicts that any explanation should help.
This notebook proposes a simpler mechanism. Learners update an internal model of the causal system by error-driven, gradient-based learning, and an explanation only changes where the updates go: the variables it mentions have their input activations amplified by a constant factor, the others attenuated. This is an attention mask. Causal selection explanations cite the specific causes, among several sufficient ones, that brought about an outcome (“the match caused the fire” rather than “the oxygen caused the fire”), and they point attention at the right variables. Other explanations that are true but less selective point it elsewhere.
The experiment (Figure 1) asked participants to learn a rule over four binary variables from ten observations. Three groups saw the same observations: with no explanation (observations only), with causal selection explanations, or with other explanations that cite causes less selectively. All were then tested on all 16 combinations of the variables.
2 The model
The learner is a small feed-forward network: four input units for A to D, four hidden units and one output unit for the outcome E; the hidden and output units use \tanh activations. Inputs are +1 when a variable is present and -1 when it is absent. An explanation \xi marks each variable x as mentioned (\xi(x) = +1) or not (\xi(x) = -1), and scales its input by an attention factor:
\begin{aligned} \operatorname{Att}(x, \xi) &= e^{\alpha\, \xi(x)}, \\[4pt] a_x &\leftarrow a_x \cdot \operatorname{Att}(x, \xi) \end{aligned} \tag{2.1}
where \alpha \ge 0 is the strength of attention. A mentioned variable is amplified by e^{\alpha}, an unmentioned one attenuated by e^{-\alpha}. After each observation \omega the network takes one gradient step on the squared error,
\begin{aligned} L_{\omega} &= \tfrac{1}{2}(y_{\omega} - \hat{y}_{\omega})^2, \\[4pt] w &\leftarrow w - \eta\, \frac{\partial L_{\omega}}{\partial w} \end{aligned} \tag{2.2}
with \eta = 1. Because the gradient of a weight leaving an input is proportional to that input’s activation, the attended variables receive larger updates when the network errs (Figure 2). With \alpha = 0 the model is a plain network learning from observations, which is how the observations-only group is simulated.
- Present(+1)
- Absent(-1)
- Mentioned in the explanation
Each simulated participant learns from the same ten observations in a random order, with one explanation chosen per observation as in the experiment, for a number of passes (epochs) drawn from a Poisson distribution, with at least one pass. Response times on the learning trial motivate letting this number vary between learners (Appendix B).
Note
The dissertation describes the number of epochs with an exponential distribution of rate \lambda, that is, with mean 1/\lambda. Here the number is drawn from a Poisson distribution with the same mean, the natural distribution for a count of passes (a Poisson count is the number of events in a process whose waiting times are exponential), and every learner makes at least one pass. The two formulations are equivalent for the results reported here: at the dissertation’s value, \lambda = 1000, the mean is 0.001 and every simulated learner trains for exactly one epoch under either.
3 What the data show
Figure 3 and Table 1 show how many of the 16 test items participants answered correctly. Causal selection explanations improved learning, and the other explanations made it worse than observing alone.
| Condition | N | Mean correct | SD | 95% CI |
|---|---|---|---|---|
| Observations only | 103 | 11.90 | 2.32 | 11.45–12.36 |
| Other explanations | 98 | 11.01 | 1.88 | 10.63–11.39 |
| Causal selection | 97 | 12.76 | 2.33 | 12.29–13.23 |
4 What the model predicts
4.1Fitted to the data
The model has two free parameters, the attention strength \alpha and the mean number of epochs. Both were fitted by grid search, minimising the squared error of the three differences between conditions (causal selection minus observations only, other explanations minus observations only, causal selection minus other explanations). Differences rather than raw means are fitted because the model learns the rule better than people do overall; what the attention account has to explain is how the conditions differ.
- Participants, mean and 95% interval
- Model
| \alpha | Poisson mean | Observations onlyObs. only | Other explanationsOther expl. | Causal selectionCausal sel. | Squared error |
|---|---|---|---|---|---|
| 1.6 | 0.500 | 13.45 | 12.58 | 13.66 | 0.881 |
| 1.7 | 0.500 | 13.45 | 12.52 | 13.61 | 0.916 |
| 1.8 | 0.001 | 13.42 | 12.47 | 13.56 | 0.957 |
| 1.6 | 0.001 | 13.42 | 12.63 | 13.63 | 1.013 |
| 1.5 | 0.001 | 13.42 | 12.68 | 13.65 | 1.021 |
The five best cells all put \alpha between 1.5 and 1.8, with learners making one pass or little more over the observations, and their squared errors differ by less than 0.2: the data pin attention down to a range rather than a single value.
The fit is uneven across the two kinds of explanation. At the best cell the model reproduces the cost of the other explanations, −0.87 items against observing alone (95% Monte Carlo interval −0.91 to −0.83; participants −0.89), but gives causal selection explanations a much smaller benefit than participants showed: 0.20 items (0.17 to 0.23), against 0.86. The model learns the rule well from observations alone, which leaves little room for an explanation to help.
The dissertation illustrated the model with \alpha = 1 and a single epoch. That setting gives mean scores of 13.16 for other explanations, 13.37 for observations only and 13.60 for causal selection, the same ordering as in the experiment.
4.2Attention strength
Figure 5 isolates the effect of attention, with every simulated learner trained for a single epoch.
- Observations only
- Other explanations
- Causal selection
With causal selection explanations, a little attention already helps: accuracy climbs from 13.4 at \alpha = 0 to about 13.7 by \alpha = 0.1, stays near that level up to \alpha = 1.5, and falls off beyond it, where a learner that attends too narrowly loses information about the variables it was not told about. With other explanations, attention brings at most a slight gain (0.09 items at \alpha = 0.1, at the edge of the simulation’s precision), and from \alpha = 0.6 on those learners do clearly worse than learners who only observed. The model thus reproduces the experiment’s ordering, other explanations below observations only below causal selection, which a learner who treats every explanation as added information would not.
5 What this shows, and its limits
A single change to an ordinary learning rule, scaling the inputs that an explanation mentions, is enough to make explanations useful when they select the right causes and harmful when they do not. The model needs no representation of the explainer’s beliefs and no inference about why they said what they said. Its limits are those of a proof of concept: the network is deliberately minimal, the learning rate is fixed, and it fits the differences between conditions rather than absolute accuracy, which the model overestimates. Because the network learns the rule well from observations alone, it captures the cost of non-selective explanations more closely than the benefit of selective ones.
A Data preparation
One CSV per participant is read from data-files/actual-cause-grid-1/. Test accuracy is the number of the 16 test items answered correctly; conditions are coded 1 (observations only), 2 (other explanations) and 3 (causal selection).
attention_raw(), attention_scores(), attention_rts()
# Experiment data for the attention notebook: one CSV per participant in
# ../data-files/actual-cause-grid-1 (relative to models/).
attention_raw <- function(path = "../data-files/actual-cause-grid-1") {
files <- list.files(path, pattern = "\\.csv$", full.names = TRUE)
if (!length(files)) stop("No CSV files found in: ", path)
dplyr::bind_rows(lapply(files, function(f) read.csv(f, header = TRUE, na.strings = c("", "NA"))))
}
# Test accuracy per participant: number of the 16 test items answered correctly
attention_scores <- function(raw) {
test <- raw[raw$questionId == "trial_test", ]
parse <- function(x) { x2 <- gsub("\\[|\\]", "", x); if (!nchar(x2)) NA else as.numeric(strsplit(x2, ",")[[1]]) }
key <- lapply(test$test_key, parse); resp <- lapply(test$response, parse)
test$correct <- mapply(function(k, r) if (anyNA(k) || anyNA(r) || length(k) != length(r)) NA_real_ else sum(k == r), key, resp)
s <- dplyr::summarise(dplyr::group_by(test, subject_ID, group), score = mean(correct, na.rm = TRUE), .groups = "drop")
s$condition <- factor(c("OBS", "AC", "CS")[s$group], levels = c("OBS", "AC", "CS"))
s
}
# Response times on the learning trial, in seconds; times of 200 s or more are removed (9 participants)
attention_rts <- function(raw) {
rt <- raw$rt[raw$questionId == "trial_observations"]
rt <- rt[!is.na(rt) & rt > 0]
list(rt = rt[rt < 200000] / 1000, removed = sum(rt >= 200000), removed_max = max(rt) / 1000)
}B Response times
The number of training epochs is a property of each learner, so the distribution of time spent on the learning trial is informative about it. Figure 6 and Table 3 compare three distributions fitted to those response times.
- Ex-Gaussian
- Normal
- Exponential
| Distribution | Parameters | AIC | BIC |
|---|---|---|---|
| Ex-Gaussian | 3 | 2712.1 | 2723.1 |
| Normal | 2 | 2805.8 | 2813.1 |
| Exponential | 1 | 2861.7 | 2865.4 |
The response times are best described by an ex-Gaussian distribution, a normal component plus an exponential tail, rather than by either component alone. The long tail is what suggests that learners differ in how many passes they make over the observations. The model does not use these fits directly: the number of passes is a separate parameter, drawn per learner from a Poisson distribution whose mean is fitted with \alpha (Section 4.1). At the dissertation’s \lambda = 1000 every learner makes a single pass, and the best cells of the grid search also have a single pass or close to it.
C Stimuli as data
| ABCD | Outcome | Shown | Other explanations (one per participant) | Causal selection |
|---|---|---|---|---|
| ●○●○ | win | 4 | “A and C” | “C” |
| ●●●● | win | 1 | “A”, “D”, “A and D”, “A and C”, “C and D”, “A and C and D” | “C” |
| ○●○○ | loss | 1 | “A”, “D”, “A and D”, “A and C”, “A and C and D” | “C” |
| ●●○○ | loss | 4 | “C”, “D” | “C and D” |
| Item | ABCD | Correct answer |
|---|---|---|
| 1 | ●●●● | win |
| 2 | ●●●○ | win |
| 3 | ●●○● | win |
| 4 | ●●○○ | loss |
| 5 | ●○●● | win |
| 6 | ●○●○ | win |
| 7 | ●○○● | win |
| 8 | ●○○○ | loss |
| 9 | ○●●● | win |
| 10 | ○●●○ | win |
| 11 | ○●○● | loss |
| 12 | ○●○○ | loss |
| 13 | ○○●● | win |
| 14 | ○○●○ | win |
| 15 | ○○○● | loss |
| 16 | ○○○○ | loss |
_R/attention/stimuli.R
# Stimuli: the four training observations per condition and the 16 test items.
# The rule is E <- C and E <- A, D (E is true when C is, or when A and D both are).
# 'CS' condition (Causal Selection) data
training_data_CS <- list(
list(
input = c(A=1, B=-1, C=1, D=-1),
target = c(E=1),
possible_explanations = c("C"),
repetition_count = 4
),
list(
input = c(A=1, B=1, C=1, D=1),
target = c(E=1),
possible_explanations = c("C"),
repetition_count = 1
),
list(
input = c(A=-1, B=1, C=-1, D=-1),
target = c(E=-1),
possible_explanations = c("C"),
repetition_count = 1
),
list(
input = c(A=1, B=1, C=-1, D=-1),
target = c(E=-1),
possible_explanations = c("C&D"),
repetition_count = 4
)
)
# 'AC' condition (Any Cause) data
training_data_AC <- list(
list(
input = c(A=1, B=-1, C=1, D=-1),
target = c(E=1),
possible_explanations = c("A&C"),
repetition_count = 4
),
list(
input = c(A=1, B=1, C=1, D=1),
target = c(E=1),
possible_explanations = c("A", "D", "A&D", "A&C", "C&D", "A&C&D"),
repetition_count = 1
),
list(
input = c(A=-1, B=1, C=-1, D=-1),
target = c(E=-1),
possible_explanations = c("A", "D", "A&D", "A&C", "A&C&D"),
repetition_count = 1
),
list(
input = c(A=1, B=1, C=-1, D=-1),
target = c(E=-1),
possible_explanations = c("C", "D"),
repetition_count = 4
)
)
# Testing data (16 possible input combos)
testing_data <- list(
# 1) A= 1, B= 1, C= 1, D= 1 => E=1
list(input = c(A=1, B=1, C=1, D=1), target = c(E=1)),
# 2) A= 1, B= 1, C= 1, D=-1 => E=1
list(input = c(A=1, B=1, C=1, D=-1), target = c(E=1)),
# 3) A= 1, B= 1, C=-1, D= 1 => E=1
list(input = c(A=1, B=1, C=-1, D=1), target = c(E=1)),
# 4) A= 1, B= 1, C=-1, D=-1 => E=-1
list(input = c(A=1, B=1, C=-1, D=-1), target = c(E=-1)),
# 5) A= 1, B=-1, C= 1, D= 1 => E=1
list(input = c(A=1, B=-1, C=1, D=1), target = c(E=1)),
# 6) A= 1, B=-1, C= 1, D=-1 => E=1
list(input = c(A=1, B=-1, C=1, D=-1), target = c(E=1)),
# 7) A= 1, B=-1, C=-1, D= 1 => E=1
list(input = c(A=1, B=-1, C=-1, D=1), target = c(E=1)),
# 8) A= 1, B=-1, C=-1, D=-1 => E=-1
list(input = c(A=1, B=-1, C=-1, D=-1), target = c(E=-1)),
# 9) A=-1, B= 1, C= 1, D= 1 => E=1
list(input = c(A=-1, B=1, C=1, D=1), target = c(E=1)),
# 10) A=-1, B= 1, C= 1, D=-1 => E=1
list(input = c(A=-1, B=1, C=1, D=-1), target = c(E=1)),
# 11) A=-1, B= 1, C=-1, D= 1 => E=-1
list(input = c(A=-1, B=1, C=-1, D=1), target = c(E=-1)),
# 12) A=-1, B= 1, C=-1, D=-1 => E=-1
list(input = c(A=-1, B=1, C=-1, D=-1), target = c(E=-1)),
# 13) A=-1, B=-1, C= 1, D= 1 => E=1
list(input = c(A=-1, B=-1, C=1, D=1), target = c(E=1)),
# 14) A=-1, B=-1, C= 1, D=-1 => E=1
list(input = c(A=-1, B=-1, C=1, D=-1), target = c(E=1)),
# 15) A=-1, B=-1, C=-1, D= 1 => E=-1
list(input = c(A=-1, B=-1, C=-1, D=1), target = c(E=-1)),
# 16) A=-1, B=-1, C=-1, D=-1 => E=-1
list(input = c(A=-1, B=-1, C=-1, D=-1), target = c(E=-1))
)D Network and training code
train_network(), process_distribution(), sample_epochs()
### Generate Subject Training Data:
### This takes one of the training set list, with the appropriate observation, and then returns a set of 10 observations, with the appropriate repetitions, as well as a vector of explanation associated with each observation, again in accordance with the condition.
### It will be run once per subject, ensuring that the mapping between observations and the appropriate explanations stays constant throughout training epochs for a given subject.
generate_subject_training_data <- function(base_training_data) {
# For each observation in base_training_data, pick one explanation
# for all of its repeated trials, then shuffle.
expanded_list <- list()
input_neurons <- c("A","B","C","D")
for (obs in base_training_data) {
chosen_expl <- NULL
if (!is.null(obs$possible_explanations) && length(obs$possible_explanations) > 0) {
chosen_expl <- sample(obs$possible_explanations, 1)
}
# Build explanation vector
if (!is.null(chosen_expl)) {
vars <- strsplit(chosen_expl, "&")[[1]]
vars <- trimws(vars)
expl_vector <- setNames(rep(0, length(input_neurons)), input_neurons)
expl_vector[vars] <- 1
} else {
expl_vector <- setNames(rep(0, length(input_neurons)), input_neurons)
}
# replicate obs for repetition_count times
for (i in seq_len(obs$repetition_count)) {
expanded_list[[length(expanded_list)+1]] <-
list(input = obs$input,
target = obs$target,
explanation = expl_vector)
}
}
# Shuffle
expanded_list <- sample(expanded_list)
return(expanded_list)
}
# Helper functions:
tanh_activation <- function(x) {
tanh(x)
}
tanh_derivative <- function(x) {
1 - tanh(x)^2
}
# Global learning parameters
learning_rate <- 1 # We fix this to 1 throughout. It is made a parameter here for readers interested in playing around with it.
num_hidden_neurons <- 4 # Same for this
### Early stopping rule just in case we do a manipulation error with the number of epochs; this will never be reached in all normal uses.
threshold <- 1e-6
max_epochs <- 5000
#training function
train_network <- function(
num_epochs,
num_hidden_neurons,
alpha,
training_data_sampled
) {
# 1) Define neuron sets
input_neurons <- c("A","B","C","D")
output_neurons <- c("E")
hidden_neurons <- paste0("H", seq_len(num_hidden_neurons))
num_inputs <- length(input_neurons)
num_hidden <- num_hidden_neurons
num_outputs <- length(output_neurons)
# 2) Initialize weights & biases
W_input_hidden <- matrix(
rnorm(num_inputs * num_hidden, mean=0, sd=0.1),
nrow = num_inputs, ncol = num_hidden,
dimnames = list(input_neurons, hidden_neurons)
)
W_hidden_output <- matrix(
rnorm(num_hidden * num_outputs, mean=0, sd=0.1),
nrow = num_hidden, ncol = num_outputs,
dimnames = list(hidden_neurons, output_neurons)
)
bias_hidden <- setNames(runif(num_hidden, min=-0.1, max=0.1), hidden_neurons)
bias_output <- setNames(runif(num_outputs, min=-0.1, max=0.1), output_neurons)
previous_loss <- Inf
# 3) Training loop
for (epoch in seq_len(num_epochs)) {
total_loss <- 0
# Shuffle the data inside each epoch
training_data_epoch <- sample(training_data_sampled)
for (sample in training_data_epoch) {
activations_input <- sample$input # e.g. c(A=..., B=..., C=..., D=...)
target_output <- sample$target # c(E= ±1)
explanation_vec <- sample$explanation[input_neurons] # here written as 0/1 vector, so we can use the ifelse() function below; but the notation in terms of -1/1 in the thesis chapter is more natural
# Single-alpha attention scheme:
# e^(alpha) for mentioned, e^(-alpha) for unmentioned
attention_vector <- ifelse(explanation_vec == 1, exp(alpha), exp(-alpha))
# Adjust inputs
activations_input_adj <- activations_input * attention_vector
# Forward pass
net_hidden <- as.vector(activations_input_adj %*% W_input_hidden + bias_hidden)
act_hidden <- tanh_activation(net_hidden)
net_output <- as.vector(act_hidden %*% W_hidden_output + bias_output)
act_output <- tanh_activation(net_output)
# Error
error_output <- act_output - target_output
loss_sample <- sum(error_output^2)/num_outputs
total_loss <- total_loss + loss_sample
# Backprop
delta_output <- error_output * tanh_derivative(net_output)
delta_hidden <- as.vector((W_hidden_output %*% delta_output) *
tanh_derivative(net_hidden))
# Gradients
grad_W_input_hidden <- (activations_input_adj %*% t(delta_hidden))
grad_bias_hidden <- delta_hidden
grad_W_hidden_output<- (act_hidden %*% t(delta_output))
grad_bias_output <- delta_output
# Update (no regularization)
W_hidden_output <- W_hidden_output - learning_rate * grad_W_hidden_output
bias_output <- bias_output - learning_rate * grad_bias_output
W_input_hidden <- W_input_hidden - learning_rate * grad_W_input_hidden
bias_hidden <- bias_hidden - learning_rate * grad_bias_hidden
}
# Early stopping
avg_loss <- total_loss / length(training_data_sampled)
if (abs(previous_loss - avg_loss) < threshold) {
break
}
previous_loss <- avg_loss
}
# 4) Store final network
trained_network <- list(
W_input_hidden = W_input_hidden,
W_hidden_output= W_hidden_output,
bias_hidden = bias_hidden,
bias_output = bias_output
)
# 5) Evaluate on the test data with the final network
total_error_original <- 0
for (test_sample in testing_data) {
test_inp <- test_sample$input
target_out <- test_sample$target
net_h <- as.vector(test_inp %*% W_input_hidden + bias_hidden)
act_h <- tanh_activation(net_h)
net_o <- as.vector(act_h %*% W_hidden_output + bias_output)
act_o <- tanh_activation(net_o)
# binarize
pred <- ifelse(act_o>0, +1, -1)
err <- sum(abs(pred - target_out))/2
total_error_original <- total_error_original + err
}
num_samples_test <- length(testing_data)
accuracy_original <- (num_samples_test - total_error_original)
# 6) Return final object
list(
W_input_hidden = W_input_hidden,
W_hidden_output = W_hidden_output,
bias_hidden = bias_hidden,
bias_output = bias_output,
accuracy_original = accuracy_original
)
}
### Then a function to aggregate the results of several networks as we generate predictions out of each of them:
process_distribution <- function(dist_name,
sampled_epochs,
num_hidden_neurons,
alpha,
training_data)
{
# Filter out invalid epochs
sampled_epochs <- sampled_epochs[sampled_epochs>0 & sampled_epochs<=max_epochs]
if (length(sampled_epochs)==0) {
warning("No valid epochs for ", dist_name)
return(NULL)
}
results_list <- list()
for (num_epochs in sampled_epochs) {
# 1) Generate subject-specific training data
training_data_subj <- generate_subject_training_data(training_data)
# 2) Train
res <- train_network(
num_epochs = num_epochs,
num_hidden_neurons = num_hidden_neurons,
alpha = alpha,
training_data_sampled= training_data_subj
)
results_list[[length(results_list)+1]] <- res
}
# 3) Build data frame from results
df <- data.frame(
Epochs = sampled_epochs,
Accuracy_Original = sapply(results_list, function(x) x$accuracy_original),
Distribution = dist_name,
NumHiddenNeurons = num_hidden_neurons,
Alpha = alpha
)
df
}
### Number of training epochs per simulated learner.
### The dissertation frames this as an exponential distribution with rate lambda
### (epochs = round(Exp(lambda)), at least 1). Here the count is drawn from a
### Poisson distribution with mean 1/lambda, the count distribution that
### corresponds to exponential waiting times; at least one epoch is always run.
### With lambda = 1000 (mean 0.001) every learner trains for exactly one epoch.
sample_epochs <- function(n, mean_epochs) {
pmax(1, stats::rpois(n, mean_epochs))
}
### Simulate n learners in one condition: OBS uses the AC observations with alpha = 0
simulate_condition <- function(condition, alpha, mean_epochs, n = 500, num_hidden = 4) {
base <- if (condition == "CS") training_data_CS else training_data_AC
a <- if (condition == "OBS") 0 else alpha
process_distribution(dist_name = condition, sampled_epochs = sample_epochs(n, mean_epochs),
num_hidden_neurons = num_hidden, alpha = a, training_data = base)
}E Fitting and reproduction
_R/compute/attention.R
# Attention model (attention notebook Figs. 4, 5, A1; Tables 2, B1; Appendix E).
# 1. accuracy against alpha with one training epoch, per learner (for Monte Carlo intervals)
# 2. grid fit of alpha and the mean number of epochs, minimising the squared error of the
# three between-condition differences (the original sse_total objective)
# 3. learner-level accuracies at the best fit, for the data-versus-model figure
# 4. response-time distribution fits (normal, ex-Gaussian, exponential)
source("_R/compute/common.R")
source("_R/attention/stimuli.R"); source("_R/attention/model.R"); source("_R/attention/data.R")
suppressMessages({ library(gamlss); library(gamlss.dist) })
seed <- 2030; set.seed(seed); started <- Sys.time()
N <- as.integer(Sys.getenv("NB_LEARNERS", "4000"))
raw <- attention_raw(); scores <- attention_scores(raw)
obs <- tapply(scores$score, scores$condition, mean)
# 1. alpha sweep, one epoch
alphas <- c(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 1, 1.25, 1.5, 2)
sweep <- do.call(rbind, lapply(alphas, function(a) do.call(rbind, lapply(c("AC", "CS"), function(cn) {
d <- simulate_condition(cn, a, 0.001, n = N)
data.frame(alpha = a, condition = cn, n = nrow(d), mean = mean(d$Accuracy_Original), sd = sd(d$Accuracy_Original))
}))))
obs_one <- simulate_condition("OBS", 0, 0.001, n = N)
sweep_obs <- data.frame(n = nrow(obs_one), mean = mean(obs_one$Accuracy_Original), sd = sd(obs_one$Accuracy_Original))
message("sweep done")
# 2. grid fit
grid_alpha <- seq(0, 2, by = 0.1); grid_mu <- c(0.001, 0.5, 1, 2, 3, 5, 8, 12)
fit <- list()
for (mu in grid_mu) {
m_obs <- mean(simulate_condition("OBS", 0, mu, n = N)$Accuracy_Original)
for (a in grid_alpha) {
m_ac <- mean(simulate_condition("AC", a, mu, n = N)$Accuracy_Original)
m_cs <- mean(simulate_condition("CS", a, mu, n = N)$Accuracy_Original)
sse <- ((m_ac - m_obs) - (obs[["AC"]] - obs[["OBS"]]))^2 + ((m_cs - m_obs) - (obs[["CS"]] - obs[["OBS"]]))^2 +
((m_cs - m_ac) - (obs[["CS"]] - obs[["AC"]]))^2
fit[[length(fit) + 1]] <- data.frame(alpha = a, mean_epochs = mu, OBS = m_obs, AC = m_ac, CS = m_cs, sse = sse)
}
message("grid mu = ", mu)
}
fit <- do.call(rbind, fit)
best <- fit[which.min(fit$sse), ]
# 3. learner-level accuracies at the best fit (larger sample)
at_best <- do.call(rbind, lapply(c("OBS", "AC", "CS"), function(cn) {
d <- simulate_condition(cn, best$alpha, best$mean_epochs, n = 4 * N)
data.frame(condition = cn, epochs = d$Epochs, score = d$Accuracy_Original)
}))
# 4. response times
rts <- attention_rts(raw)
x <- rts$rt
fits <- list(normal = gamlss(x ~ 1, family = NO, trace = FALSE),
`ex-Gaussian` = gamlss(x ~ 1, family = exGAUS, trace = FALSE),
exponential = gamlss(x ~ 1, family = EXP, trace = FALSE))
rt_table <- data.frame(model = names(fits), df = sapply(fits, function(f) f$df.fit),
AIC = sapply(fits, AIC), BIC = sapply(fits, BIC))
rt_params <- lapply(fits, function(f) c(mu = fitted(f, "mu")[1], sigma = if ("sigma" %in% f$parameters) fitted(f, "sigma")[1] else NA,
nu = if ("nu" %in% f$parameters) fitted(f, "nu")[1] else NA))
save_result(list(observed = scores, sweep = sweep, sweep_obs = sweep_obs, grid = fit, best = best, at_best = at_best,
rt = x, rt_removed = rts$removed, rt_table = rt_table, rt_params = rt_params),
"attention", seed, list(learners = N, alphas = alphas, grid_alpha = grid_alpha, grid_mu = grid_mu), started, code_dirs = "_R/attention")| Result | Seed | Learners per cell | Run time | Date (UTC) |
|---|---|---|---|---|
attention | 2030 | 4000 | 43.1 min | 2026-10-10 |