²¹It may be intuitively clear that acknowledging the uncertainty of perception should lead to “weaker” signalling of frustration; in this footnote, I explain how to make that idea more rigorous. We can consider, as an illustrative example, one of the simplest online learning tasks, namely linear regression. There, we minimize a quantity such as ∑ _t(y_t - ax_t)²∕σ _t² where x is input, y is output, σ_t² is noise level, and a is a parameter to be estimated. The magnitude of the error signal for each data point is proportional to the inverse of the noise level σ_t². Thus, for a high noise level (large uncertainty), the error signal is smaller. If the noise level is estimated separately for each data point (or time point t), this will have the effect of reducing the error signal at time points where there is a lot of uncertainty as modelled by the noise level σ_t². The concrete algorithm used here might be what is called the delta rule; see Korenberg and Ghahramani (2002) as an example of a related if slightly more complex model, and Kendall and Gal (2017) for a more sophisticated deep learning model. In the context of reinforcement learning, Mai et al. (2022) propose a closely related weighting for RPE, where indeed we see how the learning proceeds by minimizing the expected (squared) RPE so that it is down-weighted by the estimated variance of the RPE.