⁷This footnote discusses the difference between threat and risk as well as the exact measures of used in more detail. While the economic literature considers many different kinds of risks, what we call threat is a special case, more specifically related to the downside risk, i.e. the risk of outcomes which are particularly bad. Therefore, I make a clear distinction between threat and risk, and use “uncertainty” synonymously with risk. I do not restrict myself to any specific definition of risk here but rather consider it as a general concept with many instantiations, of which threat is one. In conventional economic theory, especially finance, risk is modelled by the variance of the quantity to be maximized (here, the total future reward). Alternatively, economic theory uses concave utility functions to induce risk-averse behavior, which has been applied in reinforcement learning by Wu et al. (2021); Zhang et al. (2020), but the basic effect seems quite similar to using variance. However, the crucial point here is that using variance seems rather inadequate to measure threat since it does not focus on downside risk. Thus, I prefer to equate large variance with general uncertainty and one kind of risk, but not threat. Threat, as defined in this book, is all about the probability of bad outcomes, while variance is measuring uncertainty in both positive and negative directions; if a very good outcome is possible, that also increases variance. How the downside risk of a distribution should exactly be defined and measured to measure threat is a complex question to which I’m not going to give a single answer; I discuss some options in what follows. One well-known economic theory which is relevant here considers the probability of “ruin” (i.e. bankruptcy), typically used in insurance theory. Such ruin could be equated to the destruction (death) of an agent, and is not completely different from our concept of threat, especially in the context of evolutionary modelling. Lipton et al. (2016) propose a framework related to ruin probabilities in reinforcement learning, measuring the distance to what they call catastrophic events, which could be death and serious injury in the case of a biological agent, or, from the viewpoint of making robots safe to humans, it could be defined as the robot injuring a human being; see also (Martin et al., 2016). Further possibilities for modelling threat can be found in financial theory. One option is skewness (Ebert and Karehnke, 2021; Trautmann and van de Kuilen, 2018), which is a measure of the asymmetry of a probability distribution; however, it is not clear if it is enough in itself as a measure of threat: it may need to be combined with variance. Fortunately, financial theory has also developed measures such as conditional value-at-risk, also called expected shortfall, which measures the negative tails and could in fact be quite suitable as a measure of downside risk of reward loss, and thus threat, in our framework. Bellemare et al. (2023) discusses them from the viewpoint of reinforcement learning.