¹⁹Here we focused on the uncertainty in obtained reward. It could also be asked if there can be uncertainty in the expected reward. In an orthodox Bayesian interpretation, it may in fact not be possible to say that there is any uncertainty about the expected reward, since the expected reward is a subjective quantity, something purely defined by what the agent believes and expects. In contrast, in a frequentist intepretation, the expected reward is an objective quality in the outer world (how much the agent would get on average if it repeated the same action many times) that could further be considered a parameter in a statistical model. Therefore, it can be misestimated, thus adding to the uncertainty of reward loss. Notwithstanding such theoretical arguments, I think it is clear that for biological organisms, understanding the real evolutionary value of, say, a piece of food may actually be a highly complex process involving a lot of learning and computation, so it can surely go wrong, as in the case of sugary food, which means it is meaningful to say that there is uncertainty about the expectation.