So far in this book, we have seen several computational ideas related to suffering. We started by considering two basic mechanisms for suffering in Chapter 2: frustration and threat to the person or the self. We first defined frustration as not reaching a goal (Chapter 3) and later in terms of reward loss and reward prediction error (Chapter 5). In fact, these two kinds of frustration align well with the dual-process theory—slow vs. fast or GOFAI vs. neural networks—considered in Chapter 8. In Chapter 6, we further argued that the concept of self includes higher-order desires related to self-evaluation or self-preservation, and these can also be frustrated. Chapter 7 developed a theory of threats and fear based on predicted reward loss in the future. To sum up, we obtained a theory in which suffering is based on error signals given by frustration or prediction of frustration. In this chapter, we summarize the ideas of the previous chapters, emphasizing the many different forms that frustration can take.
Consider a case where you are yourself going to fetch the orange juice from the fridge. You formulate a plan which involves high-level actions such as going to the fridge, opening the door, etc. Once you are in front of the fridge, your habit-based system suggests you open the door by a certain sequence of muscle contractions which you have performed hundreds of times and which has become quite automated.
Now, suppose you follow the habit-based system and pull the door handle, but the door does not quite open. This kind of “frustrates” your habit of opening the door. But do you suffer? Probably not very much; you just pull again with more force, and if it opens, you hardly register anything out of the ordinary happened. In contrast, if you don’t get the juice at all—because the door is somehow broken and does not open at all— your long-range plan is frustrated, and you will definitely suffer. There is a good reason for that suffering: all that planning and even the walking was in vain. A strong error signal has to be sent throughout your brain, and that is suffering.
This example points out one important aspect of action selection: its temporally hierarchical nature, involving simultaneous computations on different time scales.1 In the brain, there are also processes operating at many different time scales. Reality is, of course, a bit more complex than the clean division into planning and habit-based actions we have discussed so far.
Some form of frustration can be operating on many different levels simultaneously. In one extreme, the agent may be planning long action sequences, and if they fail, frustration ensues in the sense of not reaching the goal. In the other extreme, a habit-based reinforcement learning system builds predictions on what kind of rewards or changes in state-values are associated with different actions, and computes whether there is reward loss or an RPE. Predictions are made on a millisecond time scale as well as on the time scale of days if not years. Each such time scale has its own learning mechanism using its own errors.2
Such division into time scales brings us to the concept of intention—defining intention as commitment to a goal, as discussed in Chapter 3. The point in intentions is to partly resolve conflicts between long-term and short-term optimization. I can have many desires simultaneously and spend some time thinking about each of them, and perhaps even planning each of them to some extent. But I’m not really hoping to reach all the goals related to those desires. Once I decide to commit to one of the goals, that is what sets the goal, which can then be frustrated. I would argue that in the case of planning, frustration is not so much due to desire itself but to the ensuing intention. This is in line with the more elaborate expositions of the Buddha’s philosophy on suffering which divide desire into initial desire and a later part called attachment (also translated as “clinging” or “grasping”). Attachment is a process where after an initial feeling of desire (“Nice, chocolate, I would like to have it”), you firmly attach to the object of your desire (“I must have that chocolate”). This distinction seems to be similar to the distinction between desires and intentions in our terminology. Buddhist philosophy suggests a central role for attachment, or intention, in the process which creates suffering. While such attachment or intention is not necessary for frustration to occur, I propose that it greatly amplifies it. This is logical because intentions consider longer time scales, and thus an error related to intention is more serious, since more time and energy were lost in formulating and executing the plan that failed.
We have also seen two different kinds of frustration: not reaching a goal vs. incurring a reward loss. One underlying difference between the two cases is that reward loss is based on violation of expectations, while not reaching the goal is in line with the typical definition of frustration as not getting what one wants, i.e. violation of desires. It may thus seem that our definitions are to some extent contradictory. One way of resolving this is to consider that the term “expectation” may have different meanings in different contexts.3 The agent is executing a plan in order to get to the goal state, and it is in that sense “expecting” to get to that goal state. Earlier, we saw (🡽) how Epictetus talks about desire “promising” the attainment of its object. Thus, the expectation related to planning could simply be defined as the goal state being reached. Then, reward loss would be the same as the frustration of not reaching the goal, that is, the object of the desire (using the definition of desire given in Chapter 3).
Alternatively, we could see frustration of desires and reward loss (based on expectation) as two distinct, if closely related phenomena, both of which produce suffering. What they have in common is that some kind of error occurred. This opens up the possibility of a very general viewpoint where the connection between suffering and error signalling does not need to be concerned with goals or rewards at all. We all know that it is unpleasant if we expect something and then it does not happen, even if the event we were predicting was neutral in the sense of providing no reward. Thus, it is possible that there is some kind of suffering in almost any prediction error.4 Most interestingly, it has been proposed that dopamine signals prediction errors even for events not related to reinforcement, so it might provide a neural mechanism for general signalling of errors.5
The meaning of such errors is further modified by the context. If you are deliberately engaged in the learning of, say, a new skill, errors are quite natural, and you are likely to feel less frustration; in a sense, you are expecting that there are errors. Or, if your prediction of the reward is uncertain, i.e. only very approximate, the frustration is likely to be weaker. We will have much more to say about such effects in later chapters.
Depending on the context, what I call frustration in this book can actually correspond to different concepts with slightly different meanings. Disappointment is a closely related term; it can also be used when no particular action is taken by the agent while the world turns out to be worse than expected. Irritation and even anger can also be used to describe feelings very similar to frustration. While anger can more specifically mean the interpersonal feeling of anger towards other people, it is often caused by the fact that their actions lead to frustration (more on this in Chapter 10). Regret can be seen as frustration specifically based on our own actions,6 often amplified by recalling past frustration (more on this in Chapter 11).
Frustration on different time scales leads us to the frustration of self-needs treated in Chapter 6. Self-needs often work on time scales of days, months, even years, thus an even longer time scale than ordinary planning and attachment. These different time scales can actually be related to van Hooft’s different kinds of frustration discussed in Chapter 2: frustration of biological functioning, of desires and emotions (in his terminology), of more long-term life goals, and even of the sense of the meaning of one’s existence. It may very well be that such self-related frustration produces some of the very strongest frustration and suffering. Frustration of self-needs is also closely connected to suffering coming from threats to the “intactness of the person” à la Cassell, thus combining the two mechanisms of suffering.7
In Chapter 7, self-related frustration was even considered as a means to reduce the concept of threat to a form of frustration—of self-needs such as the desire for safety—to obtain a unified theory on suffering. However, while such a theoretical simplification has some interest, it was considered to go a bit too far; the connection between frustration and threat is certainly more complex. In Chapter 7, we actually saw a fundamental distinction that can be made between frustration and threats: threats are about predicting that a bad thing might happen, while frustration is fundamentally about realizing the bad thing did already happen. Nevertheless, it is interesting to ask if it is also possible to see the connection between frustration and threats from the opposite angle: can frustration be seen as a special case of a threat to the self?
We can indeed consider that failing in a task implies a threat to one’s self-image that Cassell talks about,8 and which is related to the self-evaluation of Chapter 6. A frustrating experience may imply that the agent’s positive self-image is not correct, and that it has to change its self-image so that it will consider itself less competent than it thought earlier. In this sense, frustration is a threat, especially if the frustration did not yet change the self-image but implied a certain probability that it should be done in the future. If you fail at a work task today, that is frustration but you may still think you know how to do your job. But the failure will increase your perceived probability that one day, you have to admit that you just don’t know how to do your job at all. That is a threat to your person, implying the possibility that one day you will have to update your self-image to a more negative one. Likewise, any frustration could be considered to imply a threat to the very survival of a biological agent because it suggests that the agent’s decision-making system is not working very well in the current environment, which could lead to life-threatening problems in the future. This way, frustration can be reduced to a special case of the threat of intactness of the person.9 This is in line with the thinking prevalent in Mahayana Buddhist schools, where the self is seen as the source of all desires and all suffering.
This intricate interplay between frustration and threat is seen in the very definition of threat in Chapter 7, where threat is based on anticipation of reward loss. This means that many of the properties of frustration just discussed also apply to threat: threat operates on different time scales, and threats can be based on frustrating expectations or on frustrating desires. Most importantly, if there were no reward loss, there could be no threat either. In this fundamental sense, it is the threat that is secondary and can be reduced to frustration; indeed, all fear is fear of frustration. This intimate connection will be important in Part II where we consider the conditions creating suffering, and Part III where we consider interventions to reduce suffering. As a sneak peek, consider the proposal by Seneca, a Roman Stoic, for reducing threat-based suffering: “Cease to hope (...) and you will cease to fear”.10
Going a bit beyond the theories of the preceding chapters, I am tempted to think that desire (or aversion) in itself, especially when combined with intention, can immediately create some kind of suffering even before any frustration, and even in the absence of any specific threat.11 It could be that whenever there is desire or aversion, the system predicts that there will be frustration with some probability, and this constitutes a threat, creating suffering.12 It is in fact clear that fear, which is an aversion towards possible future events, does create suffering in itself based on a threat; perhaps other kinds of aversion and even desire share similar mechanisms. Another possibility is that such suffering is based on a general error-signalling mechanism: the internal representation of a goal state which is different from the current state is an error that may automatically lead to the triggering of an error signal and to suffering.13 Understanding this kind of suffering is an important question in future research.14
To recapitulate, Part I of this book described a wide spectrum of frustration-related error signalling. Not reaching a goal, not getting an expected reward, or making an error in predicting any event, can all be seen in this same framework. They work on different time scales, and use different systems in the dual-process framework. It seems that particularly strong suffering is obtained by frustration of planning, and even stronger by frustration of self-needs.
Next, we will try to understand why there is frustration in the first place. On some level, it is obvious that we cannot always reach our goals, or get what we want, if only because of the limitations in our physical skills and strength: We cannot move mountains. The world is also inherently uncertain and unpredictable, so even the perfect plan may fail because something unexpected happens. Yet, more interesting for our purposes are the cognitive limitations. As argued earlier, cognition is something that can be relatively easily intervened on, and modified to some extent. Thus it is more feasible to reduce suffering by focusing on the cognitive mechanisms, instead of trying to develop devices that physically move mountains. Therefore, it is crucial to understand in as much detail as possible how various processes of information-processing contribute to suffering.
We have already seen several information-processing limitations that can produce or amplify frustration. For example, planning is difficult due to the exponential explosion of the number of paths, which means our plans may be far from optimal. We need a lot of data for learning: data may be lacking to build a good model of the world, or to learn quantities such as state-values. Categories are often used in action selection—in particular, if the world is divided into states—but these categories may not even be well-defined. The cognitive system may be insatiable and always want more and more rewards. There are several self-related needs which can create particularly strong suffering by mechanisms related to frustration.
Part II goes into more depth regarding such limitations that produce frustration, focusing on the origins of uncontrollability and uncertainty. Later, Part III will consider methods for reducing suffering, mainly by reducing frustration. I will summarize all the different aspects of frustration in a single “equation” (🡽) and propose various methods or interventions to reduce frustration based on the theory of Parts I and II. Such interventions will largely coincide with what Buddhist and Stoic philosophies propose, and include mindfulness meditation as an integral tool.