3A basic exposition of discounting is given by Sutton and Barto (2018). For discussions of different kinds of discounting, and in particular for comparisons between exponential and hyperbolic discounting, see Dasgupta and Maskin (2005); Ainslie (2001).