⁸For different algorithms, see (Sutton and Barto, 2018); for example, a recursion similar to that in footnote 6 above could be used.