To continue my habit of writing this stuff out in an attempt to work more Vim commands into my everyday ingrained vimming habits and memory: CTRL-I and CTRL-O (in Normal mode) are enjoyable. I have long used '' (two single-quotes) to jump me back and forth between the last cursor jump and the current position. But the two commands above jump you back deeper through the jumplist. This works on a per-window basis, meaning it'll jump you between buffers too. A "jump" is defined in the help document, but it includes G, / (i.e. searches), % and friends, and '.

See also :h jump-motions.