From the Emacs docs:

Each buffer has a default directory which is normally the same as the directory of the file visited in that buffer. When you enter a file name without a directory, the default directory is used. If you specify a directory in a relative fashion, with a name that does not start with a slash, it is interpreted with respect to the default directory. The default directory is kept in the variable default-directory, which has a separate value in every buffer.

The command M-x pwd displays the current buffer's default directory, and the command M-x cd sets it (to a value read using the minibuffer). A buffer's default directory changes only when the cd command is used. A file-visiting buffer's default directory is initialized to the directory of the file it visits. If you create a buffer with C-x b, its default directory is copied from that of the buffer that was current at the time.

This is extremely annoying. Vim leaves my working directory the hell alone, why doesn't Emacs? Vim lets you set a global working directory, and selectively (and explicitly) change it on a per-buffer basis. This is what I want.

But in Emacs, every time you open a file, the working directory automatically changes to the directory of that file. If you have multiple files open in Emacs (which of course you do), every time you move the cursor between windows, or look at a new file in your current window, your working directory just changed out from under you.

So say you open some files. Then you want to start SLIME. So you C-x 2 to split and open a new window, in which to start SLIME. Except (annoyance #448,547) Emacs doesn't open a window with a new BLANK file, as Vim sanely does via CTRL-W n; instead it puts the file you're looking at into BOTH windows, which is absolutely never what you want. As a result, the working directory of BOTH windows is the directory of whatever file you were looking at. So every time you start SLIME, you have to make sure you M-x cd back to the proper working directory first, because otherwise your Lisp process is randomly going to start in the wrong directory.

But you can change the directory of the SLIME buffer after you start SLIME. Just M-x cd or ,cd. Except if you're unlucky enough to be using SLIME for Clojure, which may have set its classpath based on its working directory when you started it. In that case you have to restart SLIME.

But if you kill SLIME, Emacs (annoyance #448,548) jams another random file into the window where SLIME just was. So your working directory just changed again! Or did it? Depends what Emacs decided to put into your window. You may think that, while SLIME is running, M-x cd and/or ,cd and then ,restart-inferior-lisp may do what you want, but you would be wrong; it always reverts back to the original working directory from when SLIME was first started.

Lost? Confused? tl;dr? Welcome to Emacs. So now I'm looking through the encyclopedic tangled mess of Emacs documentation to try to figure out how to get Emacs not to change my working directory, ever, unless I say so. This is hindered as always by (annoyance #448,549) Emacs' arcane and non-standard terminology. So far, I have committed to memory that "current directory"/"working directory" in Emacs is instead called "default directory". And you don't open files, you "visit" them. Command line? No, "minibuffer".