I found the discussion at the bottom of this page (Thomas Koch's 2nd reference) helpful:
Decomposing emacs' use of external storage more or less according to the enumerated criteria is simply good hygiene. It does not means that you have to embrace the freedesktop standard.
In my own case I have already partitioned my emacs world into ~/emacs/ (a non-hidden directory containing elements I manage via github) and ~/.emacs.d/ (everything recreatable). In particular, with the advent of better package management I put all downloaded packages beneath ~/.emacs.d/, on the theory that they can be recreated via download. By contrast I put the configuration for those packages in ~/emacs/.