If in emacs -Q, I evaluate:
(set-window-margins nil 5 5)
(set-window-fringes nil 16 16)
and switch to a different buffer, the fringes/margins will be reset. They will still be frame-default if I switch back to the same buffer.
Btw, I think it would make more sense, if the above settings where attached to a buffer and not a window. At the same time there is a buffer-local variable called window-size-fixed, which looks like it should be a window property.