[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Preview: portable dumper

From: Daniel Colascione
Subject: Re: Preview: portable dumper
Date: Mon, 28 Nov 2016 15:17:38 -0800
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux)

On Mon, Nov 28 2016, Stefan Monnier wrote:
>> I've been working on a portable dumper for GNU Emacs. The attached patch is
>> an early version of this work.
> Cool!
>> An additional optimization follows, although this part isn't implemented
>> yet: we can define a "preferred load address" for the dump and write
>> relocation information such that if the dump and Emacs end up being where we
>> expect them to be, we don't have to perform any relocations at all.
> Oh, so the in-file format is also a valid in-memory format?  Great.
> AFAIK, that's the crucial difference between a "fast .elc loader" and
> your approach.  This should be key to get really good performance
> (e.g. good enough that it doesn't slow down bootstrap noticeably).

Yes. The file is full of valid memory images of Lisp objects --- that's
why I'm guessing it might be a bit more verbose than ELC,
pre-compression. If you guess correctly at dump time what addresses the
Emacs and dump images will have, you can direct mmap it directly and
run.  With a non-PIC executable, you can do a pretty good job of that
and get performance a bit better than unexec, in theory.

If you load a dump and find that you've guessed wrong on the base
addresses, you need to relocate the image, but that can be done cheaply
and gradually, on access.

>> would benefit from compression, which will reduce IO loads: LZ4 reduces the
>> dump size for me from ~12MB to ~4MB. As in the mmap case, we can throw away
> Hmm... interesting the "dump one big .elc" approach I played with (which
> kinda works right now, tho with the known performance issues) leads to
> a dumped.elc file of a size a bit below 4MB (which gzip compresses down
> to ~1MB).
> I assume your system is using 64bit Lisp_Object, right?  So on a 32bit
> Lisp_Object your dump size is probably closer to 6-7MB.

Right --- I've tested on 64-bit only. I imagine testing 32-bit and
wide-int might reveal some latent bugs.

>> functions pdumper.h --- but even so, it's within 100ms or so of an
>> unexeced Emacs.
> 0.1s will still lead to a noticeable slow down of bootstrap, but at
> least it should be good enough for the usual end-user who doesn't run
> Emacs in batch mode.

Oh, of course. But that's without any attention paid to
optimization. I'm sure it can go much faster.

>> It's also possible to dump an already-dumped Emacs, so it should be possible
>> for users to have their own dump files.
> [ FWIW in theory my dumped.elc approach could also work on an
>   already-dumped Emacs, for use by end-users (it basically just prints the
>   contents of the obarray), tho it would indeed need extra work to weed out
>   problems printing objects like markers and such.  ]

Yeah --- then both approaches get into the problem of how to dump
arbitrary object graphs, which in practice ends up being "dump the whole
heap except for processes, windows, and frames."

reply via email to

[Prev in Thread] Current Thread [Next in Thread]