Re: About creating machines on the command line

[Luc Michel]

Hi Zoltan,

On 1/11/21 9:04 PM, BALATON Zoltan wrote:
> On Mon, 11 Jan 2021, Luc Michel wrote:
> > Hi,
> >
> > We would like to work on improving QEMU to be able to create custom 
> > machines from the command line. The goal here is to get feedback from 
> > the community and shape the future developments.
> >
> > The use case mainly comes from people working with tools to customize 
> > their designs, such as SiFive Core Designer 
> > (https://scs.sifive.com/core-designer). This kind of tools may allow 
> > creation or customization of a whole SoC, from the number of cores, to 
> > the memory and IRQ mapping of peripherals etc.
> >
> > The ultimate goal would be to be able to create any kind of machine on 
> > the command line. However we are aware that this is a substantial 
> > amount of changes in QEMU.
> >
> > In its current state, QEMU allows for very limited customization of 
> > existing machines on the command line. We identified the following 
> > limitations (feel free to add to the list):
> >
> >  - Most devices are not user creatable. Moreover, sysbus devices must 
> > be explicitly allowed by a machine to be creatable through `-device`,
> >
> >  - Memory regions cannot be created on the command line,
> >
> >  - Device MMIO regions cannot be mapped on a bus from the command line,
> >
> >  - GPIOs and clocks cannot be wired from the command line,
> >
> >  - CPUs are not sysbus devices (and not user-creatable). They need 
> > special care when creating them regarding system reset. Not being on a 
> > bus means that they must be reset manually on system reset. This is 
> > done in machines by registering a QEMU reset handler.
> >
> >  - Machine specific boot code is usually hard-coded into the machine 
> > itself. Some architectures (e.g. ARM) do factorize bootloader related 
> > code, but there is no standard way of doing that in QEMU.
> >
> > We don't want to address all those limitations at once. We plan to 
> > start with the following scenario:
> >
> >  - Start with a base machine that would handle CPU creation and 
> > bootloader stuff. Note that the "none" machine is probably not 
> > sufficient in its current shape. It does allow only one CPU and 
> > obviously does not handle the boot process.
> >
> >  - Allow for this machine every sysbus devices we want to be user 
> > command-line creatable (and mark them user_creatable if needed)
> >
> >  - Add command line options to create memory regions (probably ram 
> > ones at first)
> >
> >  - Add command line options to map a memory region (including sysbus 
> > device MMIO regions) onto another (memory_region_add_subregion)
> >
> >  - Add command line options to connect GPIOs and clocks.
>
> I'm not sure the command line would be the best interface for this, it's 
> already hard enough to get a simple VM running, adding more options 
> would just make it more obscure for mere users who just want to use 
> existing machines to create a VM. So I think some kind of config file to 
> define the machine would be better.

OK. My first thought here was to be not too much "intrusive" in the QEMU 
source code, so adding a couple a command line options seemed to be the 
way to go, as -device was already doing what we wanted.

> Another issue with command line was backward compatibility that it's 
> hard to change if it's used as an interface in scripts, etc. My personal 
> opinion is that the command line should be a user interface to run VMs 
> but for management apps and this kind of dynamic machine creation a 
> better defined API might be better. But I'm not in any way authorative 
> to decide so this is just my input to this.

Makes sense. I tend to agree with you. I'd appreciate more feedback on 
this to know if it's indeed a shared opinion.

> > This would hopefully allow for simple machines creation. We would then 
> > be able to use either the command line or the `-readconfig` option to 
> > create the machine.
>
> It was found that readconfig is limited and cannot even handle its 
> current purpose so it probably would need to be rewritten. This might be 
> a good opportunity for that. Maybe you need a better format than ini 
> file that can actually describe the machine parts and their connections 
> you want to create in a managable way. Question arises how general this 
> has to be to describe all the properties of objects and their 
> connections and if you'll end up with somthing like still needing to 
> write a board code which is the current situation just instead of 
> writing C you'd use some other language and call it a "config file". 
> What do you really want here? Avoid writing C or avoid recompiling QEMU 
> to experiment with devices?

Yes the idea is definitely to have a mean to create custom machines 
without the need of recompiling QEMU.

> In the latter case better support from the 
> QEMU monitor to create machines interactively could also be a solution 
> and if you could run scripts with monitor commands in it that could be 
> your config file.

Yes. We plan to have a script that will serve as a "frontend" anyways to 
generate the necessary commands (being on the command line, or monitor 
command or...)

> > Note that we are not planning to use QMP/HMP for now. From our 
> > understanding, a `device_add` request is always considered as 
> > hot-plug, which is not what we want here.
>
> Is that something that cannot be solved? Maybe by adding another command 
> or a state where starting with -S and adding devices before continue 
> would not be hotplug?

Sure. The goal here was to avoid adding yet another run state to QEMU. 
The command line happened to kind of support what we plan to do. If 
people are OK with enhancing the monitor for this, going the QMP way 
could be a solution. Does anyone has an opinion on this?

> These are just some random thoughts I had, feel free to ignore any of it.

Thanks for your feedback!

> Regards,
> BALATON Zoltan