[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Call for Google Summer of Code 2021 project ideas

From: John Snow
Subject: Re: Call for Google Summer of Code 2021 project ideas
Date: Tue, 12 Jan 2021 16:10:48 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0

On 1/11/21 6:47 AM, Stefan Hajnoczi wrote:
Dear QEMU, KVM, and rust-vmm community,
QEMU will apply for Google Summer of Code
(https://summerofcode.withgoogle.com/) again this year.  This internship
program offers paid, 10-week, remote work internships for
contributing to open source.  QEMU can act as an umbrella organization
for KVM kernel and rust-vmm projects too.

Please post project ideas on the QEMU wiki before February 14th:

What's new this year:
  * The number of internship hours has been halved to 175 hours over
    10 weeks. Project ideas must be smaller to fit and students will have
    more flexibility with their working hours.
  * Eligibility has been expanded to include "licensed coding school or
    similar type of program".

Good project ideas are suitable for 175 hours (10 weeks half-day) work by a
competent programmer who is not yet familiar with the codebase.  In
addition, they are:
  * Well-defined - the scope is clear
  * Self-contained - there are few dependencies
  * Uncontroversial - they are acceptable to the community
  * Incremental - they produce deliverables along the way

Feel free to post ideas even if you are unable to mentor the project.
It doesn't hurt to share the idea!

I have one that is probably way too ambitious, but requires a particular skillset that might be of good interest to a student that has some experience in the area already.

The idea is for a TUI qmp-shell (maybe using urwid?) to create an irssi-like REPL interface for QMP. The idea would be to mimic the mitmproxy TUI interface (Check it out if you haven't!)

All the ideas below are extremely tentative to give a basic gist of what I mean; exact layouts/hotkeys/etc are for the sake of explanation only.

Essentially, I want an interface like this:

| QMP Mode                                                |
|                                                         |
| Welcome to the Qemu Machine Protocol shell.             |
| Please type /help or Ctrl+H to see available commands.  |
|                                                         |
|                                                         |
|                                                         |
| >                                                       |

commands are entered in the bottom and appear in a log window above, appearing most-recent last, like irssi works.

As an example, let's say we issue block-dirty-bitmap-add:

| > block-dirty-bitmap-add node=ide0hd0 name=mybitmap0 |

(...syntax up for debate...! We want something easy to parse, but easy to type. For structured data, that's Hard.)

we press [Enter] to submit the command, and the history now shows:

| QMP Mode                                      |
|                                               |
|                                               |
|                                               |
| [11:28] block-dirty-bitmap-add [Pending ...]  |
| >                                             |

After a few moments, the command resolves:

| QMP Mode                             |
|                                      |
|                                      |
|                                      |
| [11:28] block-dirty-bitmap-add [OK]  |
| >                                    |

The basic unit of history here is either an RPC call/response pair or an asynchronous QMP event.

Clicking on the history pane or pressing Alt+↑ or Alt+↓ (Mimics irssi keys for switching buffers, let's say it's user-configurable) to change focus to the history pane:

| Command History Mode                                     |
|                                                          |
|                                                          |
|                                                          |
| [11:28] block-dirty-bitmap-add [OK]                      |
| Use ↑ or ↓ to select a command, press [Enter] to inspect |

Using the arrow keys, you can highlight the history item and then press <Enter> to change the log pane to a details pane for that item, showing:

| Details: block-dirty-bitmap-add                [X] |
|                                                    |
| [11:28:01] -> {                                    |
|    "execute": "block-dirty-bitmap-add",            |
|    "arguments": {                                  |
|      "node": "ide0hd0",                            |
|      "name": "mybitmap0"                           |
|    }                                               |
| }                                                  |
|                                                    |
| [11:28:02] <- {                                    |
|   "return": {}                                     |
| }                                                  |
|                                                    |
| Press 'Q' or Backspace to close details pane       |

From here, maybe you could do a few things, like press a button to view the raw QMP I/O for the purposes of copy-pasting elsewhere, or just press Q/H/Backspace to go back to the history view.

Interesting abilities this style of shell can provide:

(1) The ability to render asynchronous events from the server in the log window without overwriting the user's command buffer. (In contrast to qmp-shell today, which cannot show async events until the user presses enter again to prompt a check for them.)

(2) The ability to asynchronously reconnect to servers on connection failures.

(3) The ability to collapse command:response pairs into one summarized line in the log, which can be expanded for more information later with a keypress.

(4) (As a future add-on, beyond current scope) The ability to add a status pane widget that might show current running jobs, the current machine execution status, etc.

As options, users should be able to:

1. Choose the history display mode live at runtime:
  a) Summarized command history (as shown above)
  b) Summarized JSON history
     (one item per message, no grouping, purely linear,
      shows a one-line summary of the message.)
  c) Full JSON history
     (one item per message, JSON messages are rendered in full.)
  d) Pretty JSON history
     (one item per message, JSON messages are pretty-printed.)

  e.g. Ctrl+V could change the "verbosity" of the messages, and the
  history log will be re-rendered to reflect the new setting.

2. Customize hotkeys for switching between command mode and history mode; quitting the details view, etc.

3. toggle on/off automatic command execution logging; or manually (Ctrl+S?) save history to file. The format might need some light markup to indicate directionality, timestamps, and so on.

Other things that might be of interest, but are beyond the scope of a GSoC/Outreachy internship:

1. readline-esque hotkey functionality for the command input bar, including command history, autocomplete for command names, etc. If we aren't using readline itself anymore, these things will have to be implemented somehow. There are libraries that support this: https://github.com/rr-/urwid_readline

2. Recording execution macros; subsets of commands you wish to replay easily.

3. Raw input mode: allow the user to paste a raw JSON command directly into the command bar.

4. Configuring our GTK interface to open up qmp-shell inside one of the terminal panes. (Kevin Wolf suggested this or something like it once, and I rather like the idea.)

5. "Shell Builtins" -- analogous to the same in bash -- additional commands and routines that help perform certain actions that might require complex, multi-part steps.

6. Augmented commands -- commands that don't require multi-part phases, but ones in which the shell might offer enhanced readouts for. An example would be integrating Vladimir's block graph rendering script to automatically fire off when block-query is issued. (If the user has a graphical session running, the libraries are installed, etc.)

See alsos; prior art, &c:

- irssi as an example of continuous, asynchronous history:


- mitmproxy, which heavily inspired this idea with its use of urwid to show a history of "packets" which can be interrogated later from within the interface


- aioconsole -- an async python REPL loop for interactively writing async code in python. It might have good ideas to steal!


- urwid: a TUI framework for Python; it is used by mitmproxy to implement its interface.


- urwid readline library: implements readline-esque hotkeys for urwid, but (maybe) doesn't respect readline config, if any. It is a reimplementation.


Target skillset for students:
- Intermediate Python
- Some exposure to async programming, coroutines, etc.
- Some experience with GUI programming concepts, in any toolkit or language
- A willingness to use gradually typed python O:-)

I don't expect this project to involve much, if any, C; but familiarity with the linux shell ecosystem will be an asset when choosing default behaviors, mimicking other tools, etc.

I have already written a fully asyncio QMP library; I would expect this new library to be used. It should be fully operational, but might require API changes when we begin to use it for a serious endeavor.

I will review project ideas and keep you up-to-date on QEMU's
acceptance into GSoC.

For more background on QEMU internships, check out this video:


reply via email to

[Prev in Thread] Current Thread [Next in Thread]