On Mon, Jul 9, 2018 at 8:06 PM Richard W.M. Jones <
address@hidden> wrote:
On Mon, Jul 09, 2018 at 07:02:50PM +0200, Kevin Wolf wrote:
> Am 09.07.2018 um 18:52 hat Richard W.M. Jones geschrieben:
> > On Mon, Jul 09, 2018 at 07:38:05PM +0300, Nir Soffer wrote:
> > > We are discussing importing VM images to KubVirt. The goal is to be
> > > able to import an existing qcow2 disk, probably some appliance stored
> > > on http server, and and convert it to raw format for writing to storage.
> > >
> > > This can be also useful for for oVirt for importing OVA, since we like to
> > > pack
> > > disks in qcow2 format inside OVA, but the user may like to use raw disks, or
> > > for uploading existing disks.
> > >
> > > Of course converting the image using qemu-img is easy, but requires
> > > downloading the image to temporary disk. We would like to avoid temporary
> > > disks, or telling users to convert the image.
> > >
> > > Base on the discussion we had here:
> > > https://lists.ovirt.org/archives/list/address@hidden/thread/GNAVJ253FP65QUSOONES5XZGRIDX5ABC/#YMLSEGU7PN3MX5MUORGEGGAQLLSL4KKJ
> > >
> > > I think this is impossible since qcow2 is not built for streaming. But both
> > > Richard and Eric suggested some solutions.
> > >
> > > The flow is:
> > >
> > > qcow2 image -- http --> importer -> raw file
> > >
> > > Is it possible to implement the importer using qemu-img and qemu-nbd,
> > > or maybe nbdkit?
> >
> > Strictly speaking streaming qcow2 to raw is not possible. However
> > placing an overlay on top of the original remote image will allow
> > streaming to raw with only a modest amount of local storage consumed.
> >
> > You can demonstrate this fairly easily:
> >
> > $ qemu-img create -f qcow2 -b 'json: { "file.driver": "https", "file.url": "https://uk-mirrors.evowise.com/fedora/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2", "file.timeout": 10000 }' /var/tmp/overlay.qcow2
> > $ qemu-img convert -p -f qcow2 -O raw overlay.qcow2 fedora.img
>
> This overlay stays empty, so it's pretty pointless and you could just
> directly point 'qemu-img convert' to https and the real image.
Right, indeed. I was copying what virt-v2v does without thinking
about it enough. virt-v2v needs the overlay because it actually wants
to write into it, and it does copy-on-read for the first phase (not
the final ‘qemu-img convert’).
Thanks, I just tested the simple:
And it just works :-)
I got timeouts trying to download from
I guess we need to use
'json: { "file.driver": "http", "file.url": "url...", "file.timeout": 10000 }'
To change timeout? Where is these and other options documented?
I did also some timings, using sever on local network with 1g nic.
...
Length: 638043136 (608M) [application/octet-stream]Saving to: ‘Fedora-AtomicHost-28-20180625.1.x86_64.qcow2’
Fedora-AtomicHost-28-20180625.1.x86_64.qcow2 100%[=====================================================================================================>] 608.49M 107MB/s in 5.6s
2018-07-09 21:38:39 (108 MB/s) - ‘Fedora-AtomicHost-28-20180625.1.x86_64.qcow2’ saved [638043136/638043136]
real 0m5.941s
user 0m0.183s
sys 0m1.185s
(100.00/100%)
real 0m14.217s
user 0m5.235s
sys 0m2.343s
$ time qemu-img convert -p -f qcow2 -O raw Fedora-AtomicHost-28-20180625.1.x86_64.qcow2 converted.raw
(100.00/100%)
real 0m11.909s
user 0m4.728s
sys 0m1.595s
So converting on the fly is even little faster then downloading to temporary file and converting.
Nir