Re: [Qemu-devel] [RFC PATCH v2 00/10] Add colo-proxy based on netfilter

[Dr. David Alan Gilbert]

* Jason Wang (address@hidden) wrote:
> 
> 
> On 01/05/2016 12:52 AM, Dr. David Alan Gilbert wrote:
> > * Jason Wang (address@hidden) wrote:
> >>
> >> On 01/04/2016 04:16 PM, Zhang Chen wrote:
> >>>
> >>> On 01/04/2016 01:37 PM, Jason Wang wrote:
> >>>> On 12/31/2015 04:40 PM, Zhang Chen wrote:
> >>>>> On 12/31/2015 10:36 AM, Jason Wang wrote:
> >>>>>> On 12/22/2015 06:42 PM, Zhang Chen wrote:
> >>>>>>> From: zhangchen <address@hidden>
> >>>>>>>
> >>>>>>> Hi,all
> >>>>>>>
> >>>>>>> This patch add an colo-proxy object, COLO-Proxy is a part of COLO,
> >>>>>>> based on qemu netfilter and it's a plugin for qemu netfilter. the
> >>>>>>> function
> >>>>>>> keep Secondary VM connect normal to Primary VM and compare packets
> >>>>>>> sent by PVM to sent by SVM.if the packet difference,notify COLO do
> >>>>>>> checkpoint and send all primary packet has queued.
> >>>>>> Thanks for the work. I don't object this method but still not
> >>>>>> convinced
> >>>>>> that qemu is the best place for this.
> >>>>>>
> >>>>>> As been raised in the past discussion, it's almost impossible to
> >>>>>> cooperate with vhost backends. If we want this to be used in
> >>>>>> production
> >>>>>> environment, need to think of a solution for vhost. There's no such
> >>>>>> worry if we decouple this from qemu.
> >>>>>>
> >>>>>>> You can also get the series from:
> >>>>>>>
> >>>>>>> https://github.com/zhangckid/qemu/tree/colo-v2.2-periodic-mode-with-colo-proxyV2
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Usage:
> >>>>>>>
> >>>>>>> primary:
> >>>>>>> -netdev tap,id=bn0 -device e1000,netdev=bn0
> >>>>>>> -object
> >>>>>>> colo-proxy,id=f0,netdev=bn0,queue=all,mode=primary,addr=host:port
> >>>>>>>
> >>>>>>> secondary:
> >>>>>>> -netdev tap,id=bn0 -device e1000,netdev=bn0
> >>>>>>> -object
> >>>>>>> colo-proxy,id=f0,netdev=bn0,queue=all,mode=secondary,addr=host:port
> >>>>>> Have a quick glance at how secondary mode work. What it does is just
> >>>>>> forwarding packets between a nic and a socket, qemu socket backend did
> >>>>>> exact the same job. You could even use socket in primary node and let
> >>>>>> packet compare module talk to both primary and secondary node.
> >>>>> If we use qemu socket backend , the same netdev will used by qemu
> >>>>> socket and
> >>>>> qemu netfilter. this will against qemu net design. and then, when colo
> >>>>> do failover,
> >>>>> secondary do not have backend to use. that's the real problem.
> >>>> Then, maybe it's time to implement changing the netdev of a nic. The
> >>>> point here is that what secondary mode did is in fact a netdev backend
> >>>> instead of a filter ...
> >>> Currently, you are right. in colo-proxy V2 code, I just compare IP
> >>> packet to
> >>> decide whether to do checkpoint.
> >>> But, in colo-proxy V3 I will compare tcp,icmp,udp packet to decide it.
> >>> because that can reduce frequency of checkpoint and improve
> >>> performance. To keep tcp connection well, colo secondary need to record
> >>> primary guest's init seq and adjust secondary guest's ack. if colo do
> >>> failover,
> >>> secondary also need do this to old tcp connection. qemu socket
> >>> can't do this job.
> >> So a question here: is it a must to do things (e.g TCP analysis stuffs)
> >> at secondary? Looks like we could do this at primary node. And I saw
> >> you're doing packet comparing in primary node, any advantages of doing
> >> this in primary instead of secondary?
> > It needs to do this on the secondary; the trick is that things like TCP 
> > sequence
> > numbers are likely to be different on the primary and secondary; the kernel 
> > colo-proxy
> > implementation solved this problem by rewriting the sequence numbers on
> > the secondary to match the primary, after a failover, the secondary has
> > to keep doing that rewrite to ensure existing connections are OK.
> > Thus it's holding some state about the current connections.
> 
> I see.
> 
> > I think also, to be able to do a 2nd failover (i.e. recover from the 1st 
> > failure
> > and then sometime later have another) you'd have to sync this
> > state over to a new host, so again that says the state needs to be part of
> > qemu or at least easily available to it.
> >
> > Dave
> 
> Right, if it does thing like tcp seq rewrite (which is missed in current
> version), it works much more like a netfilter. Wonder if the function is
> generic enough for users other than colo.

I can imagine the sequence number rework might be, but I doubt the packet
comparison is.

Dave

> Thanks
> 
> >
> >>> and another problem is do failover, if we use qemu socket
> >>> to be backend in secondary, when colo do failover, I don't know how to
> >>> change
> >>> secondary be a normal qemu, if you know, please tell me.
> >> Current qemu couldn't do this, but I mean we implement something like
> >> nic_change_backend which can change nic's peer(s). With this, in
> >> secondary, we can replace the socket backend with whatever you want (e.g
> >> tap or other).
> >>
> >> Thanks
> >>
> >>>
> >>> Thanks for your revew
> >>> zhangchen 
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK