Re: [Qemu-devel] [RFC PATCH v2 00/10] Add colo-proxy based on netfilter

[Jason Wang]

On 12/31/2015 04:40 PM, Zhang Chen wrote:
>
>
> On 12/31/2015 10:36 AM, Jason Wang wrote:
>>
>> On 12/22/2015 06:42 PM, Zhang Chen wrote:
>>> From: zhangchen <address@hidden>
>>>
>>> Hi,all
>>>
>>> This patch add an colo-proxy object, COLO-Proxy is a part of COLO,
>>> based on qemu netfilter and it's a plugin for qemu netfilter. the
>>> function
>>> keep Secondary VM connect normal to Primary VM and compare packets
>>> sent by PVM to sent by SVM.if the packet difference,notify COLO do
>>> checkpoint and send all primary packet has queued.
>> Thanks for the work. I don't object this method but still not convinced
>> that qemu is the best place for this.
>>
>> As been raised in the past discussion, it's almost impossible to
>> cooperate with vhost backends. If we want this to be used in production
>> environment, need to think of a solution for vhost. There's no such
>> worry if we decouple this from qemu.
>>
>>> You can also get the series from:
>>>
>>> https://github.com/zhangckid/qemu/tree/colo-v2.2-periodic-mode-with-colo-proxyV2
>>>
>>>
>>> Usage:
>>>
>>> primary:
>>> -netdev tap,id=bn0 -device e1000,netdev=bn0
>>> -object
>>> colo-proxy,id=f0,netdev=bn0,queue=all,mode=primary,addr=host:port
>>>
>>> secondary:
>>> -netdev tap,id=bn0 -device e1000,netdev=bn0
>>> -object
>>> colo-proxy,id=f0,netdev=bn0,queue=all,mode=secondary,addr=host:port
>> Have a quick glance at how secondary mode work. What it does is just
>> forwarding packets between a nic and a socket, qemu socket backend did
>> exact the same job. You could even use socket in primary node and let
>> packet compare module talk to both primary and secondary node.
>
> If we use qemu socket backend , the same netdev will used by qemu
> socket and
> qemu netfilter. this will against qemu net design. and then, when colo
> do failover,
> secondary do not have backend to use. that's the real problem.

Then, maybe it's time to implement changing the netdev of a nic. The
point here is that what secondary mode did is in fact a netdev backend
instead of a filter ...

>
>
> Thanks
> zhangchen
>
>>> NOTE:
>>> queue must set "all". See enum NetFilterDirection for detail.
>>> colo-proxy need queue all packets
>>> colo-proxy V2 just can compare ip packet
>>>
>>>
>>> ## Background
>>>
>>> COLO FT/HA (COarse-grain LOck-stepping Virtual Machines for Non-stop
>>> Service)
>>> project is a high availability solution. Both Primary VM (PVM) and
>>> Secondary VM
>>> (SVM) run in parallel. They receive the same request from client,
>>> and generate
>>> responses in parallel too. If the response packets from PVM and SVM are
>>> identical, they are released immediately. Otherwise, a VM checkpoint
>>> (on
>>> demand)is conducted.
>>>
>>> Paper:
>>> http://www.socc2013.org/home/program/a3-dong.pdf?attredirects=0
>>>
>>> COLO on Xen:
>>> http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping
>>>
>>> COLO on Qemu/KVM:
>>> http://wiki.qemu.org/Features/COLO
>>>
>>> By the needs of capturing response packets from PVM and SVM and
>>> finding out
>>> whether they are identical, we introduce a new module to qemu
>>> networking
>>> called colo-proxy.
>>>
>>> V2:
>>>    rebase colo-proxy with qemu-colo-v2.2-periodic-mode
>>>    fix dave's comments
>>>    fix wency's comments
>>>    fix zhanghailiang's comments
>>>
>>> v1:
>>>    initial patch.
>>>
>>>
>>>
>>> zhangchen (10):
>>>    Init colo-proxy object based on netfilter
>>>    Jhash: add linux kernel jhashtable in qemu
>>>    Colo-proxy: add colo-proxy framework
>>>    Colo-proxy: add data structure and jhash func
>>>    net/colo-proxy: Add colo interface to use proxy
>>>    net/colo-proxy: add socket used by forward func
>>>    net/colo-proxy: Add packet enqueue & handle func
>>>    net/colo-proxy: Handle packet and connection
>>>    net/colo-proxy: Compare pri pkt to sec pkt
>>>    net/colo-proxy: Colo-proxy do checkpoint and clear
>>>
>>>   include/qemu/jhash.h |  61 ++++
>>>   net/Makefile.objs    |   1 +
>>>   net/colo-proxy.c     | 939
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++
>>>   net/colo-proxy.h     |  24 ++
>>>   qemu-options.hx      |   6 +
>>>   trace-events         |   8 +
>>>   vl.c                 |   3 +-
>>>   7 files changed, 1041 insertions(+), 1 deletion(-)
>>>   create mode 100644 include/qemu/jhash.h
>>>   create mode 100644 net/colo-proxy.c
>>>   create mode 100644 net/colo-proxy.h
>>>
>>
>>
>> .
>>
>