qemu-devel
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH V12 02/10] colo-compare: introduce colo compare

From: Zhang Chen
Subject: Re: [Qemu-devel] [PATCH V12 02/10] colo-compare: introduce colo compare initialization
Date: Wed, 31 Aug 2016 17:39:35 +0800
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 


On 08/31/2016 05:20 PM, Jason Wang wrote:



On 2016年08月31日 17:03, Zhang Chen wrote:



On 08/31/2016 03:53 PM, Jason Wang wrote:



On 2016年08月17日 16:10, Zhang Chen wrote:

This a COLO net ascii figure:

  Primary qemu Secondary qemu
+--------------------------------------------------------------+ +----------------------------------------------------------------+ | +----------------------------------------------------------+ | | +-----------------------------------------------------------+ | 
| | | |       | | | |
| | guest | | | | guest | | 
| | | |       | | | |
| +-------^--------------------------+-----------------------+ | | +---------------------+--------+----------------------------+ | | | | | | ^ | | | | | | | | | | | | +------------------------------------------------------+ | | | | |netfilter| | | | | | netfilter | | | | +----------+ +----------------------------+ | | | +-----------------------------------------------------------+ | | | | | | | out | | | | | | filter excute order | | | | | | +-----------------------------+ | | | | | | +-------------------> | | | | | | | | | | | | | | | | TCP | | | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| | | | +------------+ +---+----+---v+rewriter++ +------------+ | | | | | | | | | | |in | |in | | | | | | | | | | | | | | | | filter | | filter | | filter +------> colo <------+ +--------> filter +--> adjust | adjust +--> filter | | | | | | mirror | |redirector| |redirector| | | compare | | | | | | redirector | | ack | seq | | redirector | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +----^-----+ +----+-----+ +----------+ | +---------+ | | | | +------------+ +--------+--------------+ +---+--------+ | | | | | tx | rx rx | | | | | tx all | rx | | | | | | | | | | +-----------------------------------------------------------+ | 
| |      |             +--------------+     | |  |    | |            |
| |      |   filter excute order      |     | |  |    | |            |
| | | +----------------> | | | +--------------------------------------------------------+ | 
| +-----------------------------------------+ | | |
|        |                            | | | |
+--------------------------------------------------------------+ +----------------------------------------------------------------+ 
          |guest receive               | guest send
          |                            |
+--------+----------------------------v------------------------+
| |                          NOTE: filter direction is rx/tx/all
|                         tap | rx:receive packets sent to the netdev
| |                          tx:receive packets sent by the netdev
+--------------------------------------------------------------+
It's better to add a doc under docs to explain this configuration in detail on top of this series.
As you say, Am I add /docs/colo-proxy.txt to explain it or add this in hailiang's COLO-FT.txt after merge? 

Can you give me a way for doc?





In COLO-compare, we do packet comparing job.
Packets coming from the primary char indev will be sent to outdev.
Packets coming from the secondary char dev will be dropped after comparing. 
colo-comapre need two input chardev and one output chardev:
primary_in=chardev1-id (source: primary send packet)
secondary_in=chardev2-id (source: secondary send packet)
outdev=chardev3-id

usage:

primary:
-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown 
-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 

secondary:
-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown 
-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
-chardev socket,id=red0,host=3.3.3.3,port=9003
-chardev socket,id=red1,host=3.3.3.3,port=9004
-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1

Signed-off-by: Zhang Chen <address@hidden>
Signed-off-by: Li Zhijian <address@hidden>
Signed-off-by: Wen Congyang <address@hidden>
---
  net/Makefile.objs  |   1 +
net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 
  qemu-options.hx    |  39 ++++++++
  vl.c               |   3 +-
  4 files changed, 326 insertions(+), 1 deletion(-)
  create mode 100644 net/colo-compare.c

diff --git a/net/Makefile.objs b/net/Makefile.objs
index b7c22fd..ba92f73 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
  common-obj-y += filter.o
  common-obj-y += filter-buffer.o
  common-obj-y += filter-mirror.o
+common-obj-y += colo-compare.o
diff --git a/net/colo-compare.c b/net/colo-compare.c
new file mode 100644
index 0000000..cdc3e0e
--- /dev/null
+++ b/net/colo-compare.c
@@ -0,0 +1,284 @@
+/*
+ * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) 
+ * (a.k.a. Fault Tolerance or Continuous Replication)
+ *
+ * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ * Copyright (c) 2016 FUJITSU LIMITED
+ * Copyright (c) 2016 Intel Corporation
+ *
+ * Author: Zhang Chen <address@hidden>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qemu-common.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/error.h"
+#include "net/net.h"
+#include "net/vhost_net.h"


Looks unnecessary.


I will remove it.




+#include "qom/object_interfaces.h"
+#include "qemu/iov.h"
+#include "qom/object.h"
+#include "qemu/typedefs.h"
+#include "net/queue.h"
+#include "sysemu/char.h"
+#include "qemu/sockets.h"
+#include "qapi-visit.h"
+
+#define TYPE_COLO_COMPARE "colo-compare"
+#define COLO_COMPARE(obj) \
+    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
+
+#define COMPARE_READ_LEN_MAX NET_BUFSIZE
+
+typedef struct CompareState {
+    Object parent;
+
+    char *pri_indev;
+    char *sec_indev;
+    char *outdev;
+    CharDriverState *chr_pri_in;
+    CharDriverState *chr_sec_in;
+    CharDriverState *chr_out;
+    QTAILQ_ENTRY(CompareState) next;
This looks not used in this series but in commit "colo-compare and filter-rewriter work with colo-frame". We'd better delay the introducing to that patch. 

OK~ I got your point.




+    SocketReadState pri_rs;
+    SocketReadState sec_rs;
+} CompareState;
+
+typedef struct CompareClass {
+    ObjectClass parent_class;
+} CompareClass;
+
+typedef struct CompareChardevProps {
+    bool is_socket;
+    bool is_unix;
+} CompareChardevProps;
+
+static char *compare_get_pri_indev(Object *obj, Error **errp)
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    return g_strdup(s->pri_indev);
+}
+
+static void compare_set_pri_indev(Object *obj, const char *value, Error **errp) 
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    g_free(s->pri_indev);
+    s->pri_indev = g_strdup(value);
+}
+
+static char *compare_get_sec_indev(Object *obj, Error **errp)
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    return g_strdup(s->sec_indev);
+}
+
+static void compare_set_sec_indev(Object *obj, const char *value, Error **errp) 
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    g_free(s->sec_indev);
+    s->sec_indev = g_strdup(value);
+}
+
+static char *compare_get_outdev(Object *obj, Error **errp)
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    return g_strdup(s->outdev);
+}
+
+static void compare_set_outdev(Object *obj, const char *value, Error **errp) 
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    g_free(s->outdev);
+    s->outdev = g_strdup(value);
+}
+
+static void compare_pri_rs_finalize(SocketReadState *pri_rs)
+{
+ /* if packet_enqueue pri pkt failed we will send unsupported packet */ 
+}
+
+static void compare_sec_rs_finalize(SocketReadState *sec_rs)
+{
+    /* if packet_enqueue sec pkt failed we will notify trace */
+}
+
+static int compare_chardev_opts(void *opaque,
+                                const char *name, const char *value,
+                                Error **errp)
+{
+    CompareChardevProps *props = opaque;
+
+ if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) { 
+        props->is_socket = true;
+    } else if (strcmp(name, "host") == 0) {


Typo? net_vhost_chardev_opts() did:

    } else if (strcmp(name, "path") == 0) {
        props->is_unix = true;
    }




No, In colo-compare we use chardev like this:

-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait

If we only use "path" here will trigger a error.
Should I add anthor "path" here?
If I understand the code correctly, "is_unix" means "is unix domain socket"? If yes, according to the help:
-chardev socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds] 
[,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off]
         [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp)
-chardev socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds] 
         [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix)

"host" will not be used for UNIX domain socket.
And if UNIX domain socket is not supported, there's probably no need to differentiate it from other types. 

OK, I will remove the "is_unix" in next version.










+        props->is_unix = true;
+    } else if (strcmp(name, "port") == 0) {
+    } else if (strcmp(name, "server") == 0) {
+    } else if (strcmp(name, "wait") == 0) {
+    } else {
+        error_setg(errp,
+ "COLO-compare does not support a chardev with option %s=%s", 
+                   name, value);
+        return -1;
+    }
+    return 0;
+}
+
+/*
+ * called from the main thread on the primary
+ * to setup colo-compare.
+ */
+static void colo_compare_complete(UserCreatable *uc, Error **errp)
+{
+    CompareState *s = COLO_COMPARE(uc);
+    CompareChardevProps props;
+
+    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
+        error_setg(errp, "colo compare needs 'primary_in' ,"
+                   "'secondary_in','outdev' property set");
+        return;
+    } else if (!strcmp(s->pri_indev, s->outdev) ||
+               !strcmp(s->sec_indev, s->outdev) ||
+               !strcmp(s->pri_indev, s->sec_indev)) {
+        error_setg(errp, "'indev' and 'outdev' could not be same "
+                   "for compare module");
+        return;
+    }
+
+    s->chr_pri_in = qemu_chr_find(s->pri_indev);
+    if (s->chr_pri_in == NULL) {
+        error_setg(errp, "Primary IN Device '%s' not found",
+                   s->pri_indev);
+        return;
+    }
+
+    /* inspect chardev opts */
+    memset(&props, 0, sizeof(props));
+ if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, &props, errp)) { 
+        return;
+    }
+
+    if (!props.is_socket || !props.is_unix) {
+        error_setg(errp, "chardev \"%s\" is not a unix socket",
+                   s->pri_indev);
+        return;
+    }
+
+    s->chr_sec_in = qemu_chr_find(s->sec_indev);
+    if (s->chr_sec_in == NULL) {
+        error_setg(errp, "Secondary IN Device '%s' not found",
+                   s->sec_indev);
+        return;
+    }
+
+    memset(&props, 0, sizeof(props));
+ if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, &props, errp)) { 
+        return;
+    }
+
+    if (!props.is_socket || !props.is_unix) {
+        error_setg(errp, "chardev \"%s\" is not a unix socket",
+                   s->sec_indev);


I believe tcp socket is also supported?
If I understand correctly, "tcp socket" in here is the "-chardev socket". 
I will rename "unix socket" to "tcp socket".




+        return;
+    }
+
+    s->chr_out = qemu_chr_find(s->outdev);
+    if (s->chr_out == NULL) {
+        error_setg(errp, "OUT Device '%s' not found", s->outdev);
+        return;
+    }
+
+    memset(&props, 0, sizeof(props));
+ if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, &props, errp)) { 
+        return;
+    }
+
+    if (!props.is_socket || !props.is_unix) {
+        error_setg(errp, "chardev \"%s\" is not a unix socket",
+                   s->outdev);
Ditto, and there's code duplication, please introduce a helper to do above. 

I don't understand what the "helper"?
In here we check each chardev, will I change to "goto error;" ?
A helper to avoid the code duplication for socket type inspection for pri_in,scr_in and chr_out. 

I got it~~
I will add it in next version.









+        return;
+    }
+
+    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
+
+    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
+
+    qemu_chr_fe_claim_no_fail(s->chr_out);
+
+    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
+    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
+
+    return;
+}
+
+static void colo_compare_class_init(ObjectClass *oc, void *data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+
+    ucc->complete = colo_compare_complete;
+}
+
+static void colo_compare_init(Object *obj)
+{
+    object_property_add_str(obj, "primary_in",
+ compare_get_pri_indev, compare_set_pri_indev, 
+                            NULL);
+    object_property_add_str(obj, "secondary_in",
+ compare_get_sec_indev, compare_set_sec_indev, 
+                            NULL);
+    object_property_add_str(obj, "outdev",
+                            compare_get_outdev, compare_set_outdev,
+                            NULL);
+}
+
+static void colo_compare_finalize(Object *obj)
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    if (s->chr_pri_in) {
+        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL);
+        qemu_chr_fe_release(s->chr_pri_in);
+    }
+    if (s->chr_sec_in) {
+        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL);
+        qemu_chr_fe_release(s->chr_sec_in);
+    }
+    if (s->chr_out) {
+        qemu_chr_fe_release(s->chr_out);
+    }
+
+    g_free(s->pri_indev);
+    g_free(s->sec_indev);
+    g_free(s->outdev);
+}
+
+static const TypeInfo colo_compare_info = {
+    .name = TYPE_COLO_COMPARE,
+    .parent = TYPE_OBJECT,
+    .instance_size = sizeof(CompareState),
+    .instance_init = colo_compare_init,
+    .instance_finalize = colo_compare_finalize,
+    .class_size = sizeof(CompareClass),
+    .class_init = colo_compare_class_init,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};
+
+static void register_types(void)
+{
+    type_register_static(&colo_compare_info);
+}
+
+type_init(register_types);
diff --git a/qemu-options.hx b/qemu-options.hx
index 587de8f..33d5d0b 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} to the file specified by The file format is libpcap, so it can be analyzed with tools such as tcpdump 
  or Wireshark.
address@hidden -object colo-compare,address@hidden,address@hidden,address@hidden, 
address@hidden
+
+Colo-compare gets packet from address@hidden and address@hidden, than compare primary packet with 
+secondary packet. If the packet same, we will output primary


s/If the packet same/If the packets are same/.


OK.




+packet to address@hidden, else we will notify colo-frame
+do checkpoint and send primary packet to address@hidden
+
+we can use it with the help of filter-mirror and filter-redirector.
s/we/We/ and looks like colo compare must be used with the help of mirror and redirector? 

Currently yes.


Then please change the doc here.


s/We can use it/We must use it.

Thanks
Zhang Chen



Thanks






+
address@hidden
+
+primary:
+-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown 
+-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
+-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
+-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
+-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
+-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
+-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
+-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
+-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
+-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 
+
+secondary:
+-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown 
+-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
+-chardev socket,id=red0,host=3.3.3.3,port=9003
+-chardev socket,id=red1,host=3.3.3.3,port=9004
+-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
+-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
+
address@hidden example
+
+If you want to know the detail of above command line, you can read
+the colo-compare git log.
+
@item -object secret,address@hidden,address@hidden,address@hidden|base64}[,address@hidden,address@hidden @item -object secret,address@hidden,address@hidden,address@hidden|base64}[,address@hidden,address@hidden 
  diff --git a/vl.c b/vl.c
index cbe51ac..c6b9a6f 100644
--- a/vl.c
+++ b/vl.c
@@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type) 
      if (g_str_equal(type, "filter-buffer") ||
          g_str_equal(type, "filter-dump") ||
          g_str_equal(type, "filter-mirror") ||
-        g_str_equal(type, "filter-redirector")) {
+        g_str_equal(type, "filter-redirector") ||
+        g_str_equal(type, "colo-compare")) {
          return false;
      }




.







.



--
Thanks
zhangchen
reply via email to
[Prev in Thread] Current Thread [Next in Thread]