Re: [PATCH V8 1/6] qapi/net: Add IPFlowSpec and QMP command for COLO passthrough

[Markus Armbruster]

"Zhang, Chen" <chen.zhang@intel.com> writes:

>> -----Original Message-----
>> From: Markus Armbruster <armbru@redhat.com>
>> Sent: Wednesday, June 16, 2021 2:04 PM
>> To: Zhang, Chen <chen.zhang@intel.com>
>> Cc: Lukas Straub <lukasstraub2@web.de>; Daniel P.Berrangé
>> <berrange@redhat.com>; Li Zhijian <lizhijian@cn.fujitsu.com>; Jason Wang
>> <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>; Dr. David
>> Alan Gilbert <dgilbert@redhat.com>; Gerd Hoffmann <kraxel@redhat.com>;
>> Zhang Chen <zhangckid@gmail.com>; Eric Blake <eblake@redhat.com>
>> Subject: Re: [PATCH V8 1/6] qapi/net: Add IPFlowSpec and QMP command
>> for COLO passthrough
>> 
>> "Zhang, Chen" <chen.zhang@intel.com> writes:
>> 
>> >> -----Original Message-----
>> >> From: Markus Armbruster <armbru@redhat.com>
>> >> Sent: Tuesday, June 15, 2021 10:43 PM
>> >> To: Zhang, Chen <chen.zhang@intel.com>
>> >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
>> >> devel@nongnu.org>; Eric Blake <eblake@redhat.com>; Dr. David Alan
>> >> Gilbert <dgilbert@redhat.com>; Daniel P.Berrangé
>> >> <berrange@redhat.com>; Gerd Hoffmann <kraxel@redhat.com>; Li
>> Zhijian
>> >> <lizhijian@cn.fujitsu.com>; Lukas Straub <lukasstraub2@web.de>; Zhang
>> >> Chen <zhangckid@gmail.com>
>> >> Subject: Re: [PATCH V8 1/6] qapi/net: Add IPFlowSpec and QMP
>> command
>> >> for COLO passthrough
>> >>
>> >> Zhang Chen <chen.zhang@intel.com> writes:
>> >>
>> >> > Since the real user scenario does not need COLO to monitor all traffic.
>> >> > Add colo-passthrough-add and colo-passthrough-del to maintain a
>> >> > COLO network passthrough list. Add IPFlowSpec struct for all QMP 
>> >> > commands.
>> >> > All the fields of IPFlowSpec are optional.
>> >> >
>> >> > Signed-off-by: Zhang Chen <chen.zhang@intel.com>
>> >> > ---
>> >>
>> >> The QAPI schema looks good to me, but the interface documentation is
>> >> still not quite clear enough.  To make progress, I'm going to make
>> >> concrete suggestions wherever I can despite being quite clueless
>> >> about the subject matter.  Risks me writing something that's clearer,
>> >> but wrong.  Keep that in mind, please.
>> >>
>> >> >  net/net.c     | 10 +++++++
>> >> >  qapi/net.json | 74
>> >> > +++++++++++++++++++++++++++++++++++++++++++++++++++
>> >> >  2 files changed, 84 insertions(+)
>> >> >
>> >> > diff --git a/net/net.c b/net/net.c
>> >> > index 76bbb7c31b..f913e97983 100644
>> >> > --- a/net/net.c
>> >> > +++ b/net/net.c
>> >> > @@ -1195,6 +1195,16 @@ void qmp_netdev_del(const char *id, Error **errp)
>> >> >      }
>> >> >  }
>> >> >
>> >> > +void qmp_colo_passthrough_add(IPFlowSpec *spec, Error **errp) {
>> >> > +    /* TODO implement setup passthrough rule */ }
>> >> > +
>> >> > +void qmp_colo_passthrough_del(IPFlowSpec *spec, Error **errp) {
>> >> > +    /* TODO implement delete passthrough rule */ }
>> >> > +
>> >> >  static void netfilter_print_info(Monitor *mon, NetFilterState *nf)  {
>> >> >      char *str;
>> >> > diff --git a/qapi/net.json b/qapi/net.json index
>> >> > 7fab2e7cd8..91f2e1495a 100644
>> >> > --- a/qapi/net.json
>> >> > +++ b/qapi/net.json
>> >> > @@ -7,6 +7,7 @@
>> >> >  ##
>> >> >
>> >> >  { 'include': 'common.json' }
>> >> > +{ 'include': 'sockets.json' }
>> >> >
>> >> >  ##
>> >> >  # @set_link:
>> >> > @@ -696,3 +697,76 @@
>> >> >  ##
>> >> >  { 'event': 'FAILOVER_NEGOTIATED',
>> >> >    'data': {'device-id': 'str'} }
>> >> > +
>> >> > +##
>> >> > +# @IPFlowSpec:
>> >> > +#
>> >> > +# IP flow specification.
>> >> > +#
>> >> > +# @protocol: Transport layer protocol like TCP/UDP, etc. The protocol 
>> >> > is the
>> >> > +#            string instead of enum, because it can be passed to 
>> >> > getprotobyname(3)
>> >> > +#            and avoid duplication with /etc/protocols.
>> >>
>> >> The rationale is good, but it doesn't really belong into the
>> >> interface documentation.  Suggest:
>> >>
>> >>    # @protocol: Transport layer protocol like TCP/UDP, etc.  This will be
>> >>    #            passed to getprotobyname(3).
>> >>
>> >
>> > OK.
>> >
>> >>
>> >> > +#
>> >> > +# @object-name: The @object-name means packet handler in Qemu. Because 
>> >> > not
>> >> > +#               all the network packet must pass the colo-compare 
>> >> > module,
>> >> > +#               the net-filters are same situation. There modules 
>> >> > attach to
>> >> > +#               netdev or chardev to work, VM can run multiple modules
>> >> > +#               at the same time. So it needs the object-name to set
>> >> > +#               the effective module.
>> >>
>> >> I still don't understand this, and I'm too ignorant of COLO and
>> >> networking to suggest improvements.
>> >
>> > Let me use qemu boot parameter to clear it.
>> > For colo-compare, it needs chardev as the source to handle network packet.
>> > -object 
>> > colo-compare,id=comp0,primary_in=chardev-input0,secondary_in=chardev-input1,outdev=chardev-output0,iothread=iothread0.
>> >
>> > For net filters, it needs attached on netdev.
>> > -object 
>> > filter-redirector,id=red0,netdev=hn0,queue=rx,outdev=chardev-output1
>> > -object filter-mirror,id=mirror0,netdev=hn0,queue=rx,outdev=chardev-output2
>> >
>> > And we can use -chardev socket combine the filter and the colo-compare.
>> >
>> > Back to the @object-name, One guest maybe have multi colo-compare as the 
>> > same time, with different object name from different source.
>> > So we need assign the IPFlowSpec to one object as the handler. Same as the 
>> > net-filters.
>> > Each object instance has its own passthrough list.
>> 
>> So the @object-name here references one of the "packet handler objects"
>> (colo-compare, filter-redirector, filter-mirror) by @id.  Correct?
>
> Yes.
>
>> 
>> In other words, @object-name is the ID of a QOM object, and the QOM
>> object must be of a certain kind (i.e. provide certain functionality).
>> Correct?
>
> Yes.

Got it.

>> What exactly makes a QOM object a "packet handler object?"
>> 
>
> Firstly, the original object need have basic network packet input/output 
> capability.
> It's a good question, maybe we need add a flag in general object structure to 
> show the capability.

A QOM interface might fit the bill: a QOM type is a packet handler if
and only if it implements the packet handler interface.
 
>> Right now, the packet handler object types are colo-compare, filter-
>> redirector, filter-mirror, and that's all.  Correct?
>
> No, this series just make colo-compare become a packet handler, This is a 
> beginning, I plan to make other filters support it.

Okay.

Are these other filters similarly related to COLO?  I'm asking because
the commands are called colo-passthrough-FOO.  If this goes beyond COLO,
we may want to name them differently.

>> Another question the doc comment needs to answer: what happens when
>> @object-name is absent?
>
> Please see the explanation below.

You seem to consider making it mandatory there.  My question would be
moot then.

>> >> Jason or David, perhaps?
>> >>
>> >> > +#
>> >> > +# @source: Source address and port.
>> >> > +#
>> >> > +# @destination: Destination address and port.
>> >> > +#
>> >> > +# Since: 6.1
>> >> > +##
>> >> > +{ 'struct': 'IPFlowSpec',
>> >> > +  'data': { '*protocol': 'str', '*object-name': 'str',
>> >> > +    '*source': 'InetSocketAddressBase',
>> >> > +    '*destination': 'InetSocketAddressBase' } }
>> >> > +
>> >> > +##
>> >> > +# @colo-passthrough-add:
>> >> > +#
>> >> > +# Add passthrough entry IPFlowSpec to the COLO-compare instance.
>> >> > +# The protocol and source/destination IP/ports are optional. if
>> >> > +the user # only inputs part of the information, this will match all 
>> >> > traffic.
>> >>
>> >> Actually, all arguments are optional.
>> >>
>> >> Suggest:
>> >>
>> >>    # Add an entry to the COLO network passthrough list.
>> >>    # Absent protocol, host addresses and ports match anything.
>> >>
>> >> If there is more than one such list, then "to a COLO network passthrough 
>> >> list"
>> >> instead.
>> >
>> > Yes, more than one list.
>> >
>> >>
>> >> Still missing then: meaning of absent @object-name.  Does it select
>> >> the COLO network passthrough list, perhaps?
>> >
>> > Yes, Please see the explanation above. Each object instance has its own 
>> > passthrough list.
>> 
>> Got it now.
>> 
>> >> > +#
>> >> > +# Returns: Nothing on success
>> >> > +#
>> >> > +# Since: 6.1
>> >> > +#
>> >> > +# Example:
>> >> > +#
>> >> > +# -> { "execute": "colo-passthrough-add",
>> >> > +#      "arguments": { "protocol": "tcp", "object-name": "object0",
>> >> > +#      "source": {"host": "192.168.1.1", "port": "1234"},
>> >> > +#      "destination": {"host": "192.168.1.2", "port": "4321"} } }
>> >> > +# <- { "return": {} }
>> >> > +#
>> >> > +##
>> >> > +{ 'command': 'colo-passthrough-add', 'boxed': true,
>> >> > +     'data': 'IPFlowSpec' }
>> >> > +
>> >> > +##
>> >> > +# @colo-passthrough-del:
>> >> > +#
>> >> > +# Delete passthrough entry IPFlowSpec to the COLO-compare instance.
>> >> > +# The protocol and source/destination IP/ports are optional. if
>> >> > +the user # only inputs part of the information, this will match all 
>> >> > traffic.
>> >>
>> >> I suspect this command doesn't actually match traffic, it matches
>> >> entries added with colo-passthrough-add.
>> >
>> > Yes.
>> >
>> >>
>> >> Can it delete more than one such entry?
>> >>
>> >
>> > Currently no, but it easy to match one more entry to delete.
>> 
>> If the passthrough list entries had some unique ID, we could refer to one
>> entry by its ID.  It's how things commonly work.
>> 
>> Without an ID, we need to match by value, like you do.  I can see three
>> possible behaviors:
>> 
>> 1. Select first entry that matches.
>> 
>> 2. Select all entries that match.
>> 
>> 3. If exactly one entry matches, select it.
>> 
>> The second design choice is behavior when nothing gets selected:
>> 
>> a. Silently do nothing
>> 
>> b. Error
>> 
>> Which one did you implement?  My guess based on your answers is 1a.
>
> Re-think about it,  If we want to match by value, we need know which object 
> have the capability and search in each object passthrough list.
> Obviously, we haven't such flag in object structure. So It more reasonable to 
> make @object-name as a must at the beginning.
> Because the passthrough list always in the network handler object. Maybe we 
> need a global passthrough list for each guest to handle it in the future.
> It will have two-level passthrough list to control network.

I'm not sure I understand.

If you make @object-name mandatory both for colo-passthrough-add and
colo-passthrough-del, then we can simply use @object-name to find the
object, check it implements the packet handler interface, use the packet
handler interface to get its passthrough list, then add to / delete from
that list.

If we find a use for making @object-name optional later, we can do so
without breaking compatibility.