Re: What shall the filter do to bottommost translators

[olafBuddenhagen]

Hi,

On Mon, Feb 09, 2009 at 09:46:30PM +0200, Sergiu Ivanov wrote:
> On Thu, Jan 29, 2009 at 9:19 AM,  <olafBuddenhagen@gmx.net> wrote:
> > On Tue, Jan 13, 2009 at 10:55:05PM +0200, Sergiu Ivanov wrote:
> >> On Fri, Jan 9, 2009 at 9:01 AM, <olafBuddenhagen@gmx.net> wrote:

> > For each translator in the original tree, we get one nsmux instance.
> 
> Hm, really... I somehow thought that there should be one instance per
> each dynamic translator. And then I thought about doing something like
> ls -R /usr,,x/ which might launch quite a lot of translators.

You are right; I hadn't thought of that: there would be one
(distributed) nsmux instance for each dynamic translator too, not only
for each static one. And when doing any kind of recursive invocation, it
can become quite a lot.

Yet, in this situation we already have a lot of translators in the first
place. Doubling this number is not exactly neglectible, but it won't
make the difference between a situation that is OK and one that is not
acceptable...

As I already pointed out, sooner or later we will problably need some
framework to conflate simimlar translator instances in a single process
-- while dynamic translators are likely to make the problem evident in
many cases, the problem itself is not specific to them...

> I've suddenly realized that, although I remember you saying that
> shadow nodes should be thought as separate translators, I do not fully
> understand the reason for this. Why won't we think of shadow nodes as
> of separate filesystem nodes, bearing a translator?..

Who says we don't?...

Shadow nodes are a special kind of proxy nodes. (And are meant to bear a
dynamic translator.) As such, they are separate nodes, representing an
alternative view on the same filesystem location.

At some point I suggested implementing them as distinct translators,
because they behave differently than the normal nsmux proxy nodes, and I
thought it could simplify nsmux to factor them out -- but then I
realized that nsmux would still need special handling for them, so not
much gained here. However, I believe that it's still useful to think of
them as separate translators conceptually, because the interaction
between shadow nodes and other nodes becomes more explicit that way.

> >> > And it is actually not entirely correct: Let's say we use a
> >> > filter to skip "b" (and the rest of the stack on top of it). The
> >> > filter would traverse the stack through all this shadow
> >> > machinery, and would end up with a port to the root node of "a"
> >> > that is still proxied by the various shadow nodes. The filter
> >> > should return a "clean" node, without any shadowing on top of it;
> >> > but the shadow nodes don't know when the filter is done and
> >> > passes the result to the client, so they can't put themselfs out
> >> > of the loop...
[...]
> The only problem I can see now (which might, however, prove
> critical...) is that applications that do some exotic port
> manipulations (like filters) should be designed specifically to comply
> with nsmux standards. Could you please point out other issues?

Indeed, the problem is that the dynamic translators are not entirely
transparent -- some things that normally work, could break, unless
specially handling dynamic translators.

What other issues do you need?...

> > Unfortunately I realized now that traversing top to bottom only
> > partially solves this problem; we will have to address it
> > explicitely after all... [sigh]
> 
> Could we possibly define a new RPC (a bit frightening for me,
> unfourtunately) to make the shadow nodes yield the real port? That is,
> when the client should invoke this RPC on the port it obtains from the
> filter, they should get the port directly, not involving shadow nodes?

That is actually the solution I was considering. I don't really like it,
because this way nsmux is not transparent to the filter: the filter
needs to be aware that it is run by nsmux, and handle it specially. I
can't think of any other approach, though.

(At least this way nsmux/the shadow nodes would be transparent again
once the filter is done.)

> >> However, how do you suggest the top-to-bottom traversal should take
> >> place (in terms of RPCs)? Shall we reuse some existing RPCs?..
> >
> > Not sure. Any suggestions? :-)
> 
> I am somehow inclined to be against reusing the existing RPCs... It
> looks a bit evil, especially if we would like some exotic use-cases to
> be feasible (like setting nsmux, descending into the mirrored tree and
> launching another instance of nsmux on top of a mirrored node or even
> in a dynamic translator stack).

Indeed it is "a bit evil" -- though not really different than what you
did previously, with the kludge for obtaining the untranslated version
of the underlying node...

> I'd probably stand for introducing and additional RPC, although it
> would require playing with MiG, which I've tried to avoid so far...

No need to fear it, writing new RPCs is almost trivial. (I did it
recently for my KGI port.)

My major concern here is deciding whether to add the new RPC(s) to the
existing interfaces, or create a completely new one...

> Still, I cannot really discern the solution to the problem: the most
> natural way to traverse a translator stack is bottom-to-top (since the
> standard RPC provide the technique). As we have said not a single time
> before, a top-to-bottom traversal cannot be implemented but with some
> additional facilities (like proxy nodes, as far as I can understand).

There is really no difference here -- we need special facilities in
either case. When traversing bottom-to-top, we need a way to obtain the
completely untranslated node; when traversing top-to-bottom, we need a
way to obtain the underlying node, i.e. one translation layer lower.

> Now, what are the additional facilities for static translators? Is it
> right that nsmux will have, for instance, to traverse the static
> translator stack bottom-to-top to create a list which could then be
> traversed in a kind of top-to-bottom fashion?.. I feel I'm missing
> something fundamental...

I'm sharing this feeling... ;-)

Remember that to set a filter on top of the static stack, the client
needs to look up the (statically) translated node first. Looking up a
node with a translator stack means following retries at each level of
the stack, until you arrive at the top. nsmux creates proxy nodes at
each level while forwarding the retries. Each proxy node references the
one below it. That's all the list you need...

> > It's so much simpler to make a drawing in real time, and explain
> > while going along... The great advantage of a face-to-face meeting.
> 
> That's right... I wish there were no borders and people could download
> open-source vehicles and build them at home ;-) Face-to-face meetings
> would take place more often... :-)

Well, I'm actually ambiguous about physical meetings with people coming
from all over the world. They are very useful, but also a serious burden
on the environment...

> > You must be aware that we need both the shadows and the normal
> > proxies as explicit nodes, as ports for both are handed out to
> > external processes. (The normal proxies are handed out to clients
> > doing a lookup, and the shadows are handed to the dynamic
> > translators attached to them.)
[...]
> So, it's a matter of keeping things clear and complying with the
> concept? I have nothing against it, I was just interested in whether
> inclusion of proxy nodes in the internal structure of a dynamic
> translator stack was a vital thing or a conceptual thing.

I still don't understand what you are getting at. If we need the nodes
for things to work, it's pretty vital to have them, isn't it?...

What are you trying to optimize? You can't leave out the nodes. You
*could* duplicate the proxy data in the shadow nodes, so you don't have
to follow the reference from shadow node to proxy node in some cases --
but what's the point? Save following one reference?...

> > There must be some configuration option to disable the HTML.
[...]
> I found some extra button and I hope it will solve the problem...

It helped -- no HTML baggage this time :-)

> I've never had such problems before, though :-( That's why I thought
> it was the browser...

More likely you simply didn't know of the HTML baggage before... I only
realized it when the mails were getting absurdly large.

-antrik-