Re: [GNUnet-developers] Sending large data

[Christian Grothoff]

On 10/17/19 10:52 PM, Alessio Vanni wrote:
> Christian Grothoff <address@hidden> writes:
> 
>> Segmenting is the right answer. As for 'fast enough', there is usually
>> no requirement on how fast you call those APIs. However, for a CADET
>> channel, the receiver must call "GNUNET_CADET_receive_done" after each
>> message. Until you call that function, you won't get another one. So
>> the most likely problem is that you failed to call this function after
>> receiving the first segment.
> 
> So if I understand correctly, the client can send segments at any speed
> and those messages are left "in the air" until the service is ready?  

Yes, in the worst-case they are buffered in the _client_ processes'
memory. You can check the MQ API (length, send callback) for ways to
find out if your messages are piling up (or at least have left your
process), so you DO have a way to do proper flow control.

> I
> don't really know the internal of the MQ API and the manual doesn't
> really explain if there are some dark corners.  As for CADET, I didn't
> try yet but I assume it works the same way?

MQ semantics are always the same, be it the client or the CADET API.

>> Segmentation is the answer.
> 
> In my client, I have something like this (this example is simplified):
> 
> void
> send_message(void *data) {
>      /* Frobnicate data and segment it */
>       
>      for (i=0; i<number_of_segments; ++i) {
>           GNUNET_MQ_send(mq, segments[i]);
>      }
> 
>      /* Maybe do something else like cleanup */
> }
> 
> while my service handles it like this (again, simplified):
> 
> void
> handle_message(void *cls, struct MyMessage *msg) {
>      struct Client *c = cls;
> 
>      LOG("received segment %d\n", ntohl(msg->segment));
> 
>      /* Maybe do something else with msg */
> 
>      GNUNET_SERVICE_client_continue(c->client);
> }

That's fine, if number_of_segments isn't so large that you run into RAM
trouble ;-).

> If I send a single message whose `segment' field is 0, the service
> correctly prints "received segment 0".  However, if I send three
> messages with `segment' equal to 2, 1 and 0 respectively, the service
> will print only "received segment 2" and nothing else.  I really can't
> see what the problem could be.

Are you re-using the same envelope? Did you check for memory corruption?
Is this just between your service and your client? Where is CADET here?

signature.asc
Description: OpenPGP digital signature