[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: (n-for-each-par-map 16 store download (reverse (iota (max-id))))) cr

From: Amirouche Boubekki
Subject: Re: (n-for-each-par-map 16 store download (reverse (iota (max-id))))) crash
Date: Fri, 06 Jul 2018 07:55:09 +0200
User-agent: Roundcube Webmail/1.1.2

Hi Mark,

Thanks for your support and time.

On 2018-07-05 20:30, Mark H Weaver wrote:

Amirouche Boubekki <address@hidden> writes:

I have a program that try to download hackernews locally.

What it does is simple, it fetch the max identifier and
http-get each json value starting with the most recent
item. I use n-for-each-par-map with 16 threads I have
8 cores.

Here is the full program:


(define (store pair)
  (if (null? pair)
      (format #t "X\n")
      (let ((port (open-file "hn.scm" "a")))
        (format #t "~a\n" (car pair))

These calls to (format #t ...), which write to a port that is shared by
multiple threads, should be performed while holding a mutex to prevent
concurrent writes to the same port.

Ok, but it's called by:

  (n-for-each-par-map n sproc pproc lst1 lst2)

and the manual says about SPROC that :

  The sproc calls are made serially,
  in list element order, one at a time.

In my case SPROC is 'store', so my understanding is that
it should require no lock.

I/O operations in Guile do not include built-in thread synchronization,
at least not in the fast path cases.  However, an effort was made to
avoid _crashes_ on common architectures in the event of concurrent use
of the same port.  Our hope was that the worst that would typically
happen is garbled I/O.  Perhaps we failed to realize that hope.

I saw that behavior while use fibers.

How can I debug this?

It would be helpful to see a GDB backtrace from a crash in this program.

I will retry without mutex and only gdb to reproduce the crash
and have a trace.

Does it help to protect the shared port with a mutex?

I ran the program with gdb and a lock, it downloads more suff,
but it's also much slower.


Thanks again!

Amirouche ~ amz3 ~

reply via email to

[Prev in Thread] Current Thread [Next in Thread]