[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Gnumed-devel] pipelining experiment

From: Syan Tan
Subject: [Gnumed-devel] pipelining experiment
Date: Sun, 02 Oct 2005 23:53:23 +0800


Some holiday fun while Karsten's away:-


 the pyPgSQL module uses cursors so it takes I think 3 -4 round trips (create cursor. execute cursor, fetch result, close cursor) for every sql statement , so this slows down  the access remotely salaam.

I used the script from test-area/sjtan/  to retrieve a Kirk's record ( which is Id= 12),

and this took some time too. reorganizes the single threaded single connection retrieval into

a multi threaded worker pool each with it's own connection, and uses a Queue.Queue to submit statements to available workers. 

 I was able to use about 30 threads; It looks faster, but using unix "time python" the user time is still about a minute.

( Each statement will still take 3 round trips because of the use of sql cursor, although now there is some pipelining, due to concurrently running connections ).

How long does it take running as a batch file to psql  ?

If the XMLRPCServer serving yaml, text, xml , serialized python object or whatever, did materialize, what would be the gain ?

( using a web server, one could retrieve a constructed record in 10 seconds , I think).



reply via email to

[Prev in Thread] Current Thread [Next in Thread]