[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[help-GIFT] Fwd: Big database on GIFT. gift-generate-inverted-file error
From: |
Wolfgang Müller |
Subject: |
[help-GIFT] Fwd: Big database on GIFT. gift-generate-inverted-file error. |
Date: |
Wed, 25 Feb 2004 08:41:35 +0100 |
User-agent: |
KMail/1.5 |
Forwarded message from Seventeen, to make discussion public.
-
Dear all,
I'm using gift-0.1.9 and has a image database which has 678929 images
(about 20G in size).
Few month ago, I devide these images into 5 directories and do
gift-add-collection.pl
This was ok to run without any error, just took a lots of time. (about
17 days on a P4 2.8G machine)
After I check the gift-add-collection.pl. I find that this perl script
mainly do 3 different things.
1. imagemagick convert to generated thumbnail and ppm files.
2. gift-extract-features and gift-write-feature-descs to get features
and creat .fts files.
3. gift-generate-inverted-file to generate InvertedFile.db.
I merge these 5 directories into single one directory and modify the
url2fts.xml.
The 1 & 2 stage dont need to be run again.
While I was doing gift-generate-inverted-file. It spends lots of time
(about 300 mins), and creat the gift-auxiliary-1 9.2G big. But finally
come out the following error message.
__FILE__:__LINE__: lToBeSorted false, after seekg(0)
__FILE__:__LINE__: lToBeSorted false, after seekg(-2147483648)
__FILE__:__LINE__: lToBeSorted false, after seekg(0)
__FILE__:__LINE__: lToBeSorted false, after seekg(-2147483648)
9.2G Feb 24 19:32 gift-auxiliary-1
672k Feb 24 11:46 InvertedFileFeatureDescription.db
173M Feb 24 11:46 url2fts.xml
Actually it's part of my master thesis work. I'm try to put GIFT with
big medial image database into a grid enviroment. I hope the grid
technology can give me a reasonable speed to run the GIFT with the large
medial image database. So I'd like to have a measurement of the speed of
1 nodes vs. 4, 8 16 or even 32 nodes.
I think the gift-add-collection.pl can speed up by put into distrubuted
cluster. And the idea to me is to seperate the big InvertedFile.db into
small one over lots of nodes. Then query these nodes (by
gift-mrml-client.pl) and get those result for finally processing.
So can you please tell me what's the error message mean and give me some
suggesstions.
Thanks in advance.
--
Seventeen Chen RHCE
Graduate Institute of Medical Informatics
Center for Biomedical Informatics
Taipei Medical University
250 Wu Hsing Street
Taipei City 110
TAIWAN
TEL:+886-2-23776730 ext 107
Fax: +886-2-27279049
E-mail: address@hidden
- [help-GIFT] Fwd: Big database on GIFT. gift-generate-inverted-file error.,
Wolfgang Müller <=