[Gnumed-devel] HL7 lab data importer (branch)

[Luke Kenneth Casson Leighton]

hi, this is just a quick message to let people know that i've been
working on a lab data importer for gnumed, it takes HL7 v3 (XML) data.
 if anyone would like to help with testing (by e.g. writing unit tests
or by just playing with it to see what squeaks and what works), you
will need:

* an HLv3 capable version of hl7 library, which you can get at:
   http://github.com/lkcl/hl7

* the luke-labs branch of
git://gitorious.org/gnumed-shilbert/gnumed.git - the .git/config file
should look like this:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = git://gitorious.org/gnumed-shilbert/gnumed.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master
[branch "lukes-lab"]
        remote = origin
        merge = refs/heads/lukes-lab

and you can then do "git checkout -b lukes-lab" after a git clone.
something like that, anyway.

to test, you can do:
cd client/business
python gmHL7.py test {name_of_XML_file_with_HL7Messages.xml}

the format and importing is _mostly_ compliant with this:
http://wiki.gnumed.de/bin/view/Gnumed/LabImporters#British_Columbia_Canada_source_E

there's a couple of extra things that have been done - one of them is
to treat test results as indexable by test-type, per HL7 message.
thus, on subsequent imports, the status of a test result will be
correctly updated, rather than added again and again.  an example -
OBX|1|ST|14682-9^Creatinine|... - is uniquely indexable in that
message by "14682-9" and "Creatinine".  thus, any future imports /
updates from the same lab, for the same patient, with the same lab
request id, the results end up being correctly replaced/updated.  i
hope!!  ok, yes, it does actually work, for the tests i've carried out
so far.

please bear in mind this is _complex_ stuff, and it has only been
written in the past 3 days.  that's a long time, for me, but i have
had to take some shortcuts (no unit tests) and there is further work
to be done (lab reviewers are *dropped* because there's not yet a
place in the gnumed database design to put them).  etc. etc.

also, i intend to improve the hl7.py library by taking what MIRTH have
done, with their support for multiple versions (2.1, 2.2, 2.3, 2.31,
2.4, 2.5) which are encoded into some XML files, and i intend to take
those XML files, auto-generate some python, and thus hl7.py will
magically grow support for all those different message revision
formats for HL7.  the neat thing about using what MIRTH have done, in
this way, is that the benefit of all their expertise and all the
testing done, suddenly now the hl7.py library "gains" all that
expertise "for free", with very little effort required.  no, i will
_not_ be adding a whopping great amount of XML to hl7.py, i can't
_stand_ XML.  i'd rather auto-generate some python classes or some
python dictionaries than inflict XML onto people :)

so, that's the story: gnumed seems to have magically sprouted an HL7
lab data importer - it'll need reviewing and testing.

l.