|
From: | Kristoff |
Subject: | Re: how to forward RTCM data to gpsd? |
Date: | Sat, 12 Dec 2020 00:55:10 +0100 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 |
Hi all, Small follow-up.I decided to have a good fresh look at the code of ifgps.c the next morning. It turns out that it is actually a lot easier then what I first thought. The format is actually very simple: every byte starts with 2 bits '01', followed by 6 bits of data in *reverse* in order. That's it.
I created a small python application that functions as TCP-server so I could pipe the received dgps signals into 'nc ... | gpsdecode' or gpsd. I managed to get dgps data from my receiver -just the signals as received over the air without any processing- into gpsdecode and gpsd, ... most of the time! Quite often, both gpsdecode and gpsd seams to lose synchronisation with the incoming dgps signals (Although my own dgps-decoding python code I had running at the same time did not have any issues). (*)
Gpsd does forward the received dgps messages to the ublox-8 as expected.The net-result is that the receiver switches from the mode "3 constellation (gps, glonass, Galileo) + egnos" to "only gps + dgps". Stopping the rtcm messages from reaching the gnss receiver made it switch back to 3 constellation + egnos. (**)
So, unless I am completely wrong, this means that the rtcm messages do are received and processed by the ublox gnss receiver.
(Somebody correct me if I am wrong) A note concerning the accuracy:In the tests I conducted, GPS + dgps resulted in the roughly the same accuracy as GPS + Galileo + glonass + egnos. My gnss receiver has about a 180 degree view of the sky (as it is located on my side of my house), and the error of the position as reported by the two tests -using google-maps as a reference- was roughly the same in both cases: a 4 meter error in the test using dgps, 3 meter in the other test.
This test was done based on a DGPS station roughly 2 km from here.An interesting test - I think- would be to compare GPS + EGNOS with GPS + DGPS.
(*) Small sidenote: gpsd 3.17 (which is the standard version on rasbian / debian 10) crashed when receiving the first RTCM messages. I compiled gpsd 3.22, which did work correctly.
(**) Something strange:When not using dgps (so all three 3 + sbas enabled), xgps did say "3D FIX DGPS", but all three egnos satellites where marked as "used: no". So what 'DGPS' does xgps refer to when all the egnos/sbas satellites are marked as 'not in use'?
Cheerio! Kr. Bonne. On 10/12/2020 2:32 a.m., Kristoff wrote:
Hi Gary, OK, I searched a little bit more on this.Using gpsdecode, I managed to extract one single RTCM message from rtcm2.log{"class":"RTCM2","device":"stdin","type":9,"station_id":268,"zcount":249.6,"seqnum":1,"length":5,"station_health":0,"satellites":[{"ident":13,"udre":0,"iod":3,"prc":-26.120,"rrc":0.068},{"ident":2,"udre":0,"iod":73,"prc":1.220,"rrc":-0.080},{"ident":8,"udre":0,"iod":22,"prc":23.760,"rrc":0.030}]}It turns out this message is stored in exactly 35 bytes. If I run gpsdecode on these 35 bytes, it decodes.But if I change the very first bit or the very last bit of this 35 bytes file with a hex-editor and run 'gpsdecode' on it again, it did not decode anymore. This indicates that the message boundaries are correct.The message has "length=5", so it you include the header, this is 7 words (of 30 bits), i.e. 210 bitsIf we reorder the 35 bytes in 7 rows, and do a bitdump, we get this: 01 110011 - 01 001000 - 01 000101 - 01 100110 - 01 001111 01 011000 - 01 000001 - 01 100100 - 01 100010 - 01 110111 01 100111 - 01 000000 - 01 000101 - 01 110011 - 01 111110 01 011101 - 01 111111 - 01 111001 - 01 110111 - 01 001010 01 000000 - 01 111000 - 01 011101 - 01 000011 - 01 111000 01 110110 - 01 011110 - 01 111111 - 01 011011 - 01 011111 01 101101 - 01 011111 - 01 111000 - 01 010010 - 01 110010As we can see, It turns out that every bytes starts with '01'. (hence the different in size: 280 bits in the file for a RTCM message of 210 bits = a 4/3 ratio)Now, just removing the '01' did not seams be enough. The message-structure still does not match. I need to look further. Two quick ideas at this point. - either they use a scrambler on the stream (i.e. they reorder the bits).I could image this would be logical as -if this indeed is the stream that is actually transmitted from the GPS satellites- it makes sense to reorder the bit so that bit-errors also get spread out over the different words increasing the chance that the FEC error-correction mechanism is able to handle it.- Or it could be that they just run a exor-stream on the data. This is also a standard-trick on radio, as this generate more bit-changes during transmission which helps to keep the receiver syncronised with the incoming stream.OK, time to get some sleep now, but if somebody has another idea, feel free to reply! :-)This is really starting to look like a CTF! :-) Cheerio! Kr. Bonne. On 10/12/2020 12:59 a.m., Gary E. Miller wrote:Yo Kristoff! On Thu, 10 Dec 2020 00:39:16 +0100 Kristoff <kristoff@skypro.be> wrote:On 9/12/2020 2:19 a.m., Gary E. Miller wrote:I am new to the mailing-list, with just a quick question. Can somebody provide a quick overview of what gpsd expects RTCM messages should look like?gpsd does not care. gpsd just forwaards to the GNSS receiver. The receivers are very picky.Apparently, gpsd will only forward it if the format it receives matches what it think it should receive.That is a good thing.As part of an exercise to learn Software Defined Radio and GNU Radio, I "build" a receiver in GNU Radio for the marine DGPS radio-stations in the mediumwave band, including some python-code to decode the RTCM messages. (not yep posted online .. first code-clean-up :-) )gpsd can also decode them. Check out drivers/driver_rtcm[23].cdid that. However, driver_rtcm is the higher-layer code. The lower layer is done by isgps.c. If you look at the the data structures as found on rtcm2, they do not match how the bits are transmitted on the air. The structures are based on words (32 bit) with the 2 "pad" bits being the 2 last bits of the previous word (which have to be repeated).That is very common. All the GNSS receivers I know do that. For RTCM and for Subframe data.So, this means that something is reordering the datastream -as transmitted on air- to this structure: either the DGPS receiver, or isgps.c.Makes sense.I did look at isgps.c, but -to be honest- it looks more like a CTF challenge in a cybersecurity conference.Yes, very ugly. Much easier for you to add the 2-bits of padding.So the question remains, what is the format that gpsd expect from a tcp-feed?Same thing.My plan is to write a small TCP-server in python and let dgps connect to it.You will find little improvement in the accuracy of a recent receiver unless the RTCM bas is very close (<0.5 km)I live about 2 km from the dgps station, so that should not be so bad.I live 1 km from a base station. I see no improvement with RTCM2 or RTCM3. YMMV.Now, the question is, what is the format for RTCM messages should look like?They should look exactly like RTCM messages. The trick is what your receiver wants.At this point, the trick is to know that gpsd want. If that does not match, it it will even think of relaying the message.gpsd wants what every receiver I have seen wants.gpsd has several samples: test/daemon/*rtcm*.logI had already looked at it. It's clearly not the bitformat as transmitted on air. Same thing for sample.rtcm2Just plus the 2 padding bits?If you run gpsdecode on the rtcm2 file, it are all type-9 messages. As the message type-number comes just after the sync-pattern, you should see 01100110 before or after 100100 (or with bits inverted or reversed). Try looking for that!RTCM makes my brain hurt. Maybe after I get the subframe working I'll look at RTCM. It does what I want, works for me, and others so I have no rush. s it is, ain't broke, nothing to fix.Also, if you look at the output of gpsdecode for that file, you have 44 type-9 messages with length 4 (6 words in total), and 87 message with length 5 (7 words in total). so, in total that is 873 words, or -if if you take a word being 32 bits instead of just 30-, that is equivalent to 3492 bytes. The sample.rtcm2 file -if you strip of the comments in the beginning- is 4690 octets. This means there is all kind of other stuff in that file too. Same thing for rtcm2.logIt is what it is. Those are raw captures from real sources, and they work when passed on the test receivers. You need to figure out how to duplicate them.Much of the RTCM code has been driver by submissions. Like most of gpsd.I do not know if this so for all RTCM messages. or if this is something specific for the marine DGPS service.Dunno. All gpsd does it take RTCM in and pass it on.Is there still somebody on the list who actually wrote or maintains that code?No patches in a while. People have been using it, and no reported bugs.Just stuff the bits as I receive them from the radio to gpsd?Well, you wrote the radio part. So write your radio so that is the case. Like everyone else did.The radio just creates bits one by one, and -if you look at the specs that are out there- that just matches how it is received.Matches the wire format, not the de-factor GNSS reciver format. I have asked several vendors to document the difference, but never got any answers. RGDS GARY---------------------------------------------------------------------------Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 gem@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can't measure it, you can't improve it." - Lord Kelvin
[Prev in Thread] | Current Thread | [Next in Thread] |