Android Positioning API WTF

mike_james

(prompted by the posting on the calorie/Calorie/kCal/Avocado counting app..)

Quite a few GPS tracking apps on my Samsung Galaxy SII show big jumps to lat, long 0,0 which is on the equator somewhere off the coast of Nigeria, especially if I keep the phone in my pocket while it is tracking. No my pocket is not a steamy equatorial jungle or an equally hot and boring bit of sea with a risk of piracy ....

Digging deeper into the APIs presented by Android, they state that the 'position' 0,0 indicates 'No GPS position'. You can ask other APIs for the number of GPS satellites but the position reported from the API may actually not be a GPS position - could be a cell based position with 0 GPS satellites used.

So the apps have to pick up exactly (0,0) which is a valid position and treat that as a no-position. Maybe on a lot of Android phones the GPS system doesnt output 0,0 for no position.

WTF didnt Google define something sensible like latitude > 90 degrees as a no-position then all of the app coders would have been forced to deal with it.

In my previous life, we met the same problem , but this was GPS on an oil survey ship and again some GPS receivers will output lat0,lon0 for no position (along with other flags which should tell you but which became confused - it could still see N satellites, used all of them to get a fix but could not actually solve a position because the geometry was bad ... ) .

We got around this because the GPS positions were input to a Kalman filter, and so if the ship surveying off Nigeria (quite possible) happened to pass near 0,0 it was extremely unlikely to go through exactly 0,0 and even if it did, the next position would be a few metres away .

So we just deleted any exact lat0,lon0 positions and let the filter interpolate across the hole.

ochrist

Interesting explanation. That would certainly account for some of the mysterious routes reported by CardioTrainer on my Samsung Galaxy SII. But it still doesn't explain why that app sometimes report me running several kilometers out in the water (I live not far from the coast, but I'm not related to that guy from Nazareth).

PJH

@ochrist said:

But it still doesn't explain why that app sometimes report me running several kilometers out in the water (I live not far from the coast, but I'm not related to that guy from Nazareth).

Insufficient satellites to pinpoint your location with any accuracy? Reflections?

ochrist

@PJH said:

Insufficient satellites to pinpoint your location with any accuracy? Reflections?

If I use Googles MyTracks instead on the same device I don't have any problems. Anything else I'm overlooking?

pjt33

@ochrist said:

If I use Googles MyTracks instead on the same device I don't have any problems. Anything else I'm overlooking?

Have you checked that the difference is the reported location rather than the coastline data?

dhromed

I read a while ago, but don't remember where or what or when*, that GPS communication data is so horribly underspecified that applications and APIs basically have guess half the time what a piece of data means. Thus if those APIs give balf-baked output, cut them some slack.

*) Appeal To Anti-Authority?

@dhromed said:

I read a while ago, but don't remember where or what or when*, that GPS communication data is so horribly underspecified that applications and APIs basically have guess half the time what a piece of data means.

This, perhaps? [url]http://esr.ibiblio.org/?p=801[/url]

ochrist

@pjt33 said:

Have you checked that the difference is the reported location rather than the coastline data?

Not sure I understand your question, and now my answer was swallowed by CS, so I'll try to recap:

I live a couple of kilometers from the beach. There's a railway and a major road in between, and I tend to run on this side of the railway.

CardioTrainer would report me running in a straight line from out in the water and into land without apparent reason, whereas Googles MyTracks show a very precise route (just came back from a small training session).

Someone_You_Know

@ochrist said:

But it still doesn't explain why that app sometimes report me running several kilometers out in the water

It used to be that when I would drive down Storrow Drive in Boston (a long expressway next to the river) the Google Maps app would almost always indicate that I was driving down the middle of the Charles River. This would always ruin the turn-by-turn navigation, because it couldn't find a street route from 221B The Middle Of The Charles River Avenue to wherever I was trying to go. Then I got an over-the-air update to Android 2.3, and now the GPS doesn't work at all, ever. Progress.

tOmcOlins

@ochrist said:

Interesting explanation. That would certainly account for some of the mysterious routes reported by CardioTrainer on my Samsung Galaxy SII. But it still doesn't explain why that app sometimes report me running several kilometers out in the water (I live not far from the coast, but I'm not related to that guy from Nazareth).

Really? You have the same last name...

blakeyrat

@Spectre said:

@dhromed said:
I read a while ago, but don't remember where or what or when*, that GPS communication data is so horribly underspecified that applications and APIs basically have guess half the time what a piece of data means.

This, perhaps? http://esr.ibiblio.org/?p=801

The sad thing is that all those problems are fixable. The GPS data isn't bad because the data is bad, it's bad because the ASCII format used for reporting the data is bad. One would hope that in the years since that article was written:

1) The GPS format standards committee would create a format that wasn't shit (although they sounded a lot like the "we simply do not give a shit" type)

2) Devices like cellphones would be using some kind of binary interface that doesn't share the same deficiencies

DOA

@Spectre said:

This, perhaps? http://esr.ibiblio.org/?p=801

That was a good read. Someone tell that guy about TDWTF.

ochrist

@tOmcOlins said:

Really? You have the same last name...

All right, I was lying. He's my cousin. I'm just mad at him cause he never taught me that trick.....

The_Assimilator

I had to go through similar pain as ESR last year... GPS data in NMEA format really is a crapshoot (and filtering the USB bus to find an actual GPS device... fun fun fun!). Although to be fair, NMEA does stand for "National Marine Electronics Association", so ESR's kvetching about not having altitude data is somewhat misplaced. If the GPS vendors wanted to include altitude data, they should've got off their lazy asses and devised a spec that includes it.

TRWTF is that the NMEA-183 specification itself, which costs upwards of $140 but is wholly documented in numerous places online.

As for the Android stupidity... it doesn't surprise me. When you allow Java developers to build an OS, this is the kind of shit you end up with.

Xyro

@blakeyrat said:

2) Devices like cellphones would be using some kind of binary interface that doesn't share the same deficiencies

That is worse, not better! Did you read the full post?

blakeyrat

For a phone with a known GPS model and a driver, it would be better. For software with an unknown GPS model and no driver, it would be worse. Context matters.

RobFreundlich

@Someone You Know said:

Then I got an over-the-air update to Android 2.3, and now the GPS doesn't work at all, ever. Progress.

I had this same problem with my Droid X after the 2.3 update from Motorola, and I found that clearing my system cache solved it. Here's a link to the posting I found telling me how to do it:

http://cellphoneforums.net/motorola-droid/t328906-clear-system-cache-droid-x-x2.html

FrostCat

@blakeyrat said:

@Spectre said:
@dhromed said:
I read a while ago, but don't remember where or what or when*, that GPS communication data is so horribly underspecified that applications and APIs basically have guess half the time what a piece of data means.

This, perhaps? http://esr.ibiblio.org/?p=801

The sad thing is that all those problems are fixable. The GPS data isn't bad because the data is bad, it's bad because the ASCII format used for reporting the data is bad. One would hope that in the years since that article was written:

1) The GPS format standards committee would create a format that wasn't shit (although they sounded a lot like the "we simply do not give a shit" type)

2) Devices like cellphones would be using some kind of binary interface that doesn't share the same deficiencies

Except that the standards committees apparently can't be bothered. Just about all the chipmakers have binary protocols that give more information, but--surprise--they're all incompatible. ESR talks about it every once in a while, so we're pretty much stuck with NMEA 0183 indefinitely.

Even if someone came up with a new standard, we'd probably be stuck with a transitional period of years.

Someone_You_Know

@RobFreundlich said:

@Someone You Know said:
Then I got an over-the-air update to Android 2.3, and now the GPS doesn't work at all, ever. Progress.

I had this same problem with my Droid X after the 2.3 update from Motorola, and I found that clearing my system cache solved it. Here's a link to the posting I found telling me how to do it:

http://cellphoneforums.net/motorola-droid/t328906-clear-system-cache-droid-x-x2.html

Yeah, I've tried every crazy trick I could find using Google, all to no effect. It never detects any satellites or reports any data; it just sits there and says "Waiting for Location". At this point I'm assuming that the GPS hardware is just toast and I'm going to have to get a new phone.

Digging deeper into the APIs presented by Android, they state that the 'position' 0,0 indicates 'No GPS position'. You can ask other APIs for the number of GPS satellites but the position reported from the API may actually not be a GPS position - could be a cell based position with 0 GPS satellites used.

Where is this? I'm pretty sure that this is not true, and returning invalid 0/0 responses should actually be something that the manufacturer is required to rectify, especially a new phone like the Galaxy S II.

Sadly, I think I have a guess why this error exists for you, especially if this is a VZW GSII, but this is really not an Android fault. The expected behavior is to simply not provide a location update if there's no valid position

RobFreundlich

@Someone You Know said:

@RobFreundlich said:
@Someone You Know said:
Then I got an over-the-air update to Android 2.3, and now the GPS doesn't work at all, ever. Progress.

I had this same problem with my Droid X after the 2.3 update from Motorola, and I found that clearing my system cache solved it. Here's a link to the posting I found telling me how to do it:

http://cellphoneforums.net/motorola-droid/t328906-clear-system-cache-droid-x-x2.html

Yeah, I've tried every crazy trick I could find using Google, all to no effect. It never detects any satellites or reports any data; it just sits there and says "Waiting for Location". At this point I'm assuming that the GPS hardware is just toast and I'm going to have to get a new phone.

Bummer! Mine was seeing two satellites and not locking onto them, and clearing system cache made it (and me) much happier. Sounds like you're right, though, that you hardware may be blown. What phone? Any chance of a repair/exchange?

Someone_You_Know

@RobFreundlich said:

@Someone You Know said:
@RobFreundlich said:
@Someone You Know said:
Then I got an over-the-air update to Android 2.3, and now the GPS doesn't work at all, ever. Progress.

I had this same problem with my Droid X after the 2.3 update from Motorola, and I found that clearing my system cache solved it. Here's a link to the posting I found telling me how to do it:

http://cellphoneforums.net/motorola-droid/t328906-clear-system-cache-droid-x-x2.html

Yeah, I've tried every crazy trick I could find using Google, all to no effect. It never detects any satellites or reports any data; it just sits there and says "Waiting for Location". At this point I'm assuming that the GPS hardware is just toast and I'm going to have to get a new phone.

Bummer! Mine was seeing two satellites and not locking onto them, and clearing system cache made it (and me) much happier. Sounds like you're right, though, that you hardware may be blown. What phone? Any chance of a repair/exchange?

It's a Samsung Fascinate (Verizon's Galaxy S). I can probably get it replaced at a Verizon store; I just haven't had the time to drag my ass down there and do it.

Bulb

@Xyro said:

@blakeyrat said:
2) Devices like cellphones would be using some kind of binary interface that doesn't share the same deficiencies

That is worse, not better! Did you read the full post?

Most of them do, or at least most vendors write their own GPS drivers for Android, which suggests that they do. Most of those drivers suck. Some of them suck really hard. I've seen things like driver, that does not report GPS even exists in the device before it has a fix. And takes about 20 minutes to get a fix. So I wouldn't be the least surprised by one reporting it does not have a fix incorrectly.

Many devices also randomly occasionally report one location with bogus heading, some even bogus position (you can't really trust the position until you got at least 3 consistent ones). That's nothing to do with Android. Most cheap GPS receivers do.

And no, after I had the pleasure to work with Bada, I wouldn't expect Samsung to have any higher quality than random Chinese noname when it comes to software (just getting their SDK upload something to the emulator was utter pain).

mike_james

I think that the problem is that there is a lot of information in the location class regarding a successful position fix, but nothing in the location class saying categorically 'This device really does not know where it is', or 'this device just used Cell ID to position itself, not GPS'

Basically they handle failure by denial.

Because of the way GPS calculates position, the chance of latitude and longitude being being exactly 0 at the same time is almost 0 . So next best is for app writers to filter 0,0 positions as bad.

Secondly for position jumping :

The problem with a GPS in your pocket or on your body is the antenna only sees a part of the sky. Small errors in ranges to satellites tend to provide a position fix that averages to being along a line rather than a point. (you lose ranges that cancel out the error coming from the other side of you) If the GPS is in the middle of your chest and you are facing east for example, the positions you see will tend to be clustered along a north / south line , possibly hundreds of metres from your position.

Try leaving a GPS on a windowsill (in a brick house or one with aluminum insulation) and log its position. You will get a cluster of points around the real position, and occasional 'movement' out to some distance and back again, generally lined up with the wall of the house.

And elsewhere on the net you will find people who have hacked their Galaxy S GPS antennas with bits of tinfoil to make them work better.

It seems to be a design problem with that model of phone. The SII has a much better GPS performance.

mike_james

Before launching US carrier attacks, the phone is an O2 (UK) branded GT19100 model.

boomzilla

@mikedjames said:

Before launching US carrier attacks, the phone is an O2 (UK) branded GT19100 model.

Daaaangggg....I've heard of mobile phone triggered IEDs, but now they're launching airstrikes?

@mikedjames said:

I think that the problem is that there is a lot of information in the location class regarding a successful position fix, but nothing in the location class saying categorically 'This device really does not know where it is', or 'this device just used Cell ID to position itself, not GPS'
Basically they handle failure by denial.

This is a joke, right? You're joking, right?

Location | Android Developers

Boy, if only there was a way you could check WHAT PROVIDER the Location object you're provided in the callback came from? Man! That would be swell! And if the device is reporting network based positioning as a GPS position, then that's flat out a manufacturing fuckup, not Android

blakeyrat

Whoa, someone defending Android.

heterodox

@blakeyrat said:

Whoa, someone defending Android.

Well, to quote you, "If you're going to be a pedantic dickweed, at least make sure you're actually correct."

Any misapprehension that assertive (read: self-righteous), that easily researchable, and that wrong really needs to be instantly corrected. Volmarias beat me to it by a nap's length. :(

@blakeyrat said:

Whoa, someone defending Android.

I have significant experience in Android, and in particular in the location apis at the framework level and lower.

Suffice to say I know that most of this is utter horseshit and is probably a result of either not knowing the apis or taking OEM fuckups as the official standard. Android isn't perfect by any stretch of the imagination but if you're going to claim that the docs say that 0,0 is the default "error" position you'd damned well better have a link, because I can assure you that this is horseshit.

Watson1

@boomzilla said:

@mikedjames said:
Before launching US carrier attacks, the phone is an O2 (UK) branded GT19100 model.

Daaaangggg....I've heard of mobile phone triggered IEDs, but now they're launching airstrikes?

Don't worry; the missiles all have GPS guidance.

ekolis

@Watson said:

Don't worry; the missiles all have GPS guidance.

Oh, good, then they'll all head for Nigeria and wipe out the spammers!

mike_james

Point taken about the provider check. Apologies.

Maybe I should focus more on investigating the Galaxy SII Samsung/Broadcom GPS implementation, which is giving 0,0 positions - shows up as a problem even in Google Maps, and other applications.(Navionics marine charts, Open StreetMap, Sygic GPS navigation all show the same jump to 0,0 with poor GPS rather than a fallback to cell ID based positions..)

Somewhere somebody has taken the GPS 'invalid position flag' and mapped it onto something that looks like a valid position.

But I was looking at the source of a simple GPS yacht anchor drag alarm and this used the lat=0, long= 0 position as a means of detecting 'no fix' .

Its not a very sophisticated app, but it shows that other models of phones also show the 0,0 problem ...

http://code.google.com/p/anchor-alarm-free/source/browse/trunk/AnchorAlarm/src/com/nickthorneltd/anchoralarm/AnchorAlarm.java

tharpa

But are you related to that guy from Lumbini? He was teaching people to walk on the water (as part of a long list of lesser powers) 500 years before the guy from Nazareth.