TULIP results
Faisal generated TULIP results for SLAC as target (134.79.18.188) and TULIP geo-located it to be in Wyoming. Screenshot here. This is obviously way off.
CBG results
Dr. Les explained what TULIP was doing and wanted to do a quick analysis of the same location (i.e. SLAC) using CBG with trilateration. So I found a target located at SLAC (134.79.18.134) in CBG list. We ran CBG with trilateration for this target. The results were way off. The error distance was of the order of ~3200 km.
...
CBG with trilateration is performing well.
CBG multilateration vs CBG trilateration comparison
Spreadsheet shows a comparison of error (in km) between CBG multilateration and CBG trilateration. The technique I've followed:
...
- Avoiding duplicate landmarks.
- Reason: If you look at the spreadsheet you will notice that there are duplicate entries for multilateration as well. You can infer this from matching Estimated Lat/Longs to Actual Lat/Longs and by observing the distance to the nearest landmark values. Also a few targets don't have more than two landmarks and in all such cases those are duplicates (in terms of Lat/Longs). So in such a case I don't have an option but to use the duplicate ones.
- Avoiding landmarks present within a target's vicinity.
- Reason: Closely related to the point mentioned above.
Results, observations and explanation
In the spreadsheet we have made various calculations in order to understand the results. The following are a few observations and their explanation.
...
1. Amount of NaNs (in error distance) for multilateration and trilateration
...
The table below shows number of NaN occurrences for both the techniques.
...
See latest spreadsheet for details.
...
2. Some results have enormous errors (|error|>1000)
...
This is true for both multilateration and trilateration. And reasons could be one or more of the following:
...
These have been inferred from looking at the Target files.
...
3. Trilateration is performing better than multilateration in some cases
...
There are 23 instances where trilateration performs better than multilateration, 29 instances where multilateration performs better than trilateration and in the rest both perform equally well. The reason as far as I understand is:
...
So If we have say 10 landmarks and 4 of them had relatively lower RTT to the target, multilateration will give good results. Even if some values aren't really good, it won't cause multilateration to behave in an entirely different way. However in case of trilateration, better the landmark estimates we have, the better the results are. Since trilateration considers three values, even a single one of those three values can make a big difference.
CBG trilateration vs Improved trilateration comparison
Spreadsheet here shows comparison between CBG's trilateration and Farrah's improved trilateration. A few important points:
- There are a total of 174 targets for CBG out of which 131 remain after ignoring values that either have error in the range "error<1km" (i.e. the target and at least one landmark are probably in the same location) or contain "NaN".
- Improved trilateration by Farrah produced results for only 78 targets so far. Her method produced 7 NaN values which she ignored.
- Only 74 targets overlap between CBG trilateration and improved trilateration.
- If I don't ignore CBG's values that have estimate error "error<1km" then CBG trilateration performs 64/74 times better and improved trilateration performs only 10/74 times better.
- Even if I ignore values with error estimate "error<1km" then CBG performs 32/74 times better, improved trilateration performs 10/74 times better and the rest are unaccounted for.
Procedure to create CBG trilateration and multilateration comparison
First the target files need to be sorted according to the RTT in order to have better landmarks at the top of the file for every target. This creates no impact on multilateration but it helps trilateration achieve better results.
...