Tri-lateration with CBG

TULIP results

Faisal generated TULIP results for SLAC as target (134.79.18.188) and TULIP geo-located it to be in Wyoming. Screenshot here. This is obviously way off.

CBG results

Dr. Les quickly explained to me what TULIP was doing and wanted to do a quick analysis of the same location (i.e. SLAC) using CBG with tri-lateration. So I found a target located at SLAC (134.79.18.134) in CBG list. We ran CBG with tri-lateration for this target. The results were way off. The error distance was of the order of ~3200 km.

However Dr. Les pointed out that we should be using three landmarks that have the minimum RTT values from the target. Thus we shuffled our values accordingly and re-ran the test with two different set of landmarks. This helped to verify our results. Table below shows the results:

Other results are also encouraging, so far I've analyzed around 20 targets, comparing multi-lateration and tri-lateration results. CBG with tri-lateration seems to be performing well.

Points to consider

These results prove that tri-lateration works. So the question is why is TULIP failing so miserably?

CBG multi-lateration vs CBG tri-lateration comparison

Spreadsheet here shows a comparison of error (in km) between CBG with multi-lateration and CBG with tri-lateration. The technique I've followed:

1. Sorting target lists in ascending order on the basis of distance between the target location and landmark location.
2. Re-running the CBG code for new results.
3. Populated the spreadsheet with results.

What I didn't do so far and why:

1. Avoiding duplicate landmarks.
   1. Reason: If you look at the spreadsheet you will notice that there are duplicate entries for multi-lateration as well. You can infer this from matching Estimated Lat/Longs to Actual Lat/Longs and by observing the distance to the nearest landmark values. Also a few targets don't have more than two landmarks and in all such cases those are duplicates (in terms of Lat/Longs). So in such a case I don't have an option but to use the duplicate ones. However I do require comments on this - whether I should remove duplicates or not. The two reasons of my concern are that multi-lateration uses duplicate landmark values and a few targets having none but duplicate landmarks.
2. Avoiding landmarks present within a target's vicinity. This requires taking note of the location.
   1. Reason: Closely related to the point mentioned above.

What I've tried and didn't work:

• Sorting target lists on the basis of RTT to bring those landmarks at the top (of the Target files) which have lowest RTT values. This technique fails since in some cases the top three landmarks in the Target files are located thousands of kilometers apart. I don't know how these landmarks managed to have such low RTT values from the target. CBG with tri-lateration failed on such a sorting technique .. because landmarks were way off!

Results, some discrepancies and explanation

In the spreadsheet we have made various calculations in order to understand the results.

1. Amount of NaNs in error for multi-lateration and tri-lateration

67/171 for multi-lateration and 6/171 for tri-lateration.

NaN (Not a Number) is a value of numeric data-type representing an undefined or unrepresentable value, especially in floating point calculations. More here.

According to the CBG code NaNs represent "bad pairs that lead to no region". This means that landmarks that fail to produce intersection regions, consequently also fail to produce an estimate for the location of the target and instead giving out an erroneous value. Author has handled such values with NaN (code snippet below).

if constraintType
  warning('Trying speed of light')
  constraintType = 0;
  % switch the constraint type and try again
  [locest,actual,error,regarea,distNearestLandmark,target_id,constraintType,inRegion\] = geolocate(file,extension,hullbool, constraintType, bestlineTable);
  return;
else
  % find the badPairs that lead to no region, write them to stdout
  badPairs = analyzeNoRegion( measurements )
  %error(\['No SOL intersection region for ', char(file)\])
  region = \[NaN NaN\];
  locest = \[NaN NaN\];
  error = NaN;
  regarea = NaN;
  results = \[target_id error; distNearestLandmark regarea; locest; actual; region\];
  dlmwrite(\[char(file),char(extension)\],results,' ');
  return;
end;

2. Some results have enormous errors (|error|>1000)

This is true for both multi-lateration and tri-lateration. And reasons could be one or more of the following:

• Number of landmarks aren't enough i.e. only 3 or 2 or less.
• Number of landmarks aren't enough and those which are present are duplicates.
• There are enough number of landmarks but none are good enough i.e. the RTT is in the order of 50+ ms (true for error distances in the order of thousands of km).
• There are enough number of landmarks but mostly aren't good enough i.e. the RTT is in the order of 25+ ms (true for error distances in the order of 1000+ km).

I've inferred these from looking at the Target files.

3. Tri-lateration is performing better than multi-lateration

There are 37 instances where tri-lateration performs better than multi-lateration, 26 instances where multi-lateration performs better than tri-lateration and in the rest both perform equally well. The reasons for this could be one or more of the following:

• We sorted the Target files on the basis of distance between the target and landmark. This promoted those lat/long values to the top of the list which were closer to the target. Though any such sorting technique on these values doesn't affect multi-lateration results but it makes a huge impact on tri-lateration results. The reason being the way these two techniques use these values. Multi-lateration considers all values and figures out regions of intersection whereas tri-lateration simply takes three values to find an intersection region. If we have more landmarks and some of them fairly close to the target multi-lateration will give good results. Even if some values aren't really good, it won't cause multi-lateration to behave in an entirely different way. However in the case of tri-lateration the better the landmark estimates we have the better the results are. Since we are only considering three values here, even a single one of those three values can make a big difference.

I'm inferring this from sorted Target files for tri-lateration. Before sorting Target files we were getting way-off results. We then tested our approach of taking RTT into account and manually re-arranged values to produce fairly accurate results. Later sorting on the basis of RTT failed as RTT isn't always directly related to distance. Our latest try involved sorting on the basis of distance instead of RTT, which proved to be quite successful.

So in short for those Target files which have both good and not-so-good landmark values, tri-lateration performs better than multi-lateration because good values are moved to the top of the list and are exploited by tri-lateration whereas multi-lateration takes into account all values.