To find new targets in a country (TLD) we created the HostSearcher.pl script. It uses the Google search parameter q=site:<TLD> feature, e.g. for Malaysia (<TLD>=my)
- wget -U \"Mozilla/5.0\" 'http://www.google.com/search?q=site:my&num=100&hl=en&hs=xEp&lr=&client=firefox-a&rls=org.mozilla:en-US:official_s&as_qdr=all&start=900&sa=N'
It first interrogates Google for up to 1000 unique hosts in a selected country (using the Top Level Domain (TLD) feature). The tool then pings each host 10 times (by default) to ensure it responds and saves the min-RTT. We manually filter out hosts that do not have reasonable ping RTTs for the specified country, and also those identified as not being in the right location (country) according to https://api.ipdata.co. We then manually browse the host web site, looking for possible location information (e.g. street address, map etc.). We use this information together with Google maps (see https://support.google.com/maps/answer/18539?co=GENIE.Platform%3DDesktop&hl=en) to get the latitude and longitude. The final list of hosts obtained can be further filtered by whether they are really in the country, their loss rate, their min-RTT, diverse location within the country etc. Though not always accurate this has been a great aid on numerous occasions. Unfortunately for a few developing countries (e.g. Chad, Libya) even this method fails to produce suitable monitorable hosts.
For more information on using HostSearcher.pl see Finding PingER Hosts.