Menu
AdkernelTopicCrawler is an automated site analysis tool that serves as a component of Adkernel’s contextual advertising solution for topic analysis and discovery of new content.
Adkernel works with publishers to increase the monetization of their properties by categorizing their inventory for contextual bidding by advertisers seeking high quality content across the web.
User Agents:
AdkernelTopicCrawler/2.0 (compatible; Mozilla/5.0; +http://adkernel.com/robot/)
Webmasters: AdkernelTopicCrawler respects all robots.txt rules. To disallow the bot, please add the following to your robots.txt file:
User-agent: AdkernelTopicCrawler
Disallow: /Our bot's traffic is validated through Reverse DNS (rDNS) Lookup to ensure authenticity. All bot requests originate from IPs that resolve to our domain, ***.crawler.adkernel.com, and the resolved domain must also forward-resolve to the original IP.
Perform a reverse DNS lookup using nslookup, dig or host:
nslookup [IP Address]
dig -x [IP Address]
host [IP Address]
Confirm that the PTR record resolves to ***.crawler.adkernel.com. Any IP not resolving to ***.crawler.adkernel.com may not be associated with our bot. If it does, proceed to step 2.
Forward-resolve the domain name back to the original IP:
nslookup -query=A ***.crawler.adkernel.com # for IPv4
nslookup -query=AAAA ***.crawler.adkernel.com # for IPv6
dig A ***.crawler.adkernel.com # for IPv4
dig AAAA ***.crawler.adkernel.com # for IPv6
host ***.crawler.adkernel.com
Ensure that the returned IP matches the original IP.
Note: Any mismatch between the domain and IP may indicate unauthorized traffic and such IP may not be associated with our bot.
