• We have redone the forum. If you notice any issues or errors please open a Support Ticket under the Members dropdown and let us know.
I

Not all bots are being detected

  • Thread starter Thread starter ivp
  • Start date Start date
I

ivp

Guest

XF\Data\Robot::getRobotUserAgents() detects 35 bots only.

This function should be replaced with CrawlerDetect or similar library, which detects 1,000's of bots/spiders/crawlers:
github.com

GitHub - JayBizzle/Crawler-Detect: 🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent - JayBizzle/Crawler-Detect
github.com
github.com

Here is the example for one thread:
  • Internal XenForo counter reports 519 views.
  • In server logs seeing 903 requests. After manually removing bots from the list, there are 355 views.
  • Google Analytics reports 167 views.

Continue reading...
 
Similar threads Most view View more
Back
Top