How can I prevent bot attacks?
What is a bot?
A bot is an automated program that browses websites without human interaction. In this context, bots are relevant because they can indirectly impact your Doofinder usage.
Why is it important to check bot activity?
When bots visit your website, they may trigger search requests that count toward your Doofinder consumption, even though no real user is behind them. It is important to note that Doofinder does not suffer from bot attacks directly, but instead, from indirect attacks to your web.
This guide explains how you can check for potential bot activity on your website.
Bots can attack through your internal or native search. When a bot searches your site, it sends those searches to Doofinder, causing an increase in consumption. This happens because of how your plugin is configured to replace your site's internal search with Doofinder. When is this applicable?
- If you have a deprecated plugin that supports that functionality (currently, none of our up to date modules manages internal search through API).
- If you have some configuration via API (custom implementation).
Bots usually trigger keyword searches to your server to obtain information on specific product availability, prices, etc. Hence, they usually spam your search results page.
How do I know I have bots?
Sudden spikes in Doofinder requests volume, 0% clicks through rate or long lists of random keywords and SKUs found on your search stats, might be a sign of a bot attack.
Need more details?
Check the Doofinder logs and get the details of the Doofinder search requests. Bots will most likely repeatedly make the same search or random searches with numbers and symbols. See here how to download your logs.
Specific IPs can be blocked to prevent access to our services. Check the How to Block IPs section above in this documentation.
How to prevent bot attacks?
Doofinder can't prevent bots from reaching and attacking your server, consider (a web security service or firewall) to prevent bots from communicating with your servers. However, you can take actions directly from Doofinder to avoid consuming and affecting your Doofinder analytics from bot attacks. Here's how:
- Maintain the Doofinder plugin up to date, in case of plugin integrations.
- Maintain the Doofinder script up to date, to know if you are using the latest version see here.
- Deactivate CORS from the security section of the Doofinder admin.
- In case of custom API integration, maintain the API integration up to date.
- In case of custom API integration, your server will be the one communicating with Doofinder directly. In those cases, we won't be able to block the original bot IP or prevent them from consuming your requests, you will need to ensure bots can't access your website.
Working with the Doofinder plugin integration?
If you’re using one of our modules below, make sure it's up to date:
Shopify - Module Update
Prestashop - Module Update
Shopware 5 - Module Update
Shopware 6 - Module Update
Magento - Module Update
JTL - Module Update
Woocommerce - Module Update
Bigcommerce - Module Update
Maintaining an up-to-date module is essential for maximizing its effectiveness in countering bot activity.
Key Takeaways
- Doofinder Live Layer V9 / V6 API does not suffer from bot attacks.
- Bots attacking your website will not reach Doofinder directly but indirectly through your server.
- Keep the script, modules and plugins updated to prevent vulnerabilities.
- Always check analytics in your stats to verify legitimate usage.
My website requires access credentials
You can include the access credentials in the URL of the feed so that it can be indexed, for example:
http: // usr: psw @ server / Tests
You can also include our IPs so that those credentials are not required. Our IPs:
Europe: 54.171.4.216
USA: 52.2.218.41
Asia: 18.143.220.25
What's doofinder user agent?
Doofinder/4.0 (+http://www.doofinder.com)
Which are the Doofinder IPs?
The Doofinder IPs are:
Europe: 54.171.4.216
USA: 52.2.218.41
Asia: 18.143.220.25
Allowlisting Doofinder's IP
Doofinder IP addresses must be allowlisted to ensure our products function correctly. Please add our IPs to your server and firewall systems, such as Cloudflare.
Our integrations communicates server-to-server, and security systems verify the sender’s IP address. If the IP is not allowlisted in firewalls or services such as Cloudflare, requests may be blocked even if authentication credentials are valid.
Doofinder and the Data Usage
In an effort to respect user privacy and comply with current regulations, Doofinder has chosen to use LocalStorage instead of cookies for storing relevant data on the user’s browser. To know more, click here.