Product Not Showing Up

Why Is a Product Not Showing Up in Search?

Normally, when a product does not appear in the search, it may be due to one of these reasons:

• It is not indexed in the feed. The product may not be included in the feed because it was added after the last indexing, so it won't appear in the search results. If you integrated Doofinder in your site with one of the official plugins, they should automatically update search indices for you when there's a change in your catalog.

  • Get your products website link > look for your products details such as SKU, MPN, title, etc > with that info, search for the products under results preview, where you can test your Search Engine.
  • If not showing, make sure to check if the product is in your feed or available on your backend reindex.
  • If showing there, make sure you are searching for the right information in the search layer. Otherwise, let us know with examples.

• Product is out of stock. Check if your Search Engine is configured to exclude out of stock products from search results, read our "Out of stock items" article for more information. If your Search Engine is not configured to exclude out of stock products, go to Configuration > Excluded Results > Add results > Rules > Availability is out of stock to exclude them. Notice that this will affect all services.

• Check if the product is excluded from the search, go to Configuration > Excluded Results; and, check if the product is affected by some kind of visibility rule in your site like entirely hidden categories, or similar. Nonetheless, if you are using a data feed to index products, check if the product appears in the feed.

• A query that should display the product is matching a custom result. Use the search feature Results Preview to verify if that's the case. If so, you can check the configuration of that custom result at Search > under Promotional Tools > Custom Results. Learn more about Custom Results here.

• There was a temporary problem with your site or the indexing service. In this case, a quick and easy solution is going to your site's admin and saving again the product so it triggers the required updates. In case you see an error message in the indexing log, try to fix the problem if it happens to be on your side. You will find an article explaining the most common errors here. If you need assistance, contact us via our contact form and we'll help you.

• The data feed was not entirely read. If the feed comes in CSV format, and the transmission was interrupted, the feed may have partially indexed. This does not happen, for instance, with XML feeds, which have more strict syntax requirements.

• If you’re using any of our modules, please update to the latest version and then reindex the feed again: Shopify, Prestashop, Magento, BigCommerce, Woocommerce and Shopware 5 and Shopware 6.

• A missing or outdated product is very often caused by a cached copy of your feed being served instead of the live file, either on your own server or on a CDN/security layer in front of your site. See Caching Layers (Cloudflare, Sucuri, and Others) below to confirm and fix it.

Remember, if your catalog is correct, just re-index the data feed manually in your Doofinder Admin Panel.

Caching Layers (Cloudflare, Sucuri, and Others)

A caching or security layer in front of your website can serve Doofinder an outdated copy of your feed, even after you update your catalog. When this happens, your feed file is correct on your server, but the version Doofinder downloads is an old, stored copy, so your changes never reach search.

When to suspect that your feed is cached

• You add or update a product, it appears correctly when you open your feed URL in a browser, but it never shows up in Doofinder.
• The indexing log reports that the feed has no changes, even though you know you changed it.
• Everything worked for a long time and the problem started suddenly.

How to check whether your feed is cached

You can inspect the response headers of your feed URL to see if a cache is serving a stored copy.

1. Open your feed URL in a browser, open the developer tools (Network tab), reload, click the request, and look at the Response Headers. Alternatively, run curl -I "YOUR-FEED-URL" from a terminal.
2. Look for headers that indicate the response is being served from a cache:
   • cf-cache-status: HIT, Cloudflare is serving a cached copy.
   • x-sucuri-cache: HIT, Sucuri is serving a cached copy.
   • x-cache: HIT, a CDN or proxy (Fastly, Varnish, and others) is serving a cached copy.
   • cache-control: max-age=<large number>, the file is stored for a long time before it is refreshed (for example, max-age=315360000 keeps the same copy for up to 10 years).

A HIT means you are being served a stored copy, not the live file from your server.

How to fix it

1. Purge the cache for the exact feed URL (only the feed path, not necessarily your whole site).
2. Reduce the cache duration (max-age / TTL) for the feed file so it refreshes at least as often as your indexing runs. A very large max-age will keep serving the same file for that entire period.
3. If you cannot reduce the cache for that URL, serve the feed from a subdomain or path that is excluded from caching.
4. After purging, trigger a manual reindex so Doofinder fetches the fresh file: Configuration > Search Engines > See indices > Indices > Process Now.

Notice that these layers are usually managed by your hosting provider, security provider, or web agency. If you are not sure whether you have one, ask them, or check the headers as described above.

Best Practices & caching considerations

If you use Cloudflare or another caching system to reduce traffic to your website, also keep in mind:

• Offload the feed URLs to a subdomain that is not cached.
• Ensure nothing times out, giving enough execution time to your web server (NGINX, Apache, etc.); this depends on the number of products in your catalog. Cloud load balancers might also time out.
• Allowlist the Doofinder IPs. See here!

Products Not Updating As Expected

If your products are not showing up as expected after indexing, here are some steps to check:

1. Check the Indexing Logs: Go to your Doofinder Admin Panel and navigate to Configuration > Search Engines > click on "See indices" > Indices. Look for any error messages in the indexing logs that might indicate what went wrong.
2. Reindex Manually: If everything seems fine in the logs, try reindexing your products manually. Go to Configuration > Search Engines > click on "See indices" > Indices and click the “Process Now” button to trigger a reindex.
3. Verify Data Feed: Ensure that your data feed is correctly set up and that it includes all the necessary product information. If you’re using a static data feed, make sure you have uploaded the most recent file.
4. Check for Out of Stock products: If your Search Engine is configured to exclude out of stock products, ensure that the products you are trying to update are in stock.
5. Automatic Indexing: If you have automatic indexing enabled, check if it has been disabled due to repeated failures. You can re-enable it from the same Indices page.

If you continue to experience issues, contact us through our contact form for further assistance.

Unexpected Results?

See here info about getting Unexpected Results and how to test your Search Engine, both documentations can help you understanding why your product is missing.

Index Products "Manually"

If you have coding knowledge you can manually index/update products programmatically with our Management API.

Notice you don't need that if you are using any of our plugins in your ecommerce because they usually allow automatic updates when you save your products in your site. The feature may appear in the plugin as update on save, automatic updates, or the like.

Remember you can always reindex your entire product catalog manually but without any coding. Just go to your Search Engine's Indices settings page in the Admin Panel > Configuration > Search Engines > click on "See indices" > Indices and press the "Process Now" button.

You might also find it interesting to check our Wrong Prices in Search Results documentation.