# Duplicate content with query parameters web crawler

**URL:** <https://discuss.elastic.co/t/duplicate-content-with-query-parameters-web-crawler/362222>\
**Category:** Elastic Search\
**Created:** [June 28, 2024, 3:58pm UTC](https://discuss.elastic.co/t/duplicate-content-with-query-parameters-web-crawler/362222 "2024-06-28T15:58:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![westonhancock](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/westonhancock/32/135655_2.png) [@westonhancock](https://discuss.elastic.co/u/westonhancock)\
**Post date:** [June 28, 2024, 3:58pm UTC](https://discuss.elastic.co/t/duplicate-content-with-query-parameters-web-crawler/362222/1 "2024-06-28T15:58:43Z")

</div>

We have an index of our site and are not allowing duplicate content. We are seeing multiple documents of the same exact page with the only difference being the query string of the url. The canonical urls are exactly the same and the page content is not dynamic based on the query string. This seems like a bug but maybe I'm missing something?

---

<div class="post-metadata">

**Author:** ![nfeekery](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nfeekery/32/117349_2.png) [@nfeekery](https://discuss.elastic.co/u/nfeekery)\
**Post date:** [July 3, 2024, 8:27am UTC](https://discuss.elastic.co/t/duplicate-content-with-query-parameters-web-crawler/362222/2 "2024-07-03T08:27:26Z")

</div>

Hi @westonhancock  
Are you using the Elastic Web Crawler on cloud? If so, I wonder if you're running into this known issue: [Elastic web crawler known issues | Enterprise Search documentation [8.14] | Elastic](https://www.elastic.co/guide/en/enterprise-search/current/crawler-known-issues.html) (down under `URLs being indexed despite having duplicate content and a canonical URL setting.`).  
Let me know if this seems applicable to your situation or not.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2024, 8:28am UTC](https://discuss.elastic.co/t/duplicate-content-with-query-parameters-web-crawler/362222/3 "2024-07-31T08:28:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.