# Web crawler body\_content should not include the contents of the title tag

**URL:** <https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944>\
**Category:** Elastic Search\
**Tags:** elastic-app-search\
**Created:** [July 18, 2022, 8:06pm UTC](https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944 "2022-07-18T20:06:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Antonio\_Gutierrez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antonio_gutierrez/32/108479_2.png) [@Antonio\_Gutierrez](https://discuss.elastic.co/u/Antonio_Gutierrez)\
**Post date:** [July 18, 2022, 8:06pm UTC](https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944/1 "2022-07-18T20:06:06Z")

</div>

When using the web crawler feature, resulting documents have a body\_content that starts with the same string as the `<title>`. I have no idea why.

The documentation at [Web crawler reference | Elastic App Search Documentation [8.3] | Elastic](https://www.elastic.co/guide/en/app-search/8.3/web-crawler-reference.html) specifically says that body\_content comes from the `<body>` tag. Having the title appended screws up the relevancy of snippets when the search term appears in the title.

---

<div class="post-metadata">

**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\
**Post date:** [July 20, 2022, 12:38pm UTC](https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944/2 "2022-07-20T12:38:20Z")

</div>

Hi @Antonio_Gutierrez ! Could you share the URL or page source, so we can take a closer look?

Thanks!

---

<div class="post-metadata">

**Author:** ![Antonio\_Gutierrez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antonio_gutierrez/32/108479_2.png) [@Antonio\_Gutierrez](https://discuss.elastic.co/u/Antonio_Gutierrez)\
**Post date:** [July 20, 2022, 1:20pm UTC](https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944/3 "2022-07-20T13:20:50Z")

</div>

Open support case: #00994044  
Example:

 ![gatsby](https://us1.discourse-cdn.com/elastic/original/3X/9/b/9b441e46dba061056e7825b7f70d8b55e072124b.png)

Crawled doc:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/c/0c8838efcbbdfd8e15485e07f53bdd2adfb9eb52.png)

The title should not be in the body. The h2 should be, and is as expected.

---

<div class="post-metadata">

**Author:** ![Carlos\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carlos_d/32/126245_2.png) [@Carlos\_D](https://discuss.elastic.co/u/Carlos_D)\
**Post date:** [July 26, 2022, 9:47am UTC](https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944/4 "2022-07-26T09:47:55Z")

</div>

This has been confirmed as a bug, thanks for raising it!

We'll keep you updated on the fix version. Stay tuned!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 23, 2022, 9:48am UTC](https://discuss.elastic.co/t/web-crawler-body-content-should-not-include-the-contents-of-the-title-tag/309944/5 "2022-08-23T09:48:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
