# Webcrawler and googleoff:all tags

**URL:** <https://discuss.elastic.co/t/webcrawler-and-googleoff-all-tags/272572>\
**Category:** Elastic Search\
**Tags:** elastic-app-search\
**Created:** [May 10, 2021, 11:32am UTC](https://discuss.elastic.co/t/webcrawler-and-googleoff-all-tags/272572 "2021-05-10T11:32:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![TimTim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timtim/32/5295_2.png) [@TimTim](https://discuss.elastic.co/u/TimTim)\
**Post date:** [May 10, 2021, 11:32am UTC](https://discuss.elastic.co/t/webcrawler-and-googleoff-all-tags/272572/1 "2021-05-10T11:32:45Z")

</div>

Hi!

I'm crawling our site and it seems to be working fine. But the issue I see is that the content contains text that is outside the **!--googleon: all--** tag. The way we have our code is like this:

```
<!--googleoff: all-->
<header>.....
<div id="breadcrumbs">...
<!--googleon: all-->
<h1>....
<div id="content-primary">
...
</div>
<!--googleoff: all-->

```

When I look at the crawled documents, the content always start with the menu-links, breadcrumbs, .... and after that I get the content.

Is this a known issue?  
Or is this a setting I missed?

Regards,  
Tim

---

<div class="post-metadata">

**Author:** ![Vadim\_Yakhin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadim_yakhin/32/56723_2.png) [@Vadim\_Yakhin](https://discuss.elastic.co/u/Vadim_Yakhin)\
**Post date:** [May 10, 2021, 12:13pm UTC](https://discuss.elastic.co/t/webcrawler-and-googleoff-all-tags/272572/2 "2021-05-10T12:13:35Z")

</div>

Hey Tim!

We do not support google tags, and there are no immediate plans to support them.

But we have plans to add support for exclusion/inclusion tags.  
It's in our short-term roadmap, so stay tuned!

Best,  
Vadim

---

<div class="post-metadata">

**Author:** ![TimTim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timtim/32/5295_2.png) [@TimTim](https://discuss.elastic.co/u/TimTim)\
**Post date:** [May 10, 2021, 12:23pm UTC](https://discuss.elastic.co/t/webcrawler-and-googleoff-all-tags/272572/3 "2021-05-10T12:23:32Z")

</div>

OK, thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 7, 2021, 12:23pm UTC](https://discuss.elastic.co/t/webcrawler-and-googleoff-all-tags/272572/4 "2021-06-07T12:23:53Z")

</div>

Closed on June 7, 2021, 12:23pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.