# Filebeat \_all field functionality

**URL:** <https://discuss.elastic.co/t/filebeat-all-field-functionality/232422>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 13, 2020, 11:41am UTC](https://discuss.elastic.co/t/filebeat-all-field-functionality/232422 "2020-05-13T11:41:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Richard\_Neely](https://avatars.discourse-cdn.com/v4/letter/r/90db22/32.png) [@Richard\_Neely](https://discuss.elastic.co/u/Richard_Neely)\
**Post date:** [May 13, 2020, 11:41am UTC](https://discuss.elastic.co/t/filebeat-all-field-functionality/232422/1 "2020-05-13T11:41:17Z")

</div>

When we were using version 5.6 of everything filebeat sent the raw unparsed json to logstash, which in turn parsed all the fields but also had a \_all/\_source field that had the entire document in there as a string that you could search.

This feature was removed in the later versions and now my setup with 7.6 filebeat breaks up the json before it even sends to logstash.

Now some of the devs are asking for that \_all field back somehow. The use case is they may add a field or tag something that won't be indexed/cached and they can't search for it without someone updating the mapping and refreshing the index in kibana. Dynamic mappings is out of the question as this grows until there's a mapping explosion.

I've read some on the copy\_to mapping but how would that work if everything is under msg.\* and encompassing any new fields that may get added?

---

<div class="post-metadata">

**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\
**Post date:** [June 1, 2020, 9:33pm UTC](https://discuss.elastic.co/t/filebeat-all-field-functionality/232422/2 "2020-06-01T21:33:13Z")

</div>

Hi Richard

I don't think there will be any comeback to the all field.

This background here may help as information:

> <https://github.com/elastic/kibana/issues/8007>
>
> We've opened an issue over on Elasticsearch about about disabling or removing the \_all field: elastic/elasticsearch#19784
> Kibana uses this field heavily through...

  

> <https://github.com/elastic/kibana/issues/10090>
>
> Related to #8007.
> elastic/elasticsearch#22144 disabled \_all in Elasticsearch by default. Furthermore, it is no longer going to be possible to configure this...

And for filebeat there is a great blog with some background information:

> **[Filebeat modules, access logs and Elasticsearch storage requirements](https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements)**
>
> In this blog post, we explore the on-disk storage requirements of data indexed using Filebeat modules and discuss further optimizations and related tradeoffs.

Now it would be helpful of what you are trying to accomplish with filebeat and regarding the json document to logstash.  
Can you give an actual example of what is not working for you any more?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 29, 2020, 9:33pm UTC](https://discuss.elastic.co/t/filebeat-all-field-functionality/232422/3 "2020-06-29T21:33:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.