Filebeat _all field functionality

When we were using version 5.6 of everything filebeat sent the raw unparsed json to logstash, which in turn parsed all the fields but also had a _all/_source field that had the entire document in there as a string that you could search.

This feature was removed in the later versions and now my setup with 7.6 filebeat breaks up the json before it even sends to logstash.

Now some of the devs are asking for that _all field back somehow. The use case is they may add a field or tag something that won't be indexed/cached and they can't search for it without someone updating the mapping and refreshing the index in kibana. Dynamic mappings is out of the question as this grows until there's a mapping explosion.

I've read some on the copy_to mapping but how would that work if everything is under msg.* and encompassing any new fields that may get added?

Hi Richard

I don't think there will be any comeback to the all field.

This background here may help as information:

And for filebeat there is a great blog with some background information:

Now it would be helpful of what you are trying to accomplish with filebeat and regarding the json document to logstash.
Can you give an actual example of what is not working for you any more?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.