Storing logs: 'message' field doubles index size?

[John16 (John)]

Hello,

I am using filebeat + logstash to store logs in ES.
When I browse my index data, I see structured fields my log lines are parsed into (date, http return code, request line, etc) as well as 'message' field which represents the whole log line itself.

So as far as I understand, every log line is stored twice: first time in structured format, and second time as a whole in 'message' field.

Why? Is it possible to get rid of this duplication and to trim that 'message' in order to save some disk space?

Thanks.

[Christian_Dahlqvist]

You can add a mutate filter and remove the message field in Logstash. If you would like to keep it, but do not need to be able to search it, you can alter the index template and set the message field to be not indexed, which probably will save a fair amount of space.