Index pattern fields - disk space cost?


I´m using logstash to send to elastic all traffic that comes through my load balancer. With logstash, i´m parsing messages in this fields:

I parse the http_request field in other fields depending on the number of params it has: request_param, request_type, etc... When i see the index pattern, it has a lot of fields, lot of them are related with those possible request params. I´m not sure if having all this fields cost more disk space in spite not always all these fields are using becase there are to many different http_request values.



There is a cost, yes. If you have duplicated ones then you can remove them before sending to Elasticsearch.

I mean not always all http_request fields are used on each document because there are many differents. In the index pattern fields there are no repetead ones. For example, in index pattern there are fields like these:


For one document, only uses request_param1 and request_param2, for the next one request_param1 and request_param4. For the first document, are all of them stored in spite of not being used?



Those will be empty fields though, so you can ignore them.

Thanks a lot Mark.


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.