Index pattern fields - disk space cost?

[vdelcampo (Víctor Gonzalez del Campo)]

Hi.

I´m using logstash to send to elastic all traffic that comes through my load balancer. With logstash, i´m parsing messages in this fields:

I parse the http_request field in other fields depending on the number of params it has: request_param, request_type, etc... When i see the index pattern, it has a lot of fields, lot of them are related with those possible request params. I´m not sure if having all this fields cost more disk space in spite not always all these fields are using becase there are to many different http_request values.

Regards,

Víctor

[warkolm (Mark Walkom)]

There is a cost, yes. If you have duplicated ones then you can remove them before sending to Elasticsearch.

[vdelcampo (Víctor Gonzalez del Campo)]

I mean not always all http_request fields are used on each document because there are many differents. In the index pattern fields there are no repetead ones. For example, in index pattern there are fields like these:

request_param1
request_param2
request_param3
request_param4

For one document, only uses request_param1 and request_param2, for the next one request_param1 and request_param4. For the first document, are all of them stored in spite of not being used?

Regards

Víctor

[warkolm (Mark Walkom)]

Those will be empty fields though, so you can ignore them.

[vdelcampo (Víctor Gonzalez del Campo)]

Thanks a lot Mark.

Regards