Logstash Slow after small config change


(Allie Yang Yang) #1

Hi I have changed only one place in my log config file, by adding one field into the prune's whitelist.
Then, the logstash parses super slowly, 10X roughly. Also, the added variable does not show properly in Kibana: no cached mapping for this field.

My question is if I change the config file like this, is this the normal reaction? If so, should I rename it to a new index pattern?

PS: i also tried put mapping for this new field, but it doesn't allow me, Error is exceed 1000 field limit for each index.


(Christian Dahlqvist) #2

Please show us the config file and the change you made as that may make it easier to help. It does however sound like the issue is with the large number of fields in your mapping and that you bare hitting the limit Elasticsearch imposes in order to prevent mapping explosion. If you have a lot of different types of data in the same index, it may make sense to split them up into a few indices and thereby reduce the number of fields per index. If you have some type of data that dynamically creates field names you should look into addressing that.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.