We have an ELK-stack for our centralized logging, and we keep logs for about 3 weeks. We use dynamic mapping and over time our field list has grown quite big. Currently the index has over 2000 fields defined.
At one point we experienced that log messages were dropped, and figured out that it was because we hit the field limit in elastic search. So we increased that in order to accept our new log messages. But now we experience that our visualizations that use term aggregation don't work any more (we get "no results found"), and we wonder if this can have anything with the field limit-adjustment?
I think that many of the fields are not in use any more, since we have shut down some of the application logging. Is there a way to clean up / reduce the number of fields defined? Is it easy to figure out which fields are not in use any more?