Elasticsearch false mapping

Elastic Stack Elasticsearch
1

Hello everyone and thanks for help.

I've installed latest versions of elasticsearch, kibana and logstash (8.11.1) on test cluster. Next, created new simple logstash pipeline that listens tcp port, next send data to elasticsearch with simple ingest pipeline with 1 JSON processor
Everything seems alright, except 1 problem - mapping of index. It must contains about 50 fields, but somehow elasticsearch with it dynamic mapping added ALL of possible fields (about 1200+), that never presents in this index (data).
On elasticsearch cluster with 8.4 version this never happens, it dynamically added only fields that exists in data. Please help, what I'm doing wrong?
1277

2

About which index are you talking about?
filebeat*, logstash* or a dedicated created?

3

Do you have any index template that matches the name of the index that might be adding these fields?

1 Like
4

Index is dedicated created called kata2-*

5

Yes, I have auto-generated index template called same as index (kata2) which contains multiple fields that not used in data in my index. It looks similar to prod-cluster.
Perhaps now I understand my problem. Mapping is incorrect because of my settings in both versions, only difference is how kibana show fields (in Discover). 8.4 show only fields that contain data, 8.11 show everything that exists in mapping. Is there any way that helps me to show only fields that contain data in 8.11 ? And maybe there are exists template for auto-generated index templates? Thank you!

6

That sounds like a Kibana question, so I don't think I will be able to help with that.

7

Thanks for answers!
Find solution for me. Simply create empty index template before adding new logstash pipeline makes everything correct. Zero unnecesary fields, zero problems

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.