# Incorrect mapping not matching grok filers and fields .conf file **URL:** \ **Category:** Logstash\ **Created:** [June 18, 2019, 2:48pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288 "2019-06-18T14:48:10Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![asad\_ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asad_ali/32/47894_2.png) [@asad\_ali](https://discuss.elastic.co/u/asad_ali)\ **Post date:** [June 18, 2019, 2:48pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288/1 "2019-06-18T14:48:10Z")
Hey folks, I have .conf as ``` input { beats { port => 5044} } filter { grok { match => [ "message", "%{TIMESTAMP_ISO8601:timestamp_sting}%{SPACE}%{GREEDYDATA:line}" ] } date { match => ["timestamp_sting", "ISO8601"] } mutate { remove_field => [message, timestamp_sting] } } output { elasticsearch { hosts => ["http://localhost:9200"] index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" user => "elastic" password => "changeme" } stdout { codec => rubydebug } } ``` Output of ``` GET /filebeat-7.1.1-2008.09.15/_search { "query": { "match_all" :{} } } ``` is link [http://ge.tt/9lhWdfw2](http://ge.tt/9lhWdfw2) file name > [Logstash\_output.txt] **/root/logstash-7.1.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/elasticsearch-template-es7x.json** { "index\_patterns" : "logstash-_", "version" : 60001, "settings" : { "index.refresh\_interval" : "5s", "number\_of\_shards": 1 }, "mappings" : { "default" : { "\_all" : { "enabled" : false }, "dynamic\_templates" : [ { "message\_field" : { "path\_match" : "message", "match\_mapping\_type" : "string", "mapping" : { "type" : "text", "norms" : false } } }, { "string\_fields" : { "match" : "_", "match\_mapping\_type" : "string", "mapping" : { "type" : "text", "norms" : false, "fields" : { "keyword" : { "type": "keyword", "ignore\_above": 256 } } } } } ], "properties" : { "@timestamp": { "type": "date"}, "@version": { "type": "keyword"}, "geoip" : { "dynamic": true, "properties" : { "ip": { "type": "ip" }, "location" : { "type" : "geo\_point" }, "latitude" : { "type" : "half\_float" }, "longitude" : { "type" : "half\_float" } } } } } } } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 18, 2019, 3:29pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288/2 "2019-06-18T15:29:29Z")
Did you have a question? ---
**Author:** ![asad\_ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asad_ali/32/47894_2.png) [@asad\_ali](https://discuss.elastic.co/u/asad_ali)\ **Post date:** [June 18, 2019, 3:52pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288/3 "2019-06-18T15:52:19Z")
The problem is that the mapping or parsing is not done according to the format defined under logstash .conf file, when I get the index pattern its is extracting fields which \>50 are not defined by me, but some default template which it is using. I have tried running logstash with stdin /stdout as cli and it works great , but when stdin is from file-beat and destination as elasticsearch the mapping is mixed up. Kindly assist. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 18, 2019, 4:17pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288/4 "2019-06-18T16:17:14Z")
This is an issue with your filebeat configuration. For example the [host] object is added by the [host metadata processor](https://www.elastic.co/guide/en/beats/filebeat/current/add-host-metadata.html). If you are unable to figure out which processors are adding which fields you should ask a question in the filebeat forum, including an example of a document from elasticsearch. ---
**Author:** ![asad\_ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asad_ali/32/47894_2.png) [@asad\_ali](https://discuss.elastic.co/u/asad_ali)\ **Post date:** [June 18, 2019, 6:28pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288/5 "2019-06-18T18:28:30Z")
I tried to remove the host.metadata not extra fields are dropped down to 34 from 51. I opened new link here > [@Incorrect mapping of fields by Kibanna send from logstash](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-to-logstash/186336): > > Problem I have very simple log file for testing purposes. Below as:- 2008-09-15T11:30:00Z sarah 2008-09-15T12:18:00Z jessica 2008-09-15T13:20:00Z parker lee On testing stdout of logstash is { "type" =\> "log", "input" =\> { "type" =\> "log" }, "tags" =\> [[0] "beats\_input\_codec\_plain\_applied" ], "@timestamp" =\> 2008-09-15T11:30:00.000Z, "ecs" =\> { "version" =\> "1.0.0" }, "host" =\> { "name" =\> "elk" }, "agent" =\> { "ephemeral\_id" =\> "de724cef-e507-4b82-909d-700ab2f14f0c", "… ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 16, 2019, 6:28pm UTC](https://discuss.elastic.co/t/incorrect-mapping-not-matching-grok-filers-and-fields-conf-file/186288/6 "2019-07-16T18:28:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.