# Incorrect mapping of fields by Kibanna send from logstash

**URL:** <https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 18, 2019, 6:25pm UTC](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336 "2019-06-18T18:25:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![asad\_ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asad_ali/32/47894_2.png) [@asad\_ali](https://discuss.elastic.co/u/asad_ali)\
**Post date:** [June 18, 2019, 6:25pm UTC](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336/1 "2019-06-18T18:25:57Z")

</div>

**Problem**  
I have very simple log file for testing purposes.

Below as:-

> ```
> 2008-09-15T11:30:00Z sarah
> 2008-09-15T12:18:00Z jessica
> 2008-09-15T13:20:00Z parker lee
> 
> ```

**On testing**

**stdout of logstash is**

> {  
> "type" =\> "log",  
> "input" =\> {  
> "type" =\> "log"  
> },  
> "tags" =\> [  
> [0] "beats\_input\_codec\_plain\_applied"  
> ],  
> "@timestamp" =\> 2008-09-15T11:30:00.000Z,  
> "ecs" =\> {  
> "version" =\> "1.0.0"  
> },  
> "host" =\> {  
> "name" =\> "elk"  
> },  
> "agent" =\> {  
> "ephemeral\_id" =\> "de724cef-e507-4b82-909d-700ab2f14f0c",  
> "type" =\> "filebeat",  
> "id" =\> "02437fe5-f069-405f-bd06-14b5501db678",  
> "version" =\> "7.1.1",  
> "hostname" =\> "elk"  
> },  
> "@version" =\> "1",  
> "log" =\> {  
> "file" =\> {  
> "path" =\> "/root/filebeat-7.1.1-linux-x86\_64/sample.log"  
> },  
> "offset" =\> 0  
> },  
> "line" =\> "sarah"  
> }  
> {  
> "type" =\> "log",  
> "input" =\> {  
> "type" =\> "log"  
> },  
> "tags" =\> [  
> [0] "beats\_input\_codec\_plain\_applied"  
> ],  
> "@timestamp" =\> 2008-09-15T12:18:00.000Z,  
> "ecs" =\> {  
> "version" =\> "1.0.0"  
> },  
> "host" =\> {  
> "name" =\> "elk"  
> },  
> "agent" =\> {  
> "ephemeral\_id" =\> "de724cef-e507-4b82-909d-700ab2f14f0c",  
> "type" =\> "filebeat",  
> "id" =\> "02437fe5-f069-405f-bd06-14b5501db678",  
> "version" =\> "7.1.1",  
> "hostname" =\> "elk"  
> },  
> "@version" =\> "1",  
> "log" =\> {  
> "file" =\> {  
> "path" =\> "/root/filebeat-7.1.1-linux-x86\_64/sample.log"  
> },  
> "offset" =\> 27  
> },  
> "line" =\> "jessica"  
> }  
> {  
> "type" =\> "log",  
> "input" =\> {  
> "type" =\> "log"  
> },  
> "tags" =\> [  
> [0] "beats\_input\_codec\_plain\_applied"  
> ],  
> "@timestamp" =\> 2008-09-15T13:20:00.000Z,  
> "ecs" =\> {  
> "version" =\> "1.0.0"  
> },  
> "host" =\> {  
> "name" =\> "elk"  
> },  
> "agent" =\> {  
> "ephemeral\_id" =\> "de724cef-e507-4b82-909d-700ab2f14f0c",  
> "type" =\> "filebeat",  
> "id" =\> "02437fe5-f069-405f-bd06-14b5501db678",  
> "version" =\> "7.1.1",  
> "hostname" =\> "elk"  
> },  
> "@version" =\> "1",  
> "log" =\> {  
> "file" =\> {  
> "path" =\> "/root/filebeat-7.1.1-linux-x86\_64/sample.log"  
> },  
> "offset" =\> 56  
> },  
> "line" =\> "parker lee"  
> }

> curl -XGET 'localhost:9200/logstash-2015.01.04/\_search?pretty&q=response=200'

```
{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : []
  }
}

```

> curl '[http://localhost:9200/\_search?pretty](http://localhost:9200/_search?pretty)'

output see file

> **[Filebin | hdnbfbhn0xtrzitp](https://filebin.net/hdnbfbhn0xtrzitp)**
>
> Convenient file sharing. Think of it as Pastebin for files. Registration is not required. Large files are supported.

* * *

**My expectation, is that beside timestamp and line it should not match or add other fields the other fields its matching is more then 30 given below and also shown in complete file in link above**

> ` "fields" : """[{"name":"@timestamp","type":"date","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true},{"name":"@version","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true},{"name":"\_id","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":false},{"name":"\_index","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":false},{"name":"\_score","type":"number","count":0,"scripted":false,"searchable":false,"aggregatable":false,"readFromDocValues":false},{"name":"\_source","type":"\_source","count":0,"scripted":false,"searchable":false,"aggregatable":false,"readFromDocValues":false},{"name":"\_type","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":false},{"name":"agent.ephemeral\_id","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},{"name":"agent.ephemeral\_id.keyword","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true},{"name":"agent.hostname","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},{"name":"agent.hostname.keyword","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true},{"name":"agent.id","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},{"name":"agent.id.keyword","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true},{"name":"agent.type","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},{"name":"agent.type.keyword","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true},{"name":"agent.version","type":"string","count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},...........

* * *

**On elastic-search logs I get**

[2019-06-18T03:34:21,099][DEBUG][o.e.x.s.a.a.OptOutQueryCache] [elk] [.kibana\_task\_manager] not opting out of the query cache; authorization is not allowed.

---

<div class="post-metadata">

**Author:** ![asad\_ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asad_ali/32/47894_2.png) [@asad\_ali](https://discuss.elastic.co/u/asad_ali)\
**Post date:** [June 18, 2019, 6:36pm UTC](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336/2 "2019-06-18T18:36:21Z")

</div>

Logstash.yml

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /root/filebeat-7.1.1-linux-x86_64/sample.log
  fields_under_root: true
  fields:
    type: log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
output.logstash:
 hosts: ["localhost:5044"]
processors:
```

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [June 19, 2019, 12:00pm UTC](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336/3 "2019-06-19T12:00:43Z")

</div>

Hi @asad_ali 🙂

I'm not sure about the problem itself. Are you having issues with Logstash or with Filebeat? That `Logstash.yml` looks like a Filebeat config, not Logstash.

---

<div class="post-metadata">

**Author:** ![sjabiulla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sjabiulla/32/48429_2.png) [@sjabiulla](https://discuss.elastic.co/u/sjabiulla)\
**Post date:** [June 20, 2019, 9:58am UTC](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336/4 "2019-06-20T09:58:28Z")

</div>

please post your `Logstash.yml`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2019, 9:58am UTC](https://discuss.elastic.co/t/incorrect-mapping-of-fields-by-kibanna-send-from-logstash/186336/5 "2019-07-18T09:58:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.