# Log entry disappears between Logstash and Kibana

**URL:** <https://discuss.elastic.co/t/log-entry-disappears-between-logstash-and-kibana/225850>\
**Category:** Kibana\
**Created:** [March 31, 2020, 12:05pm UTC](https://discuss.elastic.co/t/log-entry-disappears-between-logstash-and-kibana/225850 "2020-03-31T12:05:27Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![Axl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/axl/32/45475_2.png) [@Axl](https://discuss.elastic.co/u/Axl)\
**Post date:** [April 3, 2020, 2:20pm UTC](https://discuss.elastic.co/t/log-entry-disappears-between-logstash-and-kibana/225850/7 "2020-04-03T14:20:09Z")

</div>

I solved this. It was a _Logstash_ config issue.

In the logs of ElasticSearch (see below) I noticed the exception was occurring on field `input`, which was precisely one of the fields I was expecting to see in my "disappearing" logs.

```auto
[2020-04-03T09:53:37,718][DEBUG][o.e.a.b.TransportShardBulkAction] [instance-42] [honeypot-logstash-2020.04.03-000001][0] failed to execute bulk item (index) index {[honeypot-logstash][_doc][mo91P3EBYpA6orC0z2C2], source[_na_]}
org.elasticsearch.index.mapper.MapperParsingException: object mapping for [input] tried to parse field [input] as object, but found a concrete value

```

Then, in Logstash, I noticed this type of log, with the same error message:

```auto
Mar 31 05:44:14 instance-42 logstash[22592]: [2020-03-31T03:44:14,883][WARN][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"cowrie-logstash", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x2082e9af>], :response=>{"index"=>{"_index"=>"cowrie-logstash-2020.03.20-000001", "_type"=>"_doc", "_id"=>"Go-wLnEBYpA6orC0jiYT", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [input] tried to parse field [input] as object, but found a concrete value"}}}}

```

So, I had a look at my entries and found out that in some cases, I had `input` field with sublayers

```auto
"input" => {
        "type" => "log"
    },

```

and in other cases, directly a string.

```auto
"command" => "echo \"this is my test\""

```

I searched for the problem (in particular [this thread](https://discuss.elastic.co/t/problem-with-transfer-filebeat-6-1-3-logstash-6-1-3-elasticsearch-6-1-3/136264/7)), and found out that **this is a situation where Logstash is lost at types: in the first case, it expect an object, and it does not like in the second case to find a string**.

**Solution** : I need to **either always to have an object, or always a string, but not mix.**

The fix will depend on your log structure, in my case (`cowrie`), I was doing

```auto
json {
 source => "message"
}

```

which would read the JSON input from field `message` and create log structure from that. In my case, that was overwriting several fields such as `input`. The solution is to specify a target, so that fields from the JSON input don't overwrite other fields, and for instance, they'd go in `honeypot.input`, not `input`.

```auto
json {
 source => "message"
 target => "honeypot"
}

```

---

_[View the full topic](https://discuss.elastic.co/t/log-entry-disappears-between-logstash-and-kibana/225850)._