# Logstash not recognizing the JSON objects from my logs and displays entire log set in message field

**URL:** <https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842>\
**Category:** Logstash\
**Created:** [January 3, 2019, 4:18pm UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842 "2019-01-03T16:18:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![utki](https://avatars.discourse-cdn.com/v4/letter/u/c57346/32.png) [@utki](https://discuss.elastic.co/u/utki)\
**Post date:** [January 3, 2019, 4:18pm UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842/1 "2019-01-03T16:18:39Z")

</div>

I need to send below logs using logstash and want to create a dashboard using request\_header\_x\_forwarded\_for and request\_username fields which are JSON objects

* * *

## Dec 26 00:32:50 Web1 mars-auth-capture: [INFO][mars][auth-capture]@cee: {"profile":"[http://cee.mitre.org/1.0-beta1/coreprofile","host":"1.2.3.4","pname":"mars-auth-capture","native":{"request\_timestamp":"12/26/18](http://cee.mitre.org/1.0-beta1/coreprofile%22,%22host%22:%221.2.3.4%22,%22pname%22:%22mars-auth-capture%22,%22native%22:%7B%22request_timestamp%22:%2212/26/18) 00:20:46","request\_protocol":"HTTP/1.1","request\_method":"POST","request\_header\_x\_forwarded\_for":"a.b.c.d","request\_username":"sample\_username","response\_header\_set\_cookie":"rdAc6opdpQoEmV4hnQogPM","response\_auth\_result":"error occurred"}}

I tried using JSON filter and Grok filter but every time my entire log set is coming in the message field and it's not being split.

Can someone help as I am new to ELK and need some guidance.

Can someone help?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 7, 2019, 9:08pm UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842/2 "2019-01-07T21:08:13Z")

</div>

I would use dissect to chop it up then a json filter.

```
filter {
        dissect {
        mapping => { "message" => "%{ts} %{+ts} %{+ts} %{a} %{b}: [%{level}][%{c}][%{d}@%{e}: %{msg}" }
        }
}
filter { json { source => "msg" } }
```

---

<div class="post-metadata">

**Author:** ![utki](https://avatars.discourse-cdn.com/v4/letter/u/c57346/32.png) [@utki](https://discuss.elastic.co/u/utki)\
**Post date:** [January 8, 2019, 10:01am UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842/3 "2019-01-08T10:01:31Z")

</div>

> [@Badger](#):
>
> "%{ts} %{+ts} %{+ts} %{a} %{b}: [%{level}][%{c}][%{d}@%{e}: %{msg}"

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![utki](https://avatars.discourse-cdn.com/v4/letter/u/c57346/32.png) [@utki](https://discuss.elastic.co/u/utki)\
**Post date:** [January 8, 2019, 12:34pm UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842/4 "2019-01-08T12:34:47Z")

</div>

Thanks a lot for your last response. I was able to successfully parse my logs the way they needed to be. Unfortunately the fields I need are tagged as type:text and I am not able to create any dashboard using them (since they are not visible in fields in Visualize Tab). What I understood from the web is that field should be of type keyword to be able to use them to create dashboards. Can you provide me the quick solution on how I can send my logs so that field i need are of type keyboard.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 8, 2019, 1:30pm UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842/5 "2019-01-08T13:30:19Z")

</div>

If elasticsearch is ingesting fields as text and you want them to be keyword then I think that is an elasticsearch question, not a logstash question.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 5, 2019, 1:30pm UTC](https://discuss.elastic.co/t/logstash-not-recognizing-the-json-objects-from-my-logs-and-displays-entire-log-set-in-message-field/162842/6 "2019-02-05T13:30:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.