# How does Logstash parsing the below events into correct json data?

**URL:** <https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985>\
**Category:** Logstash\
**Created:** [January 28, 2019, 10:59am UTC](https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985 "2019-01-28T10:59:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [January 28, 2019, 10:59am UTC](https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985/1 "2019-01-28T10:59:41Z")

</div>

Note: the raw PHP larval log is consist of JSON object format text and non-JSON format text

**Original PHP larval log:**

```auto
[2019-01-28 02:28:22] prod.INFO: {"database":"test10","type":"form","id":"20821","comment":"passback:792","timestamp":"2019-01-28 02:28:22"}

```

**log is collected by filebeat ,output to logstash and logstash output to local file**

```auto
{
  "info": "{\"database\":\"test10\",\"type\":\"form\",\"id\":\"19860\",\"comment\":\"passback:792\",\"timestamp\":\"2019-01-28 02:27:39\"}"
}

```

**The output I wanted**

```auto
{"info":{"database":"test10","type":"form","id":"19860","comment":"passback:792","timestamp":"2019-01-28 02:27:39"}}

```

**logstash conf:**

```auto
#logstash for pipeline filebeat
input {
  beats {
   port => 5058
    }
}

filter {

  if [service] =~ "datapurge-log" {
     grok {
       match => ["message", "%{UNUSED_TIME:unused_time} %{UNUSED_TYPE:unused_type}\: %{INFO:info}"]
     }
     mutate {
       remove_field=> ["[beat][version]","[beat][hostname]","[host][name]","[input][type]","[prospector][type]","[offset]","[message]"],
       gsub => ["info", "[\\]", ""]
     }

   }
}

output {
   if "_grokparsefailure" not in [tags] {
     file {
       path => "/tmp/datapurge-log"
     }
   }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 28, 2019, 2:32pm UTC](https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985/2 "2019-01-28T14:32:00Z")

</div>

I would suggest chopping up the message using a dissect filter, then using a json filter to parse it.

```
dissect { mapping => { "message" => "[%{} %{}] %{}: %{msg}" } }
```

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [January 30, 2019, 3:00am UTC](https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985/3 "2019-01-30T03:00:04Z")

</div>

Hi @Badger,

we ship logs like this: **filebeat-\>logstash-\>ES**

when the Original PHP larval log is collected by filebeat, the event is added to the key **message**. and all hash within the JSON object was added the escape character **\**.  
my requirement is that remove strings both **[2019-01-28 02:28:22] prod.INFO:** and escape character **\** in the hash of the whole JSON object.

PS:

Before collecting:

> ```auto
> [2019-01-28 02:28:22] prod.INFO: {"database":"test10","type":"form","id":"20821","comment":"passback:792","timestamp":"2019-01-28 02:28:22"}
> 
> ```

After collecting：

```auto
{
 "message": "[2019-01-28 02:28:22] prod.INFO: "{\"database\":\"test10\",\"type\":\"form\",\"id\":\"19860\",\"comment\":\"passback:792\",\"timestamp\":\"2019-01-28 02:27:39\"}"}
}

```

Any help or Suggestion would be appreciate！

Thanks in advance

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 1:37pm UTC](https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985/4 "2019-01-30T13:37:32Z")

</div>

As I said, use the dissect filter that I showed to parse the message, then use a json filter to parse the field that contains the JSON.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 27, 2019, 1:37pm UTC](https://discuss.elastic.co/t/how-does-logstash-parsing-the-below-events-into-correct-json-data/165985/5 "2019-02-27T13:37:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.