# Logstash parse mixed json text

**URL:** <https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362>\
**Category:** Logstash\
**Created:** [June 16, 2022, 8:36am UTC](https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362 "2022-06-16T08:36:23Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Saathvi](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@Saathvi](https://discuss.elastic.co/u/Saathvi)\
**Post date:** [June 16, 2022, 8:36am UTC](https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362/1 "2022-06-16T08:36:23Z")

</div>

please can any one help me to parse below mixed json txt from below logentry.  
need to extract request json fields & response json fields

2020-01-06 01:02:10.869 +01:00 [Information] WIN HTTP Response at 2020-01-06T01:02:10.8598940+01:00. Response Time 2041.7975 ms.  
Request:  
{  
"Method": "GET",  
"Url": "http:\[dummyurl.com](http://dummyurl.com)",  
"Headers": {  
"Cache-Control": [  
"no-cache, no-store, max-age=0"  
],  
"Connection": [  
"close"  
],  
"Pragma": [  
"no-cache"  
],  
"Via": [  
"1 HTTP/1.1 :455"  
],  
"Accept": [  
"application/json, text/plain, _/_"  
],  
"Accept-Language": [  
"en-US,en;q=0.9"  
],  
"token": [  
"xxxxx"  
],  
"Cookie": [  
"XXXXX"  
],  
"Host": [  
"[test.com](http://test.com)"  
],  
"If-Modified-Since": [  
"Mon, 20 Jul 1995 05:00:00 GMT"  
],  
"Referer": [  
"[http://dummy.com/test](http://dummy.com/test)"  
],  
"User-Agent": [  
"Safari/537.36"  
],  
"iv-genera": [  
"1R"  
],  
"groups": [  
""USERS""  
],  
"remote-address": [  
"1.X.1.3"  
],  
"user": [  
"123456789"  
],  
"sec-ch-ua-mobile": [  
"?0"  
],  
"true-client-ip": [  
"1.1.1.2"  
],  
"sec-fetch-site": [  
"same-origin"  
],  
"server\_name": [  
"test123"  
],  
"sec-fetch-dest": [  
"empty"  
],  
"trace": [  
"ywdwdahda"  
]  
},  
"Body": ""  
}  
Response:  
{  
"StatusCode": 200,  
"Headers": {  
"Content-Type": [  
"text/json"  
],  
"test-val": [  
"1"  
],  
"Set-Cookie": [  
"HttpOnly"  
],

```
"test-Control": [
  "test"
]

```

},  
"Body": ""  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 16, 2022, 4:34pm UTC](https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362/2 "2022-06-16T16:34:41Z")

</div>

Assuming your JSON is valid (it is not, as shown) then you could use a multiline codec to put together all the lines of one message. Perhaps

```
codec => multiline { pattern => "^\d{4}-\d{2}-\d{2} " negate => true what => previous auto_flush_interval => 2 multiline_tag => "" }

```

Then

```
    dissect { mapping => { "message" => "%{[@metadata][timestamp]} %{+[@metadata][timestamp]} %{+[@metadata][timestamp]} [" } }
    if [@metadata][timestamp] { date { match => ["[@metadata][timestamp]", "YYYY-MM-dd HH:mm:ss.SSS ZZ" ] } }

    if "HTTP Response" in [message] {
        grok { match => { "message" => "Request:%{GREEDYDATA:[@metadata][request]}Response:%{GREEDYDATA:[@metadata][response]}" } }
        if [@metadata][request] { json { source => "[@metadata][request]" target => "httpRequest" } }
        if [@metadata][response] { json { source => "[@metadata][response]" target => "httpResponse" } }
        # If you never have more than one value for a header then you could also...
        ruby {
            code => '
                reqHdr = event.get("[httpRequest][Headers]")
                if reqHdr.is_a? Hash
                    reqHdr.each { |k, v|
                        event.set("[httpRequest][Headers][#{k}]", v[0])
                    }
                end
                respHdr = event.get("[httpResponse][Headers]")
                if respHdr.is_a? Hash
                    respHdr.each { |k, v|
                        event.set("[httpResponse][Headers][#{k}]", v[0])
                    }
                end
            '
        }
    }

```

which will produce

```
"httpResponse" => {
       "Headers" => {
            "test-val" => "1",
          "Set-Cookie" => "HttpOnly",
        "test-Control" => "test",
        "Content-Type" => "text/json"
    },
    "StatusCode" => 200,
          "Body" => ""
},
 "httpRequest" => {
    "Headers" => {
                   "Cookie" => "XXXXX",
              "server_name" => "test123",
           "sec-fetch-site" => "same-origin",
                   "Accept" => "application/json, text/plain, /",
     ...
```

---

<div class="post-metadata">

**Author:** ![Saathvi](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@Saathvi](https://discuss.elastic.co/u/Saathvi)\
**Post date:** [June 17, 2022, 11:30am UTC](https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362/4 "2022-06-17T11:30:03Z")

</div>

thanks Badger - solution is working..

my next step is to put the json data into database table..  
for example extract data of the fields Authorization , Referrer and iv-user and insert into database table. please could you help

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 17, 2022, 5:21pm UTC](https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362/5 "2022-06-17T17:21:31Z")

</div>

> [@Saathvi](#):
>
> my next step is to put the json data into database table..

I do not understand what you mean by that. In any case, it should be a new topic.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 15, 2022, 5:22pm UTC](https://discuss.elastic.co/t/logstash-parse-mixed-json-text/307362/6 "2022-07-15T17:22:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.