# Json file parsing not working in logstash using beats
**URL:** https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242
**Category:** Logstash
**Created:** [January 27, 2016, 1:44pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242 "2016-01-27T13:44:28Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 27, 2016, 1:44pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/1 "2016-01-27T13:44:29Z")
Hi,
I have a log which contain the json as input . when i was using lower version of logstash with logcouier i had processed the json input. But now im using latest version of logstash with beats. In this scenario json is not working for me. Did i miss anything to parse the file or else have to define anything in yml of beats. Please anyone help me to resolve it.
input {
beats {
port =\> 10546
type =\> "json"
}
}
filter
{
##Pattern to split the JSON format from the input.
grok {
match =\> ["message","^]?((?[[[a-zA-Z0-9,:] ]-]\*)) ?%{GREEDYDATA:app\_json}"]
}
##Parsing JSON input to JSON Filter..
json {
source =\> "app\_json"
}
}
This is how i'm processing my input. Please correct me if i did anything wrong
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 27, 2016, 2:32pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/2 "2016-01-27T14:32:11Z")
Use the beats input with Filebeat, not the courier input.
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 27, 2016, 2:34pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/3 "2016-01-27T14:34:51Z")
I'm using beats only, I was pasted wrongly here.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 27, 2016, 2:39pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/4 "2016-01-27T14:39:00Z")
Okay, so what do you actually get? The result of a `stdout { codec => rubydebug }` output would be useful.
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 27, 2016, 2:52pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/5 "2016-01-27T14:52:01Z")
This is im getting in my stdout screen
"message" =\> "DiagnosticsLog brri-storms-t01 BRRIHUB\_CEP STORM 111111111 2015-06-23T10:06:59.781Z WebServiceUtil: RESPONSE :{ "GetSpecialServiceCampaignResponse":{ "GetSpecialServiceCampaignBOD":{ "ApplicationArea":{ "CreationDateTime":"2015-06-23T10:06:58.418Z", "BODID":"", "VersionID":"2.0", "Sender":{ "UserID":"brri\_hub", "CreatorNameCode":"BRRI\_HUB", "SenderNameCode":"EIG" }, "Destination":{ "DestinationNameCode":"BRRI Hub" } }, "GetSpecialServiceCampaignDataArea":{ "BusinessContext":{ "TransactionType":"POST", "Action":"SpecialServiceCampaignRealTimeRequest", "MessageID":"88194000-F954-43B7-BE9C-23A270A0C409", "MessagePurpose":"Reporting", "FromRole":"Vehicle", "ToRole":"Dealer", "ActionType":"Insert" }, "SpecialServiceCampaignPayload":{ "VIN":"", "CampaignData":[] } } } } }",
"@version" =\> "1",
"@timestamp" =\> "2016-01-27T14:51:38.906Z",
"beat" =\> {
"hostname" =\> "vtmnalrhl216",
"name" =\> "vtmnalrhl216"
},
"count" =\> 1,
"fields" =\> nil,
"input\_type" =\> "log",
"offset" =\> 0,
"source" =\> "/beep/brri-logs/DiagnosticsLog2015-06-23-10.log",
"type" =\> "elk\_diagno\_log",
"host" =\> "vtmnalrhl216",
"app\_logType" =\> "DiagnosticsLog",
"app\_instancId" =\> " brri-storms-t01",
"app\_applicationId" =\> "BRRIHUB\_CEP",
"app\_methodName" =\> "STORM",
"app\_correlationId" =\> "111111111",
"app\_logRecorded\_Timestamp" =\> "2015-06-23T10:06:59.781Z",
"app\_logMessage" =\> "WebServiceUtil: RESPONSE :{ "GetSpecialServiceCampaignResponse":{ "GetSpecialServiceCampaignBOD":{ "ApplicationArea":{ "CreationDateTime":"2015-06-23T10:06:58.418Z", "BODID":"", "VersionID":"2.0", "Sender":{ "UserID":"brri\_hub", "CreatorNameCode":"BRRI\_HUB", "SenderNameCode":"EIG" }, "Destination":{ "DestinationNameCode":"BRRI Hub" } }, "GetSpecialServiceCampaignDataArea":{ "BusinessContext":{ "TransactionType":"POST", "Action":"SpecialServiceCampaignRealTimeRequest", "MessageID":"88194000-F954-43B7-BE9C-23A270A0C409", "MessagePurpose":"Reporting", "FromRole":"Vehicle", "ToRole":"Dealer", "ActionType":"Insert" }, "SpecialServiceCampaignPayload":{ "VIN":"", "CampaignData":[] } } } } }",
"tags" =\> [
[0] "\_grokparsefailure"
],
"app\_application" =\> "SpecialServiceCampaignRealTimeRequest",
"app\_vhrId" =\> "2015-06-23T10",
"app\_json" =\> "{ "GetSpecialServiceCampaignResponse":{ "GetSpecialServiceCampaignBOD":{ "ApplicationArea":{ "CreationDateTime":"2015-06-23T10:06:58.418Z", "BODID":"", "VersionID":"2.0", "Sender":{ "UserID":"brri\_hub", "CreatorNameCode":"BRRI\_HUB", "SenderNameCode":"EIG" }, "Destination":{ "DestinationNameCode":"BRRI Hub" } }, "GetSpecialServiceCampaignDataArea":{ "BusinessContext":{ "TransactionType":"POST", "Action":"SpecialServiceCampaignRealTimeRequest", "MessageID":"88194000-F954-43B7-BE9C-23A270A0C409", "MessagePurpose":"Reporting", "FromRole":"Vehicle", "ToRole":"Dealer", "ActionType":"Insert" }, "SpecialServiceCampaignPayload":{ "VIN":"", "CampaignData":[] } } } } }",
"fingerprint" =\> "345b358a8905d61f92bdc69e9aed5272bcb42002"
app\_json contain my json value and i'm that source in json filter.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 27, 2016, 3:01pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/6 "2016-01-27T15:01:02Z")
But the json filter you posted above parses the `notify` field.
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 27, 2016, 3:07pm UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/7 "2016-01-27T15:07:14Z")
That was copy paste error. Kindly check now i edited my input this is how i'm created logstash file.
input {
beats {
port =\> 10546
type =\> "json"
}
}
filter{
grok{
match =\>{"message" =\> "^?((?\[[[a-zA-Z0-9,\_.:] ]-]\*)) ?%{GREEDYDATA:app\_json}"}
}
json {
source =\> "app\_json"
}
}
---
### Author:  [@logstash\_oz](https://discuss.elastic.co/u/logstash_oz)
#### Post date: [January 29, 2016, 5:16am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/8 "2016-01-29T05:16:20Z")
Hi, I know this is not a correct place to ask this question. I am new to this community and also to elk. After splunk tried to bankrupt our company .. Can someone please guide me to correct place where I can get advice about what can be the replacement for splunk?
I have tried setting up ELK , got the data(json logs) into elasticsearch and was able to do some play with data on kibana. However, I am still not sure, how to achieve the very critical things which we need. Alerting...!!! We are currently using splunk to monitor ecommerce website traffic. Any help , will be very much appreciated. I am very much tired of piecing together so many components in ELK alone, and another tool for alerting only makes it more difficult, from what I have read so far. Is Graylog better than ELK to replace my use case? I know nothing may be same as splunk. I just want to replace a part of our use case, so we can bargain with splunk for a better deal,at the earliest.
Thanks
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 29, 2016, 6:31am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/9 "2016-01-29T06:31:50Z")
Please start your own thread instead 🙂
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 29, 2016, 6:51am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/10 "2016-01-29T06:51:36Z")
Can any one help me out of the json issue
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 29, 2016, 6:55am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/11 "2016-01-29T06:55:41Z")
> [@Ganesh2303](#):
>
> %{GREEDYDATA:appjson}
The field you extract into is named 'appjson' but you are specifying that the JSON filter should use 'app\_json'. Make sure these are aligned and try again.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 29, 2016, 6:59am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/12 "2016-01-29T06:59:12Z")
> The field you extract into is named 'appjson' but you are specifying that the JSON filter should use 'app\_json'. Make sure these are aligned and try again.
No, the file actually contains app\_json. it's the Disqus formatter that interpreted the matching underscores as an emphasis and stripped them. I lost my patience after the second typo in the posted configurations.
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 29, 2016, 7:08am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/13 "2016-01-29T07:08:31Z")
Sorry for that typo @Magnus . I'm struck on the json part due to this issue im unable to move further step.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 29, 2016, 7:25am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/14 "2016-01-29T07:25:58Z")
OK. Now that I look closer I can see that some of the text is in italics. I assume this could be avoided if the provided configuration samples were formatted as 'Preformatted text' when posted?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 29, 2016, 7:42am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/15 "2016-01-29T07:42:29Z")
> [@Ganesh2303](#):
>
> { "GetSpecialServiceCampaignResponse":{ "GetSpecialServiceCampaignBOD":{ "ApplicationArea":{ "CreationDateTime":"2015-06-23T10:06:58.418Z", "BODID":"", "VersionID":"2.0", "Sender":{ "UserID":"brri\_hub", "CreatorNameCode":"BRRI\_HUB", "SenderNameCode":"EIG" }, "Destination":{ "DestinationNameCode":"BRRI Hub" } }, "GetSpecialServiceCampaignDataArea":{ "BusinessContext":{ "TransactionType":"POST", "Action":"SpecialServiceCampaignRealTimeRequest", "MessageID":"88194000-F954-43B7-BE9C-23A270A0C409", "MessagePurpose":"Reporting", "FromRole":"Vehicle", "ToRole":"Dealer", "ActionType":"Insert" }, "SpecialServiceCampaignPayload":{ "VIN":"", "CampaignData":[] } } } } }
The issue could be the backslashes in the JSON string. Try adding a mutate sub filter as follows before parsing the JSON to clean it up.
```
mutate {
gsub => ["json_app", "[\\]", "" ]
}
```
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [January 29, 2016, 7:43am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/16 "2016-01-29T07:43:47Z")
Ok @Christian. I ll implement this idea..
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:13am UTC](https://discuss.elastic.co/t/json-file-parsing-not-working-in-logstash-using-beats/40242/17 "2017-07-06T05:13:47Z")