# Parse json with filebeat
**URL:** https://discuss.elastic.co/t/parse-json-with-filebeat/105292
**Category:** Beats
**Tags:** filebeat
**Created:** [October 25, 2017, 8:21pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292 "2017-10-25T20:21:20Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@mbvelo](https://discuss.elastic.co/u/mbvelo)
#### Post date: [October 25, 2017, 8:21pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/1 "2017-10-25T20:21:20Z")
```
please assist with this config using Filebeat -> Elasticsearch , My Json is not being decomposed.
other fields are populating correctly, i want the message to be decomposed as well
filebeat.prospectors:
-
input_type: log
json.add_error_key: true
json.keys_under_root: true
paths:
- message.log
output.elasticsearch:
hosts:
- "http://localhost:9200"
```
output
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 26, 2017, 11:47am UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/2 "2017-10-26T11:47:46Z")
You have some sample input and sample output (expected + actual)?
Filebeat currently expects one json document per line. Multiline json is not really supported by the reader.
---
### Author:  [@mbvelo](https://discuss.elastic.co/u/mbvelo)
#### Post date: [October 26, 2017, 12:01pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/3 "2017-10-26T12:01:09Z")
it is populating some fields correctly but not all
i want the message part to be broken into individual fields for eg
OUT: xxxxx
count:yyyyy
---
### Author:  [@mbvelo](https://discuss.elastic.co/u/mbvelo)
#### Post date: [October 26, 2017, 12:14pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/4 "2017-10-26T12:14:54Z")
`This is my output - i want message to be decomposed into smaller parts`
{
"\_index": "filebeat-2017.10.26",
"\_type": "doc",
"\_id": "ABCD",
"\_version": 1,
"\_score": null,
"\_source": {
"@timestamp": "2017-10-26T10:21:26.748Z",
"@version": 1,
"HOSTNAME": ".com",
"IP": "[xxx.xxx.xxx.xxx](http://xxx.xxx.xxx.xxx)",
"beat": {
"hostname": "xxxx",
"name": "xxxx",
"version": "5.6.3"
},
"input\_type": "log",
"level": "DEBUG",
"level\_value": 10000,
"logger\_name": "ACTIVITY",
"message": "IN:xx.xxx.xxx.xxxx\t\t ABCD-\*\*\*\*\*\*123\t OED-OP1000\t DEX-Gax\t DM-An\t DE-\t DN-AD\t DV-null\t AV-BLE\t Cdfs-S Data: {mod=Gax\tIP=xxx.xxxx.xxxx..xx\dfofofo=080807\tsLdse=Y\tDI=Y\DN=1\tDD=123\rdoe=000\tMDD=opSS\tdop=132231\tSK\_ID=231\tFGFG=BP\tRDFF=ADSD\tserV=8.0\tTS=poi\tDFOP=dfp\tData=S}",
"offset": 564623,
"sessionId": "xxxxxx",
"source": "my.log",
"thread\_name": "field : 0",
"type": "log"
},
"fields": {
"@timestamp": [
1509013286748
]
},
"sort": [
1509013286748
]
}`Preformatted text`
---
### Author:  [@mbvelo](https://discuss.elastic.co/u/mbvelo)
#### Post date: [October 26, 2017, 1:47pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/5 "2017-10-26T13:47:29Z")
Any Ideas what i am not doing right?
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 27, 2017, 12:55pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/6 "2017-10-27T12:55:17Z")
The log is obviously already parsed into a json document. The `message` field is from the original document. The contents of `message` is no valid JSON. You will need logstash or ES ingest node for additional processing of the `message` field.
---
### Author:  [@mbvelo](https://discuss.elastic.co/u/mbvelo)
#### Post date: [October 27, 2017, 1:58pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/7 "2017-10-27T13:58:26Z")
```
Thank you
do you an example of this kind of pattern?
I tried using https://grokdebug.herokuapp.com to build a pattern, please advice
```
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 27, 2017, 2:08pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/8 "2017-10-27T14:08:19Z")
Format is quite funny. Plus I can't tell (by one message) if order of fields is always the same or `Data` is always at the end. But it looks like grok is not enough here. If Data is always at the end you can try to 'split' the document using grok before and after Data. Everything before the document kind of looks like CSV-parseable (look for CSV or kv filter in logstash). The contents in Data I can't really tell about the format due to special characters like `\d` or `\r`. Maybe you want to replace those with `\t` as well before applying CSV or kv filter in logstash. Good luck
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 24, 2017, 2:08pm UTC](https://discuss.elastic.co/t/parse-json-with-filebeat/105292/9 "2017-11-24T14:08:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.