# Unable to parse JSON logs through filebeat prospector

**URL:** https://discuss.elastic.co/t/unable-to-parse-json-logs-through-filebeat-prospector/111980
**Category:** Beats
**Tags:** filebeat
**Created:** [December 15, 2017, 12:22pm UTC](https://discuss.elastic.co/t/unable-to-parse-json-logs-through-filebeat-prospector/111980 "2017-12-15T12:22:15Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Shubham\_Mahajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_mahajan/32/22140_2.png) [@Shubham\_Mahajan](https://discuss.elastic.co/u/Shubham_Mahajan)
#### Post date: [December 15, 2017, 12:22pm UTC](https://discuss.elastic.co/t/unable-to-parse-json-logs-through-filebeat-prospector/111980/1 "2017-12-15T12:22:15Z")

</div>

I'm trying to parse multiline JSON logs into elasticsearch using filebeat.

```
{
  "took": 95,
  "timed_out": false,
  "_shards": {
    "total": 85,
    "successful": 85,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 391847,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "metrics": {
      "doc_count_error_upper_bound": 15546,
      "sum_other_doc_count": 365359,
      "buckets": [
        {
          "key": "ilb.io",
          "doc_count": 26488,
          "network_name": {
            "doc_count_error_upper_bound": 1211,
            "sum_other_doc_count": 25508,
            "buckets": [
              {
                "key": "enp130s17f2",
                "doc_count": 980,
                "series": {
                  "buckets": [
                    {
                      "key_as_string": "2017-12-15T04:00:00.000Z",
                      "key": 1513310400000,
                      "doc_count": 231,
                      "avg": {
                        "value": 0
                      }
                    },
                    {
                      "key_as_string": "2017-12-15T08:00:00.000Z",
                      "key": 1513324800000,
                      "doc_count": 749,
                      "avg": {
                        "value": 0
                      },
                      "movavg": {
                        "value": 0
                      },
                      "surprise": {
                        "value": 0
                      }
                    }
                  ]
                },
                "largest_surprise": {
                  "value": 0,
                  "keys": [
                    "2017-12-15T08:00:00.000Z"
                  ]
                }
              }
            ]
          },
          "ninetieth_surprise": {
            "values": {
              "90.0": 0
            }
          }
        }
      ]
    }
  }
}

```

And the filebeat.yml config:

```
  json.message_key: aggregations
  json.keys_under_root: true
  json.add_error_key: true
  tags: [
    "aggs",
    "anomaly",
    "json"
  ]

```

could somebody please point me in right direction,..??

---

<div class="post-metadata">

### Author: ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)
#### Post date: [December 18, 2017, 12:05pm UTC](https://discuss.elastic.co/t/unable-to-parse-json-logs-through-filebeat-prospector/111980/2 "2017-12-18T12:05:20Z")

</div>

Hi @Shubham_Mahajan,

When parsing a multiline JSON message first you have to get the full message using multiline and then apply `decode_json_fileds` processor to the result, try something like this:

```auto
filebeat.prospectors:
- paths:
    - input.json
  multiline.pattern: '^{'
  multiline.negate: true
  multiline.match: after
  processors:
  - decode_json_fields:
      fields: ['message']
      target: json

```

---

<div class="post-metadata">

### Author: ![Shubham\_Mahajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_mahajan/32/22140_2.png) [@Shubham\_Mahajan](https://discuss.elastic.co/u/Shubham_Mahajan)
#### Post date: [December 19, 2017, 6:56am UTC](https://discuss.elastic.co/t/unable-to-parse-json-logs-through-filebeat-prospector/111980/3 "2017-12-19T06:56:48Z")

</div>

Thanks a lot @exekias

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 16, 2018, 6:56am UTC](https://discuss.elastic.co/t/unable-to-parse-json-logs-through-filebeat-prospector/111980/4 "2018-01-16T06:56:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.