# Problem processing multiline json

**URL:** <https://discuss.elastic.co/t/problem-processing-multiline-json/250446>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 30, 2020, 5:13am UTC](https://discuss.elastic.co/t/problem-processing-multiline-json/250446 "2020-09-30T05:13:31Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 30, 2020, 5:13am UTC](https://discuss.elastic.co/t/problem-processing-multiline-json/250446/1 "2020-09-30T05:13:32Z")

</div>

MY INPUT FILE :

```auto
[
    {
        "value": "drrrrrrrrrrrrropfields_NTNT10",
        "Id": "drrrrrrrrrropfields_10",
        "PSShowComputerName": "dropfields_10"
    },
	{
        "value": "dropfields_nT8",
        "PSN": "dropfields_8",
        "Id": "dropfields_8",
        "PSShowComputerName": "dropfields_8"
    },
	{
        "value": "dropfields_NT9",
        "PSN": "dropfields_9",
        "Id": "dropfields_9",
        "PSShowComputerName": "dropfields_9"
    }
	
]

```

MY YML FILE :

```auto
filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - "C:\\Program Files\\Filebeat\\data\\new\\*.json"
        scan_frequency: 1s
        tail_files: false
        multiline.pattern: '},'
        multiline.negate: true
        multiline.match: before
processors:
      - drop_fields:
          fields: ["ecs.version", "host.name","agent.id","agent.hostname","agent.version","agent.type","agent.ephemeral_id","agent","ecs","host","log.file.path","log.file","log.flags","log.offset","log","input.type","input","@metadata.version"]
      - decode_json_fields:
          fields: ["message"]
          target: "datas"
output.console:
        pretty: true

```

MY OUTPUT

```auto

{
  "@timestamp": "2020-09-30T05:01:02.650Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.3.0"
  },
  "message": "[\n {\n \"value\": \"drrrrrrrrrrrrropfields_NTNT10\",\n \"Id\": \"drrrrrrrrrropfields_10\",\n \"PSShowComputerName\": \"dropfields_
10\"\n },"
}
{
  "@timestamp": "2020-09-30T05:01:02.650Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.3.0"
  },
  "message": "\t{\n \"value\": \"dropfields_nT8\",\n \"PSN\": \"dropfields_8\",\n \"Id\": \"dropfields_8\",\n \"PSShowComputerName\": \"drop
fields_8\"\n },"
}
{
  "@timestamp": "2020-09-30T05:01:02.652Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.3.0"
  },
  "message": "\t{\n \"value\": \"dropfields_NT9\",\n \"PSN\": \"dropfields_9\",\n \"Id\": \"dropfields_9\",\n \"PSShowComputerName\": \"drop
fields_9\"\n }\n\t",
  "datas": {
    "value": "dropfields_NT9",
    "PSN": "dropfields_9",
    "Id": "dropfields_9",
    "PSShowComputerName": "dropfields_9"
  }
}

```

As you can see only the last event of my JSON file is decoded properly as the first two have unwanted characters before and after the curly parenthesis "{" and "}" .

how do I fix this??

---

<div class="post-metadata">

**Author:** ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\
**Post date:** [September 30, 2020, 8:23am UTC](https://discuss.elastic.co/t/problem-processing-multiline-json/250446/2 "2020-09-30T08:23:29Z")

</div>

Hello @niv,

are you using Logstash?  
this issue can be resolved in Logstash using the Json filter.

or target to message in the JSON Processor

> **[JSON filter plugin | Logstash Reference \[6.8\] | Elastic](https://www.elastic.co/guide/en/logstash/6.8/plugins-filters-json.html)**

  

> **[Decode JSON fields | Filebeat Reference \[7.9\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html)**

Cheers

---

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [September 30, 2020, 4:05pm UTC](https://discuss.elastic.co/t/problem-processing-multiline-json/250446/3 "2020-09-30T16:05:21Z")

</div>

hey as you can see I am using a decode\_json\_fields. But the problem is that there are unwanted characters get added to the first two json events when I use the multiline to parse it. my question is how do I avoid it ?

---

<div class="post-metadata">

**Author:** ![niv](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@niv](https://discuss.elastic.co/u/niv)\
**Post date:** [October 18, 2020, 10:34am UTC](https://discuss.elastic.co/t/problem-processing-multiline-json/250446/4 "2020-10-18T10:34:22Z")

</div>

Can someone help please?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2020, 12:34pm UTC](https://discuss.elastic.co/t/problem-processing-multiline-json/250446/5 "2020-11-15T12:34:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.