# Split a JSON array into individual documents in Logstash

**URL:** <https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151>\
**Category:** Logstash\
**Created:** [July 30, 2020, 3:35am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151 "2020-07-30T03:35:21Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![jclemons7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jclemons7/32/20655_2.png) [@jclemons7](https://discuss.elastic.co/u/jclemons7)\
**Post date:** [July 30, 2020, 3:35am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/1 "2020-07-30T03:35:21Z")

</div>

I'm sorry to ask this honestly.. normally I'm a decently smart fella.. but I can't seem to grasp this. I have some JSON coming that I need to split into individual documents.

```auto

{
  "lan": {
    "inkbps": 756.2703471377937,
    "outkbps": 16469.254210648105,
    "interface": "em1"
  },
  "wan": {
    "inkbps": 12432.741640620181,
    "outkbps": 613.0682382407641,
    "interface": "pppoe0",
    "friendlyiface": "wan",
    "name": "internet_pppoe",
    "status": "okay",
    "monitorip": "8.8.8.8",
    "sourceip": "17.1.1.1",
    "delay": "53.561ms",
    "loss": "1%"
  },
  "opt2": {
    "inkbps": 33.28968847276036,
    "outkbps": 26.285475058869885,
    "interface": "ovpnc2",
    "friendlyiface": "opt2",
    "name": "pia_vpn_vpnv4",
    "status": "okay",
    "monitorip": "10.17.1.1",
    "sourceip": "10.17.1.2",
    "delay": "",
    "loss": ""
  }
}

```

It seems pretty simple based on all the examples I've found on this site and others.. and here's what I've ended up with..

```auto

filter
    {
            if [log_type] == "edge_utilization"
            {
                    json
                    {
                            source => "[messages]"
                    }
                    split
                    {
                            field => "[messages]"
                    }
            }
    }

```

Which then just throws \_grokparsefailure, \_split\_type\_failure

I've tried a bunch of different iterations, based on suggestions and articles I've found.. but even if I get the tags to go away, it still never splits the nodes into individual documents.

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [July 30, 2020, 3:45am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/2 "2020-07-30T03:45:03Z")

</div>

I can't say anything about the grok failure as you didn't include the configuration that has caused that. But your split problem is probably due to the fact that this is a hash, not an array. Using a ruby filter to change it might help?  
`event.set("messages", event.get("messages").values)`

If it doesn't work, please post the ruby debug output, so we'll know exactly what your current result looks like. And have a look at the Logstash logs. Wasn't the \_split\_type\_failure accompanied by a warning?

---

<div class="post-metadata">

**Author:** ![jclemons7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jclemons7/32/20655_2.png) [@jclemons7](https://discuss.elastic.co/u/jclemons7)\
**Post date:** [July 30, 2020, 4:08am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/3 "2020-07-30T04:08:29Z")

</div>

Hey.. thanks so much for the response. I'm not quite sure I understand.. where do I put that? If it helps, here is my entire pipeline:

```auto

input
{
	udp
	{
		port => 514
		codec => plain {charset => "ISO-8859-1"}
	}

	tcp 
	{
		port => 5040
	}

}

filter
{
	mutate
	{
		add_field => {"[raw_log]" => "%{[message]}"}
	}
}

filter
{
	if [log_type] == "edge_utilization"
	{
		json
		{
			source => "[message]"
		}
		split
		{
			field => "[message]"
		}
	}
}

output
{
	elasticsearch
	{
		hosts => ["http://localhost:9200"]
		index => "syslog-%{+YYYY.MM.dd}"
		document_id => "%{[fingerprint]}"	
	}
}

```

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [July 30, 2020, 4:13am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/4 "2020-07-30T04:13:48Z")

</div>

```auto
ruby {
  code => 'event.set("message", event.get("message").values)'
}

```

before your split filter.

---

<div class="post-metadata">

**Author:** ![jclemons7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jclemons7/32/20655_2.png) [@jclemons7](https://discuss.elastic.co/u/jclemons7)\
**Post date:** [July 30, 2020, 4:28am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/5 "2020-07-30T04:28:54Z")

</div>

I did the following:

```auto

filter
{
        if [log_type] == "edge_utilization"
        {
                json
                {
                        source => "[message]"
                }

                ruby 
                {
                       code => 'event.set("message", event.get("message").values)'
                }

                split
                {
                        field => "[message]"
                }
        }
}

```

Which then results in a \_rubyexception

Did I misunderstand where I needed to insert that line?

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [July 30, 2020, 4:50am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/6 "2020-07-30T04:50:43Z")

</div>

As I said, the logs and rubydebug output would be of interest. You can barely debug something based on tags. The data might look different from than I think.

---

<div class="post-metadata">

**Author:** ![jclemons7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jclemons7/32/20655_2.png) [@jclemons7](https://discuss.elastic.co/u/jclemons7)\
**Post date:** [July 30, 2020, 6:26am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/7 "2020-07-30T06:26:30Z")

</div>

Heh.. well, thanks anyway

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [July 30, 2020, 6:29am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/8 "2020-07-30T06:29:08Z")

</div>

I don't know what to make of that answer . You don't want to provide more information?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 27, 2020, 6:29am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151/9 "2020-08-27T06:29:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.