# Split of JSON array into multiple events in Kibana
**URL:** https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018
**Category:** Logstash
**Created:** [April 22, 2021, 8:46pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018 "2021-04-22T20:46:39Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@pedro\_gonzalez](https://discuss.elastic.co/u/pedro_gonzalez)
#### Post date: [April 22, 2021, 8:46pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/1 "2021-04-22T20:46:39Z")
Hi,
I am trying to split into different events (logs) this log schema:
```auto
"_index": "cocacola",
"_type": "raw",
"_id": "CqwJ63MBEQS11DmXsDyRZl",
"_score": 1,
"_source": {
"messageId": 33297111461,
"eventType": "EntityUpdated",
"username": "Administrator",
"timeStamp": "2020-08-13T16:59:50.87Z",
"data": {
"name": "Target:1111111",
"definition": "Target",
"is_new": null,
"user_id": null,
"usergroup_id": null,
"rules": null,
"property_changes": [
{
"property": "Target.Completed",
"data_type": "System.DateTime",
"value": {
"original": null,
"new": "2020-08-13T16:59:50.8491889Z"
}
},
{
"property": "Target.State",
"data_type": "System.String",
"value": {
"original": "good",
"new": "very good."
}
},
{
"property": "Target.State",
"data_type": "System.String",
"value": {
"original": "Processing_Completed",
"new": "Completed"
}
}
```
into something like this:
```auto
"_index": "cocacola",
"_type": "raw",
"_id": "CqwJ63MBEQSDmXsDyRZl",
"_score": 1,
"_source": {
"messageId": 33291117461,
"eventType": "EntityUpdated",
"username": "Administrator",
"timeStamp": "2020-08-13T16:59:50.87Z",
"data": {
"name": "Target:1111111",
"definition": "Target",
"is_new": null,
"user_id": null,
"usergroup_id": null,
"rules": null,
"property_changes": [
{
"property": "Target.Completed",
"data_type": "System.DateTime",
"value": {
"original": null,
"new": "2020-08-13T16:59:50.8491889Z"
}
}
```
and this:
```auto
"_index": "cocacola",
"_type": "raw",
"_id": "CqwJ63MBEQSDmXsDyRZl",
"_score": 1,
"_source": {
"messageId": 3329111461,
"eventType": "EntityUpdated",
"username": "Administrator",
"timeStamp": "2020-08-13T16:59:50.87Z",
"data": {
"name": "Target:1111111",
"definition": "Target",
"is_new": null,
"user_id": null,
"usergroup_id": null,
"rules": null,
"property_changes": [
{
"property": "Target.State",
"data_type": "System.String",
"value": {
"original": "good",
"new": "very good."
}
}
```
I am using this but it doesn't work:
```auto
input {
elasticsearch {
hosts => ["https://xxxxxxxxxxx"]
index => "xxxxxxxxx"
user => "lxxxxxxxxxxxx"
password => "xxxxxxxxxxx"
query => '{ "qxxxxxxxxxxx" }}}'
}
}
filter {
json {
source => "message"
}
split {
field => "[data][property_changes]"
}
mutate {
add_field => {
"[user][name]" => "%{[username]}"
"[event][id]" => "%{[messageId]}"
"[event][type]" => "%{[eventType]}"
"[event][action]" => "%{[data][property_changes][property]}"
}
remove_field => ["message"]
}
}
output {
elasticsearch {
hosts => "https://xxxxxxxxxxxxxxxxxxx.xxxxxxxxxx"
index => "cxxxxxxxxxxx}"
user => "xxxxxxxxxxx"
password => "xxxxxxxxxx"
}
}
```
I have been checking multiple threads, some of them have a similar case like mine but it doesn't work when I try to adapt it to my situation. I tried something similar to this: [Split nested json array](https://discuss.elastic.co/t/split-nested-json-array/147969)
Maybe you can help @magnusbaeck since I have seen you in many threads. Could it work with a ruby plugin?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 23, 2021, 3:18am UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/2 "2021-04-23T03:18:20Z")
Please refrain from pinging folks directly, this is a forum and anyone that participates might be able to assist you.
BTW I moved your question to #elastic-stack:logstash
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 23, 2021, 2:24pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/3 "2021-04-23T14:24:37Z")
> [@pedro\_gonzalez](#):
>
> it doesn't work when I try to adapt it to my situation
What do you mean by that?
---
### Author:  [@pedro\_gonzalez](https://discuss.elastic.co/u/pedro_gonzalez)
#### Post date: [April 23, 2021, 3:03pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/4 "2021-04-23T15:03:23Z")
Hi @Badger,
I meant that when I try to use what you see in that thread adjusted to my code, it doesn't split the "property\_changes" as expected.
Having in mind my initial log I would expect this:
1 log
```auto
"_index": "cocacola",
"_type": "raw",
"_id": "CqwJ63MBEQS11DmXsDyRZl",
"_score": 1,
"_source": {
"messageId": 33297111461,
"eventType": "EntityUpdated",
"username": "Administrator",
"timeStamp": "2020-08-13T16:59:50.87Z",
"data": {
"name": "Target:1111111",
"definition": "Target",
"is_new": null,
"user_id": null,
"usergroup_id": null,
"rules": null,
"property_changes": [
{
"property": "Target.Completed",
"data_type": "System.DateTime",
"value": {
"original": null,
"new": "2020-08-13T16:59:50.8491889Z"
}
}
```
2 log
```auto
"_index": "cocacola",
"_type": "raw",
"_id": "CqwJ63MBEQS11DmXsDyRZl",
"_score": 1,
"_source": {
"messageId": 33297111461,
"eventType": "EntityUpdated",
"username": "Administrator",
"timeStamp": "2020-08-13T16:59:50.87Z",
"data": {
"name": "Target:1111111",
"definition": "Target",
"is_new": null,
"user_id": null,
"usergroup_id": null,
"rules": null,
"property_changes": [
{
"property": "Target.State",
"data_type": "System.String",
"value": {
"original": "Processing_Completed",
"new": "Completed"
}
}
```
3 log
```auto
"_index": "cocacola",
"_type": "raw",
"_id": "CqwJ63MBEQS11DmXsDyRZl",
"_score": 1,
"_source": {
"messageId": 33297111461,
"eventType": "EntityUpdated",
"username": "Administrator",
"timeStamp": "2020-08-13T16:59:50.87Z",
"data": {
"name": "Target:1111111",
"definition": "Target",
"is_new": null,
"user_id": null,
"usergroup_id": null,
"rules": null,
"property_changes": [
{
"property": "Target.State",
"data_type": "System.String",
"value": {
"original": "Processing_Completed",
"new": "Completed"
}
}
```
and I end up having only the number 1 and 3. The second log is not shown in Kibana.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 23, 2021, 3:23pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/5 "2021-04-23T15:23:02Z")
If some events are being indexed but others are not then I would suspect a mapping exception. For example, if [dynamic mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html) decided that a particular field should be a date then any event where it was not a date would be rejected, and logstash would log an exception. I do not think that would happen for "[property\_changes][value][new]" being "2020-08-13T16:59:50.8491889Z" because that is not a date format that elasticsearch would auto-detect, but it is worth looking at the logstash logs.
---
### Author:  [@pedro\_gonzalez](https://discuss.elastic.co/u/pedro_gonzalez)
#### Post date: [April 25, 2021, 3:17pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/6 "2021-04-25T15:17:48Z")
You are right @Badger. That should be the problem. Before making any change in the mapping, how can I set up a determined field to contain multiple data types? For example, "value.new" can contain a string sometimes or it can be a date.
Thanks for your feedback. Really helpful.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 25, 2021, 3:26pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/7 "2021-04-25T15:26:28Z")
> [@pedro\_gonzalez](#):
>
> how can I set up a determined field to contain multiple data types? For example, "value.new" can contain a string sometimes or it can be a date.
A field in elasticsearch can only have one type. value.new would have to be a string. If dynamic mapping gets that wrong you would have to use a [template](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html) to set the type.
---
### Author:  [@pedro\_gonzalez](https://discuss.elastic.co/u/pedro_gonzalez)
#### Post date: [April 25, 2021, 5:55pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/8 "2021-04-25T17:55:28Z")
The limitation I have is that sometimes this field "value.new" contains a date or a string or a number so according to what I have been reading I cannot do too much on adjusting it to each data type.
Thank you @badger.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 23, 2021, 5:56pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018/9 "2021-05-23T17:56:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.