# \[JSON filter\] Split Dynamic fields
**URL:** https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774
**Category:** Logstash
**Created:** [February 10, 2019, 10:08pm UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774 "2019-02-10T22:08:41Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@manunc](https://discuss.elastic.co/u/manunc)
#### Post date: [February 10, 2019, 10:08pm UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774/1 "2019-02-10T22:08:41Z")
Hello, I try to create an event for each fields named "XXX.YY" coming from the following JSON message (data object):
> {"data":{
> "316.2":[{"sensorId":"316.2.3","sensorTime":1549599391,"sensorValue":0.08}],
> "315.10":[{"sensorId":"315.10.8","sensorTime":1549599019,"sensorValue":43.01}],
> "314.6":[{"sensorId":"314.6.3","sensorTime":1549599083,"sensorValue":0.76}],
> "314.9":[{"sensorId":"314.9.3","sensorTime":1549599019,"sensorValue":0.606}],
> "315.1":[{"sensorId":"315.1.3","sensorTime":1549599321,"sensorValue":0.94}],
> "313.4":[{"sensorId":"313.4.3","sensorTime":1549599576,"sensorValue":0.52}]
> }}
My filter I try to adapt:
```
json {
source => "message"
target => "[@metadata][string-indexed]"
}
ruby {
code => "
# extract the string-indexed object to a local variable
string_indexed = event.get('[@metadata][string-indexed]')
# iterate over the keys in the order of the string-encoded integer index
items_array = string_indexed.keys.sort_by(&:to_i).map do |key|
# get the item from the indexed map
string_indexed[key]
end
# set the items array.
event.set('[@metadata][items-array]', items_array)
"
}
split {
field => "[@metadata][items-array]"
target => "[@metadata][single-item]"
}
mutate {
rename => {
"[@metadata][single-item][sensorTime]" => "sensorTime"
"[@metadata][single-item][sensorValue]" => "sensorValue"
"[@metadata][single-item][sensorId]" => "sensorId"
}
}
}
```
I would like to create the following event format:
one document per XXX.YY with the following fields:
> data.316.2.sensorTime
> data.316.2.sensorId
> data.316.2.sensorValue
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 10, 2019, 11:19pm UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774/2 "2019-02-10T23:19:06Z")
Do not use @metadata until you are sure the fields have the form you want. Use
```
output { stdout { codec => rubydebug } }
```
and make sure the fields look the way you need them to look. Once you are sure that, for example, [metadata][string-indexed] looks the way you want, change the references to be [@metadata][string-indexed]
They currently do not. For example, you want the data sub-object when fetching string\_indexed
```
string_indexed = event.get('[@metadata][string-indexed][data]')
```
and the first array entry when iterating over the array
```
string_indexed[key][0]
```
Lastly, you cannot use [@metadata] as the target of the split. (Or at least I cannot get it to work. rubydebug { metadata =\> true } shows the fields exists but the mutate+rename all return nil.) Try this:
```
split {
field => "[@metadata][items-array]"
target => "[metadata][single-item]"
}
mutate {
rename => {
"[metadata][single-item][sensorTime]" => "sensorTime"
"[metadata][single-item][sensorValue]" => "sensorValue"
"[metadata][single-item][sensorId]" => "sensorId"
}
remove_field => "[metadata]"
}
```
That will get you events that look like
```
"sensorId" => "316.2.3",
"sensorTime" => 1549599391,
"sensorValue" => 0.08,
```
Extracting the first two fields of the sensorId and adding them to the field names (which sounds like a bad idea to me) is left as an exercise for the reader. I know I have written examples in this forum of iterating over the event hash and renaming fields.
---
### Author:  [@manunc](https://discuss.elastic.co/u/manunc)
#### Post date: [February 11, 2019, 6:58am UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774/3 "2019-02-11T06:58:48Z")
Thanks, you help me a lot.
I succeed to get the following event now
```
"metadata" => {
"single-item" => [
[0] {
"sensorTime" => 1549867548,
"sensorValue" => 0.52,
"sensorId" => "316.2.3"
}
]
},
```
using the following code:
```
json {
source => "message"
target => "[metadata][string-indexed]"
}
ruby {
code => "
# extract the string-indexed object to a local variable
string_indexed = event.get('[metadata][string-indexed][data]')
# iterate over the keys in the order of the string-encoded integer index
items_array = string_indexed.keys.sort_by(&:to_i).map do |key|
# get the item from the indexed map
string_indexed[key]
end
# set the items array.
event.set('[metadata][items-array]', items_array)
"
}
split {
field => "[metadata][items-array]"
target => "[metadata][single-item]"
}
mutate {
rename => {
"[metadata][single-item][sensorTime]" => "sensorTime"
"[metadata][single-item][sensorValue]" => "sensorValue"
"[metadata][single-item][sensorId]" => "sensorId"
}
remove_field => "[metadata][string-indexed]"
remove_field => "[metadata][items-array]"
}
```
My mutate function does not work, because [metadata][single-item] is an array containing 1 entry [0] ?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 11, 2019, 12:37pm UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774/4 "2019-02-11T12:37:13Z")
As I said, when iterating over the array, you should reference the array entry using
```
string_indexed[key][0]
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 11, 2019, 12:37pm UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774/5 "2019-03-11T12:37:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.