# Logstash - Parsing fields with duplicate names

**URL:** <https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131>\
**Category:** Logstash\
**Created:** [October 31, 2023, 12:11pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131 "2023-10-31T12:11:08Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mgotechlock](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\
**Post date:** [October 31, 2023, 12:11pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/1 "2023-10-31T12:11:08Z")

</div>

If I receive a log in that looks like this, how do I deal with the fact that the subfields under "records" are identical? Is there a concept of [records][operationName][0] and [1], for example?  
`{`  
`	"records": [`  
` {`  
` "time": "2023-10-31T11:21:11.0067970Z",`  
` "tenantId": "9ea725b4-0569-4102-9774-55555555",`  
` "operationName": "Publish",`  
` "category": "AdvancedHunting-DeviceProcessEvents",`  
` "Tenant": "DefaultTenant"`  
` },`  
` {`  
` "time": "2023-10-31T11:21:11.0068016Z",`  
` "tenantId": "9ea725b4-0569-4102-9774-55555555",`  
` "operationName": "Publish",`  
` "category": "AdvancedHunting-DeviceProcessEvents",`  
` "Tenant": "DefaultTenant"`  
` },`  
` {`  
` "time": "2023-10-31T11:21:11.0068068Z",`  
` "tenantId": "9ea725b4-0569-4102-9774-55555555",`  
` "operationName": "Publish",`  
` "category": "AdvancedHunting-DeviceProcessEvents",`  
` "Tenant": "DefaultTenant"`  
` },`

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 31, 2023, 12:17pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/2 "2023-10-31T12:17:09Z")

</div>

> [@mgotechlock](#):
>
> Is there a concept of [records][operationName][0] and [1], for example?

It would be `[records][0][operationName]` for example, but the best approach when you have an array of events like this is to split on this array and have one event per item.

You would have this in your Logstash configuration:

```auto
split {
    field => "records"
}

```

Then you will end up with one event for each item in the array and can access the fields with `[records][operationName]` for example.

---

<div class="post-metadata">

**Author:** ![mgotechlock](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\
**Post date:** [October 31, 2023, 12:26pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/3 "2023-10-31T12:26:38Z")

</div>

The weird thing is, and I just remembered, that I already do that. Most of my records work fine. I need to add a tag on failure to see if the split is failing.

```
split {
    field => ["records"]
}

```

---

<div class="post-metadata">

**Author:** ![mgotechlock](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\
**Post date:** [October 31, 2023, 12:51pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/4 "2023-10-31T12:51:48Z")

</div>

@leandrojmp  
I don't undestand why some of my logs split fine and others do not. I don't get any split errors.  
Could the size of the log impact this? The log I am looking at is 2130 lines long, which should probably be split into about 40 separate records but its happening.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 31, 2023, 12:56pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/5 "2023-10-31T12:56:37Z")

</div>

Not sure, you didn't share what your pipeline configuration looks like, what is your input or any error logs.

But if Logstash cannot split on the field `records` it will generate a log error.

Do you have documents where the field `records` wasn't splitted on your Elasticsearch? If so, please please share the entire json document that you can get on Kibana Discover.

---

<div class="post-metadata">

**Author:** ![mgotechlock](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\
**Post date:** [October 31, 2023, 1:19pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/6 "2023-10-31T13:19:42Z")

</div>

My bad, split is working. All is well, thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 28, 2023, 1:19pm UTC](https://discuss.elastic.co/t/logstash-parsing-fields-with-duplicate-names/346131/7 "2023-11-28T13:19:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.