# Handling events after using split

**URL:** <https://discuss.elastic.co/t/handling-events-after-using-split/235929>\
**Category:** Logstash\
**Created:** [June 5, 2020, 9:49am UTC](https://discuss.elastic.co/t/handling-events-after-using-split/235929 "2020-06-05T09:49:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pasno](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@pasno](https://discuss.elastic.co/u/pasno)\
**Post date:** [June 5, 2020, 9:49am UTC](https://discuss.elastic.co/t/handling-events-after-using-split/235929/1 "2020-06-05T09:49:59Z")

</div>

Hi,

I'm trying to figure out how to use the split-filter in Logstash correctly. I've read forum-posts and the doc, but don't get to the bottom of this.

I'm using the Azure Eventhub plugin to ingest stream of logs. The events comes in bulks (array) like this:

```
"message": {
    "records": [
        { "time": "2020-06-05T08:19:03.6240267Z", "tenantId": "", ... , "Timestamp":"2020-06-05T08:16:14.8727964Z" },
        { "time": "2020-06-05T08:19:04.6240267Z", "tenantId": "", ... , "Timestamp":"2020-06-05T08:16:14.8727964Z"},
        ...
    ]
}

```

My current filter seems to work:

```
filter {
    json {
        source => "message"
    }
    split {
        field => ["records"]
        remove_field => ["message"]
    }
}

```

It splits the records within the field "message" correctly.

Q1: But how am I able to handle these separate records after they are split? I see [the doc list some options](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html#plugins-filters-split-common-options). Are there any ways to handle the separate records more advanced than using these common options? E.g. if I want to run other plugins on the records after they are split?

Q2: I've tried to replace the @timestamp field of each record with the actual log timestamp - without luck. I see that the update-option is not available using split. After using the split-filter the actual log timestamp appear as a field called "records.properties.Timestamp". So this works:

```
filter {
    json {
        source => "message"
    }
    split {
        field => ["records"]
        remove_field => ["message"]
        add_field => { "newTimestampField" => "%{[records][properties][Timestamp]}" }
    }
}

```

But how am I able to replace the @timestamp with the [records][properties][Timestamp] ?

Thaks a million 🙂

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2020, 2:08pm UTC](https://discuss.elastic.co/t/handling-events-after-using-split/235929/2 "2020-06-05T14:08:40Z")

</div>

> [@pasno](#):
>
> But how am I able to replace the @timestamp with the [records][properties][Timestamp] ?

Use a date filter after the split.

---

<div class="post-metadata">

**Author:** ![pasno](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@pasno](https://discuss.elastic.co/u/pasno)\
**Post date:** [June 8, 2020, 1:16pm UTC](https://discuss.elastic.co/t/handling-events-after-using-split/235929/3 "2020-06-08T13:16:33Z")

</div>

Thanks for replying!

I tested a bit more and I got it to work.

It seems like everything after the split is performed on each record-element.

For the record I ended up with this which works for me;

```
######### Input #########

# Azure Eventhub
input {
    azure_event_hubs {
        event_hub_connections => ["removed"]
        threads => 3
        decorate_events => true
        consumer_group => "removed"
        storage_connection => ""
        storage_container => "removed"
        type => "azure_event_hub"
    }
}

######### Filter #########

filter {
    json {
        source => "message"
    }

    split {
        field => ["records"]
        remove_field => ["message"]
    }
    
    mutate {
        add_field => {"ingest.time" => "%{@timestamp}"}
    }
    
    date {
        match => ["[records][properties][Timestamp]", "ISO8601"]
        target => "@timestamp"
    }
    
    if [records][category] == "cat1" {
        mutate {
            add_tag => ["cat1"]
        }
    } else if [records][category] == "cat2" {
        mutate {
            add_tag => ["cat2"]
        }
    } else [records][category] == "cat3" {
        mutate {
            add_tag => ["cat3"]
        }
    } 
}

########## Output #########
output {
    elasticsearch {
        hosts => ["host:9200"]
        index => "index"
        manage_template => false
    }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2020, 1:16pm UTC](https://discuss.elastic.co/t/handling-events-after-using-split/235929/4 "2020-07-06T13:16:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.