Aggregate filter plugin does not create nested data in Elasticsearch

Elastic Stack Logstash
1

Hello, everyone. I've been using Logstash to move data from a MySQL server to Elasticsearch. It's been great so far, but I have run into some trouble when using the aggregate filter. Here's the data I'm using:

+----+----------------------------+----------------------+
| id |  name                      | email                |
+----+----------------------------+----------------------+
|  1 | Name_1                     | email_1@email.com    |
|  2 | Name_1                     | email_2@email.com    |
|  3 | Name_2                     | NULL                 |
|  4 | Name_3                     | NULL                 |
+----+----------------------------+----------------------+

Here's the Logstash filter

filter {
    aggregate {
        task_id => "%{id}"
        code => "
            map['id'] = event.get('id')
            map['name'] = event.get('name')
            map['emails'] ||= []
            map['emails'] << {'email' => event.get('email')}
        "
    }
}

The expected result that I would like to get is:

{
    "id" => 1,
    "emails" => {
        "email" =>  "email_1@email.com",
        "email" =>  "email_2@email.com"
    }
    "name" => "Name_1"
}
{
    "id" => 2,
    "emails" => {
        "email" =>  "null"
    }
    "name" => "Name_2"
}
{
    "id" => 3,
    "emails" => {
        "email" =>  "null"
    }
    "name" => "Name_3"
}

However, the result that I actually got was:

{
    "id" => 1,
    "email" => "email_1@email.com",
    "name" => "Name_1"
}
{
    "id" => 2,
    "email" =>  "null"
    "name" => "Name_2"
}
{
    "id" => 3,
    "email" =>  "null"
    "name" => "Name_3"
}

Any help and pointers on where I did wrong would be much appreciated. Thanks.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.