Combine fields inside aggregation filter

Elastic Stack Logstash
1

Hi all.

I've been googling this for a couple of days and have not found any help, although I think this problem should be very easy to solve.

I'm using Logstash to import data from MySQL database and export it to Elastic. Everything is working as expected.

But now I want to "combine" the result of 3 fields into a new one, and I cannot figure out how.

From MySQL I get this structure:

  • Ticket
    • User
      • FirstName
      • SecondName
      • LastName

I would like to combine those 3 fields for each user into new field, let's call it 'completeName'.

I've configured a filter for results like this:

filter {
    aggregate {
        task_id => "%{ticket_id}"
        code => "
            map['id'] = event.get('ticket_id')
            map['name'] = event.get('ticket_name')
            map['issueTime'] = event.get('issue_time')

            map['assignedTo'] =  {
                'id' => event.get('assignedTo_id'),
                'userName' => event.get('assignedTo_username'),
                'firstName' => event.get('assignedTo_first_name'),
                'familyName' => event.get('assignedTo_family_name'),
                'secondFamilyName' => event.get('assignedTo_second_family_name'),
                'completeName' => **_WHAT HERE?_**			
            }

        ... more code here ...

            event.cancel()
        "
        push_previous_map_as_event => true
        timeout => 10
    }
}

First 3 fields come from ticket. AssignedTo is the user that holds the ticket, that is mapped to ES as Object inside ticket.

I am getting everything mapped correctly to ES, only missing 'completeName', that sholud be: FirstName FamilyName SecondFamilyName.

Thanks for help.

Regards.

1 Like
2

You can use the usual Ruby string concatenation methods. One example would be this.

            map['assignedTo'] =  {
                'id' => event.get('assignedTo_id'),
                'userName' => event.get('assignedTo_username'),
                'firstName' => event.get('assignedTo_first_name'),
                'familyName' => event.get('assignedTo_family_name'),
                'secondFamilyName' => event.get('assignedTo_second_family_name'),
                'completeName' => [event.get('FirstName').to_s, event.get('SecondName').to_s, event.get('LastName').to_s].join(' ')			
            }
1 Like
3

Thanks @paz for your answer. I will try it.

One more question. Does it matter if any of the fields is null or empty?

I mean, for example in Java if you try to concatenate 3 strings and one of them is null, you get an exception.

In this case, 'LastName' is null in many of the users...

Regards.

4

For this specific case, not really. Join handles nils gracefully (whereas other Ruby functions/operations might be more picky).
The only side-effect with null values would be having 2 join characters together in the final field (2 spaces in that specific case).

To avoid this, you can clear the array before performing join on it, like so:

            map['assignedTo'] =  {
                ...,
                'completeName' => [event.get('FirstName').to_s, event.get('SecondName').to_s, event.get('LastName').to_s].reject(&:empty?).join(' ')			
            }
1 Like
5

Great!!

Thanks.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.