Multiple split

Elastic Stack Logstash
1

Hello,

I Have a csv file which have 3 fields
field1 : ID
field2 : Company Name
field3 : Array of contacts (separator : | )
Theis field can be :

  • null (no contact)
  • one or more contacts

But each contact is also an array with :

  • firstname
  • lastname
  • email

Exemple :
11111;Company 1;
22222:Company 2;MARC,SMITH,marcsmith@company1.com
33333:Company 3;JOHN,LENNON,johnlennon@company2.com|RINGO,STARR,rstarr@company2.com

How can I load this csv file with logstash ?
If I put this in my load.conf :
mutate {
split => { "contacts" => "|" }
}
But I want the firstname, lastname, and email in separate fields ...
Thanks for your help

Sorry for my poor english.

2

You could use

    mutate { gsub => [ "message", ":", ";" ] }
    csv { columns => [ "column1", "column2", "contacts" ] separator => ";" }
    mutate { split => { "contacts" => "|" } }
    ruby {
        code => '
            a = event.get("contacts")
            if a.is_a? Array
                newA = []
                a.each { |x|
                    x = x.split(",")
                    newA << { "lastName" => x[0], "firstName" => x[1], "email" => x[2] }
                }
                event.set("contacts", newA)
            end
        '
    }

to get

  "contacts" => [
    [0] {
         "lastName" => "JOHN",
            "email" => "johnlennon@company2.com",
        "firstName" => "LENNON"
    },
    [1] {
         "lastName" => "RINGO",
            "email" => "rstarr@company2.com",
        "firstName" => "STARR"
    }
],
3

Thank you for this solution !
Very good. All is ok for me.

4

I have another question with this example :
For each company, I can have 0, 1 or many contacts
How can I find the total contacts on my index, wich is different of the total of companies ?
Thanks for your reply

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.