Logstash split function

Elastic Stack Logstash
1

Hi,
I have this tag fields with value

 Beige, Black, Blue, boohoo, Brown, cat:dresses, DZZ56895, Green, Grey, New Arrivals, Orange, Pink, Purple, Red, subcat:cocktail:cat:dresses, White, women, Yellow, z_global_id:39039

I want to split with "," and then with ":". If array count is more then one odd will be filed name and even will be field value.

I am not sure how because I new to logstash. Please any one can help me.

Thanks in advance.

2

You can use a mutate+split filter to convert a string into an array of strings.

It is unclear what you want to do with the fields containing a colon.

3

if split with ":" say cat:dresses it become array ["cat", "dresses"]. Then I want "cat" as field and "dresses" as value in ES document.

{
  name: "some product",
  cat : "dresses"
}

name will be other field pull along with this tag field.This how I want to save in elastic search.

Thanks I hope this time is more clear.

4

Once you split that you will have an array

[ "Beige", "Black", "Blue", "boohoo", "Brown", "cat:dresses", "DZZ56895", "Green", "Grey", "New Arrivals", "Orange", "Pink", "Purple", "Red", "subcat:cocktail:cat:dresses", "White", "women", "Yellow", "z_global_id:39039" ]

Are you sure that you want to turn that into

[ "Beige", "Black", "Blue", "boohoo", "Brown", { "cat": "dresses"}, "DZZ56895", "Green", "Grey", "New Arrivals", "Orange", "Pink", "Purple", "Red", { "subcat": "cocktail:cat:dresses", "White", "women", "Yellow", {"z_global_id": "39039"} ]

which is an array in which some entries are strings and some are hashes? I am uncertain that elasticsearch will even allow this (you may get a mapping exception), and doubt it will be useable in kibana.

You would have to write a ruby filter.

5

I just want to get this

{ "cat": "dresses"}

out of the tags. As I am new I think we have split with "," first then look for hash and then split again with ":" and then create fields and value. Which I am not sure how I can do it in logstash configuration file.

6

You will need to use ruby

    mutate { add_field => { "[@metadata][fields]" => "%{message}" } }
    mutate { split => { "[@metadata][fields]" => "," } }
    ruby {
        code => '
            event.get("[@metadata][fields]").each { |x|
                x = x.strip
                matches = x.match(/([^:]+):(.+)/)
                if matches
                    event.set(matches[1], matches[2])
                end
            }
        '
    }

will produce

        "cat" => "dresses",
"z_global_id" => "39039",
     "subcat" => "cocktail:cat:dresses"
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.