Split field and reverse join


#1

I am trying to parse appache accesslogs with logstash for a project.

To let elasticsearch search efficiënt I want to reverse the domainname. So for example:

www.example.com Becomes com.example.www

I tried to split the domainname and reverse that using the logstash mutate plugin. When parsed as is I get a field

domainname : "www.example.com"

Using settings below I get the following results:

Setting:

mutate{
    split => {"domainname" => "."}
    add_field => {"reversed_domainname" => ["%{[domainname][-1]}","%{[domainname][-2]}"]}
    join => {"reversed_domainname" => "."}
}

Result:

domainname = [ "www", "example", "com" ]

reversed_domainname =[ "com", "example" ]

Everything works as intended untill the join function as seen in the results i dont get com.example

If I remove reversed from the join (split the domainname on "." and join them with "." I get the same results.

www.example.com becomes www.example.com

How can I join the fields in reversed order as it clearly should work.


(Magnus Bäck) #2

To let elasticsearch search efficiënt I want to reverse the domainname.

You should probably use the pathname analyzer on the ES side instead.

Contrary to popular belief, the options in a mutate filter are not executed in the order specificed; the order is rather fixed:

Any add_field option executes that the end like for other filters. If you have mutations that depend on each other you need to use multiple mutate filters.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.