Extract string from a field in logstash

Elastic Stack Logstash
1

I have a field in my index as:
"url" : "http://myaddress:myport/mydoc-3.docx"
I would like to create a new field ("newid") and put the extracted value mydoc to it.
basically I want to capture the document name with extension and without the number after "-".

I tried to split the "url" by "/" and capture mydoc-3.docx. Then split mydoc-3.docx by "-" and capture mydoc in two separate split as below but with not much success.

filter {
mutate {
split => { "url" => "/" }
add_field => { "my_id" => "%{[url][3]}" }
split => { [url][3] => "-" }
add_field => { "newid" => "%{[url][3][0]}" }
}
}
But the second split doesn't work.

2

You could do it using grok.

grok { match => { "url" => "/%{WORD:newid}-" } }

3

Should it be under filter like below? I do need to keep my_id and I need to create a new field newid.
For some reason below doesn't work. Most probably I didn't put grok in the right place.

filter {
mutate {
split => { "url" => "/" }
add_field => { "my_id" => "%{[url][3]}" }
}
grok { match => { "url" => "/%{WORD:newid}-" } }

}

4

The grok would have to come before the mutate+split, since the split has the effect of removing the / that the grok refers to.

5

Thanks. It worked!
Now I have to do a couple of cleanup as below:
filter {
mutate {
remove_field => [ "url"]
convert => { "newid" => "integer" }
}
}
Can I do in the same logstash and in the same filter and mutate?
For some reason I could not and I had to create a second logstash to remove url field and convert newid to integer because it is really an integer.

6

You should be able to do that in the same filter section that the grok is in, after the grok, of course. What problem did you have when you tried it?

7

I was able to merge all to one filter. it is working.
So I use index1 as input and make index2 as output.
So far so good.
The next step (hopefully the last step) is to use index2 and add index3 to it as input and make index4 as output.
Currently I made this as a separate logstash.
So I have one logstash to do:
index1 ---->user filter and some staff --->index2
And the other logstash to do:
index1,index2----->user filter and some staff--->index4
Can I merge both logstash to one?

8

I tried to merge and it worked!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.