I am parsing xml documents but there are different types with different names. Each type of document has different tags so each document will require different xpath expressions. What I want to do is write a conditional statement that basically says, if the source filename from the source has this structure, then use these xpath expressions, else if the filename has a different structure, use these xpath expressions. Is this possible in logstash?
If the [source] field contains the file path and name then you could use
if [source] =~ /foo.xml/ {
xml {
[...]
}
} else if [source] =~ /bar.xml/ {
xml {
[...]
}
}can I use a regular expression to describe the pattern of the filename instead of saying foo.xml?
Yes. Everything inside the / and / is matched as a regexp.
It seems to not be working. Does the regular expression need to be in quotes? Does it need to be like /'regular expression'/? because I cant get it to catch anything even through the expression does work.
The regexp should not be quoted in either single or double quotes, just wrapped in //
Ive tried multipple but I cant get it to work. it keeps skipping the first if statement. My regular expression is correct.
What does your if statement look like and what does the event look like?
It's an xml document and I am just trying to test the file name. There is sensitive information in the regular expression so I will replace any of that with "some word" or "some expression"
if [source] =~ /someword_\d{2,3}_someword\d{1}_CH\d_\S{1,3}\d{0,1}_GO-\d{10}_\d{0,2}-\D{3}-\d{4}_\d{6}.xml/ {
xml {
source => "message"
xpath =>
[
Some expression that is correct.
]
store_xml => false
force_array => false
}
}The if statement looks OK. If I run
input { generator { count => 1 lines => [ '' ] } }
filter {
mutate { add_field => { "source" => "someword_12_someword0_CH1_AbC_GO-0123456789_-;;;-1234_123456.xml" } }
if [source] =~ /someword_\d{2,3}_someword\d{1}_CH\d_\S{1,3}\d{0,1}_GO-\d{10}_\d{0,2}-\D{3}-\d{4}_\d{6}.xml/ {
mutate { add_field => { "matched" => true } }
}
}
then the [matched] field does get added.
even if I use the exact file name instead of the regular expression, it still doesnt catch and doesnt parse throught the xml as if the conditional failed. Also, for some reason, when I review my results in kibana, the source field that usually contains the file name isnt showing up.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.