Extract a string from logstash path

[AshishC (Ashish)]

Hi Everyone,

I am using logstash to send raw data ELK. Logstash config input section reads as:
input {
file {
path => [ "D:/apache-jmeter-3.3/bin/Booking_Stress_Test.csv"]
start_position => "beginning"
}
}

Now, in Kibana, this path i.e. "D:/apache-jmeter-3.3/bin/Booking_Stress_Test.csv" is shown under the "path.keyword" field.

I can pull the entire path using Data Table Visualization. But I want to extract just the test name instead of entire path.

In this case, I want to extract and show sub-string "Booking_Stress_Test" only. Do I need to use JSON Input formatting? Any assistance is highly appreciated!

Many Thanks,
Ashish

[Kurt_S (Kurt Schraeyen)]

In Logstash, you could apply the grok filter on the path field to extract the filename. If I remember correctly there are patterns for files and paths in de default grok patterns.

[AshishC (Ashish)]

Hi Kurt,

Thanks for the quick response! Where in logstash should I use grok? And how it will show on Kibana Dashboard?

[Kurt_S (Kurt Schraeyen)]

In the filter section.

filter {
grok {
match => { "path" => " your pattern here" }
}
}

More info on how to do this can be found in the documentation:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html

[AshishC (Ashish)]

Hi Kurt, Thanks!

earlier I couldn't get your solution but yeah now I understand bit of grok and how it works. Many thanks for your response and solving the issue.

One more thing, I see grok extracts everything that matches the pattern. Would have been great if we can extract the group(s).

path => [ "D:/apache-jmeter-3.3/bin/PerfTest_ModuleName_1_1219.csv"]

grok {
match => ["path", "(?/(\w*).)"]
}

grok output : "/PerfTest_ModuleName_1_1219."
But I was expecting the string from group, by group I mean text enclosed in ( ). i.e. PerfTest_ModuleName_1_1219.

Something similar to left, right boundaries.

Thanks,
Ashish

[Kurt_S (Kurt Schraeyen)]

I think this pattern should work to match what you look for in the field "group"

grok {
match => ["path", "/%{WORD:group}.csv$"]
}