Filtering IPs with Grok

Elastic Stack Logstash
1

Hello, I want to filter apache access logs before giving them to elastic search. I want to filter specific IPs and remove the lines where the IPs dont occur. But at the moment I am struggeling with the grok filter.

I am using Grok Debugger with the pattern %{IPV4:ip} which is able to get me one single IP from one input line. Is Grok Debugger able to work with multiple lines?

How do I search for specific IP(s) and remove the other lines?

Thanks a lot!

2

I am using Grok Debugger with the pattern %{IPV4:ip} which is able to get me one single IP from one input line. Is Grok Debugger able to work with multiple lines?

Yes, if the input lines are joined into a single Logstash event but that's not something you'll want to do with an access log. Process the lines one by one.

How do I search for specific IP(s) and remove the other lines?

Capture the IP address into a field. If you only have a small number of IP addresses to whitelist (say, single digit) you can just wrap a drop filter in a conditional that compares the IP address field against a fixed list of IP addresses, otherwise use a translate filter to look up the field with the IP address against the contents of a file.

3

Yes, it is possible with the mutliline codec provided built in. Check this out https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.