What are best practices for matching incoming log sources?


I'm getting started on sorting incoming log sources and was wondering what best practice is for doing so.

For example, I'm collecting logs from a variety of sources including Palo Alto firewalls, Aruba NAC, PulseSecure VPN, etc. Right now, I'm matching "TRAFFIC", "SYSTEM", and "THREAT" strings in an if statement like:

if "THREAT" in [message] {
  mutate {
    add_tag => "PA_THREAT" }

But I don't know if there are performance penalties to looking for strings in each message. Should I think about doing it a different way?


Any help on this would be greatly appreciated!

If you want to distinguish events within logstash you are going to have to use a conditional that tests some field on the event. It might be message, it might be a tag added at the origin, it might be a [host][name] added at the origin. It is going to depend on the data and we do not know what your data looks like.

Thank you for the response. Yeah, it obviously varies on the log source -- I just didn't know if matching by 'if "foo" in [message]' was more expensive resource-wise than other ways, such as by source host IP.

Thank you again -- I really do appreciate your help!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.