What are best practices for matching incoming log sources?

[meatwad]

Hi,

I'm getting started on sorting incoming log sources and was wondering what best practice is for doing so.

For example, I'm collecting logs from a variety of sources including Palo Alto firewalls, Aruba NAC, PulseSecure VPN, etc. Right now, I'm matching "TRAFFIC", "SYSTEM", and "THREAT" strings in an if statement like:

if "THREAT" in [message] {
  mutate {
    add_tag => "PA_THREAT" }
}

But I don't know if there are performance penalties to looking for strings in each message. Should I think about doing it a different way?

Thanks!

[meatwad]

Any help on this would be greatly appreciated!

[Badger]

If you want to distinguish events within logstash you are going to have to use a conditional that tests some field on the event. It might be message, it might be a tag added at the origin, it might be a [host][name] added at the origin. It is going to depend on the data and we do not know what your data looks like.

[meatwad]

Thank you for the response. Yeah, it obviously varies on the log source -- I just didn't know if matching by 'if "foo" in [message]' was more expensive resource-wise than other ways, such as by source host IP.

Thank you again -- I really do appreciate your help!