Easily parseable log format (ideally, CEF format) in ELS / Kibana / Fluentd

Hi there, I'm new to this stack and trying to evaluate it as an option for my app's log collection and visualization. I've got Kibana/ELS/Fluentd setup on Kubernetes. Extracting useful information from my logs on Kibana has proven difficult -- all non-Kubernetes/Docker fields are dumped into a "log" field, which isn't parsed/is hard to split apart for visualization purposes. Ideally, I'd like to be able to have my log fields parsed, such that I can, for example, easily search by log level, e.g. INFO, WARN, ERROR, FATAL. With the current setup, that isn't very easy to do in a Lucene-like way. I'm investigating using a standard logging format, like CEF, with the hopes that I can use a plug in to get my logs parsed appropriately. Logstash and fluentd both seem to have CEF plugins, but I'm having trouble finding documentation that links all of these disparate pieces together. Any help would be appreciated. Thanks.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.