Is there an Application for Semi-Automatic Grok Construction?

(Jeff Kubina) #1

As best I can tell there does not seem to be a tool or application that will in a semi-automatically, or in a semi-supervised manner, derive grok patterns from a large collection of log messages. I think such a tool would be really helpful in speeding up the ingestion of new sources of log messages. I think it would also help speed up the creation of charts in Kibana. Briefly, I would want the system to 1) statistically analyze millions of log messages, 2) cluster them based on the template used to generate the log messages, 3) suggest groks for each of the clusters, 4) have a user interface to test, analyze, and tune the suggested groks, and 5) have a way to automatically deploy and/or revert the new groks. Does anyone know of such an application?

(Mark Walkom) #2

There is, I can't comment on how effective it is though.

(Jeff Kubina) #3

Grok Constructor is for building a grok manually from a small set of log messages that are from the same template (cluster). A nice tool but not what I am looking for.

(Mark Walkom) #4

That's pretty much it at the moment.

(Jeff Kubina) #5

Ok, that's what I thought. I will post a RFC to see if there is much interest in developing such a tool/application.

(Mark Walkom) #6

I think there would be quite a lot of interest :slight_smile:

(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.