Dynamic field creation using regex and scripted fields


#1

I'm wondering if scripted fields could help me with dynamically generating kibana fields.

A small subset of my logs follows the format:
message: "My awesome thing that I just did (10ms)"

I'd like to query for all logs where the message field begins with "My awesome thing", and then extract the duration and graph the results.

I could update logstash to add this magic field, but I'd much rather do this dynamically as the result set will be small and I don't think this is a typical use case.


(Rashid Khan) #2

Query time field extraction is unfortunately not currently possible in Elasticsearch.


(Kevin Reilly) #3

Could this be done using a function_score block and script_field?


(Rashid Khan) #4

Currently kibana only supports lucene expressions in script fields, for security reasons. Lucene expressions unfortunately only support numbers, we're working to bring string support to them


(C├ęsar Augusto Ribeiro) #5

I'm totally new to this but found this grok stuff. Seems that it can extract "new fields" like you want.


(Tanya Bragin) #6

It does, but it needs to happen in Logstash, which is a step you do prior to indexing in Elasticsearch.


(Benjamin Tan) #7

Does Elasticsearch 5.0 support this feature?


(system) #8