I've been struggling for weeks to create a vizualization with dynamic labels : extracted from a a search request.
( we dont want to create a new field in logstash for some errors that occurs with a very low frequency ...we want to extract information at Vizualization creation )
I tried to use Timelion but unsuccessfully....
Is there any way to fullfill this ? It seems impossible.
Kibana currently does not provide a way to extract values from raw messages ?
The feature exists in splunk ... and is very easy to use.
Elasticsearch supports runtime scripts. In Kibana if you go to the index patterns page -> scripted fields -> add you can extract text into a new field. These will behave as if the field was indexed in Kibana.
There's a runtime performance hit for every script, it has to be calculated for every document. Performance hit may be negligible if the search size is small.
I 've already tried to test scripted fields as follow with a simple regex working in any regex tester
to extract information from string field msg :
test field:
def m = /Detected server\s+(\w+)started)/.matcher(doc["msg"].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}
and When i did a search in Discover, i had no mach at all :
Time test msg
Dec 23, 2019 @ 14:28:53.437 no match Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
Dec 23, 2019 @ 14:28:53.436 no match Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
Dec 23, 2019 @ 14:28:53.431 no match Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
Dec 23, 2019 @ 14:28:53.429 no match Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
So I enlarged the regex to include the whole log line as follow :
test field:
def m = /^(.*)$/.matcher(doc["msg"].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}
and When i did a search in Discover, i am matching just a word...; i really dont understand....
Time test msg
Dec 23, 2019 @ 14:28:53.437 BL2_500c_MBR2 Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
Dec 23, 2019 @ 14:28:53.436 BL2_500c_MBR2 Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
Dec 23, 2019 @ 14:28:53.431 BL2_500c_MBR2 Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
Dec 23, 2019 @ 14:28:53.429 BL2_500c_MBR2 Detected server BL2_500c_MBR2 started on node RefNode_REF_500B_EB
I just remind that we just wanted to use a dynamic label in a 1 vizualization : a script field is a new field....that we dont want...
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.