Note for scripted fields
Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on Kibana performance. Keep in mind that there’s no built-in validation of a scripted field. If your scripts are buggy, you’ll get exceptions whenever you try to view the dynamically generated data.
This can be achiedved like this :
Example of index
PUT my_index
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"call_duration_ms": {
"type": "long"
},
"call_duration_h": {
"type": "long"
}
}
}
}
Example of document ingested
POST my_index/_doc
{
"@timestamp": "2020-09-06T09:48:10.000",
"call_duration_ms": 7200
}
This is an example of search query that add a scripted field on the flight during the search (the scripted field can be added in kibana on the index pattern used with the code : doc['call_duration_ms'].value / 3600.00
GET my_index/_search
{
"_source": [
"@timestamp",
"call_duration"
],
"script_fields": {
"duration_h": {
"script": {
"source": "doc['call_duration_ms'].value / params.factor",
"params": {
"factor": 3600.00
}
}
}
}
}
I would suggest an ingest pipeline as it intercept documents and compute the field then ingest ...
PUT _ingest/pipeline/my_index
{
"description": "this is an example to get duration in hour",
"processors": [
{
"script": {
"lang": "painless",
"source": "ctx.call_duration_h = ctx.call_duration_ms / params.factor",
"params": {
"factor": 3600.00
}
}
}
]
}
The ingest pipeline can be trigged when ingesting documents like this :
POST my_index/_doc?pipeline=my_index
{
"@timestamp": "2020-09-06T10:48:10.000",
"call_duration": 9200
}
Or it can setted up in the settings of your index
PUT my_index
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"index.default_pipeline": "my_index",
"index.final_pipeline": "foo"
},
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"call_duration_ms": {
"type": "long"
},
"call_duration_h": {
"type": "long"
}
}
}
}
If you are using logstash as ingest tool, i prefer using a ruby filter to that calculation