I'm splitting data in a Kibana graph (plotting multiple lines on one graph), and the splitting seems to slow everything down.
Question in short:
I'm mostly wondering if Kibana is just slow in general when splitting allot of data, or if I'm doing something wrong.
Question in long:
I'm working with results from elemental analysis, where I have two fields: "Element" and "Quantity". Element refers to chemical element (Fe, Si, Cr, etc...) and quantity is just the quantity in grams.
I want to plot this on a line graph, splitting the data on the "Element" field.
I'm able to do this by using the "Split series" bucket when creating the visualization, so I see 12 lines on the graph for 12 elements, but doing that makes the whole graph super sluggish. If I disable the "Split series" bucket, the graph shows just a single line and everything is super fast.
Here are my visualization settings:
I'd like to add that the number of hits from the Elasticsearch query are on the range 5.000 to 50.000 (depending on the source of the data and chosen interval). Does that sound like allot for Kibana to handle? Say that I'm graphing 12 lines for one year with 1 day interval, that should give me 12*365 = 4.380 dots on the graph. Is that to much?
My hunch is that I'm to blame 
and I'm making Kibana do to much work which might be done by Elasticsearch. And that I should design the visualization and the query in some way to offload work from Kibana. Does that make sense?
More detail
To elaborate a bit then here is an example of the data that I'm working with. Part of the data (showing only 3 elements) for two measurements would look like this:
Results from midnight 2020-01-01:
{
"@timestamp" : "2020-01-01T00:00:00.000Z",
"Element" : "Fe",
"Quantity" : 21,5,
},
{
"@timestamp" : "2020-01-01T00:00:00.000Z",
"Element" : "Si",
"Quantity" : 27.6,
},
{
"@timestamp" : "2020-01-01T00:00:00.000Z",
"Element" : "Cr",
"Quantity" : 31.0,
}
Results from midnight 2020-02-01:
{
"@timestamp" : "2020-02-01T00:00:00.000Z",
"Element" : "Fe",
"Quantity" : 22.3,
},
{
"@timestamp" : "2020-02-01T00:00:00.000Z",
"Element" : "Si",
"Quantity" : 24.5,
},
{
"@timestamp" : "2020-02-01T00:00:00.000Z",
"Element" : "Cr",
"Quantity" : 33.2,
},
The Elasticsearch query looks like this:
{
"aggs": {
"2": {
"date_histogram": {
"field": "@timestamp",
"calendar_interval": "1d",
"time_zone": "Atlantic/Reykjavik",
"min_doc_count": 1
},
"aggs": {
"3": {
"terms": {
"field": "data.Quantifications.Element.keyword",
"order": {
"_key": "asc"
},
"size": 62
},
"aggs": {
"1": {
"avg": {
"field": "data.Quantifications.Quantity"
}
}
}
}
}
}
},
"size": 0,
"stored_fields": [
"*"
],
"script_1fields": {},
"docvalue_fields": [
{
"field": "@timestamp",
"format": "date_time"
},
{
"field": "data.Measurement.MeasurementTimeIso",
"format": "date_time"
},
{
"field": "data.Measurement.MeasurementTimestampSeconds",
"format": "date_time"
}
],
"_source": {
"excludes": []
},
"query": {
"bool": {
"must": [],
"filter": [
{
"range": {
"@timestamp": {
"gte": "2019-06-04T21:35:47.627Z",
"lte": "2020-06-04T21:35:47.627Z",
"format": "strict_date_optional_time"
}
}
}
],
"should": [],
"must_not": []
}
}
}
And an the first lines from the response look like this:
{
"took": 33,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 14368,
"max_score": null,
"hits": []
},
"aggregations": {
"2": {
"buckets": [
{
"3": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"1": {
"value": 5.522025108337402
},
"key": "Cr",
"doc_count": 1
},
{
"1": {
"value": 13.914688110351562
},
"key": "Cu",
"doc_count": 1
},
{
"1": {
"value": 984.5307006835938
},
"key": "Fe",
"doc_count": 1
},
{
"1": {
"value": 119.0231704711914
},
"key": "Ga",
"doc_count": 1
},
{
"1": {
"value": 0.7600506544113159
},
"key": "Mg",
"doc_count": 1
},
{
"1": {
"value": 14.317068099975586
},
"key": "Mn",
"doc_count": 1
},
{
"1": {
"value": 79.67838287353516
},
"key": "Na",
"doc_count": 1
},
{
"1": {
"value": 41.67288589477539
},
"key": "Ni",
"doc_count": 1
},
{
"1": {
"value": 303.0051574707031
},
"key": "Si",
"doc_count": 1
},
{
"1": {
"value": 37.52043533325195
},
"key": "Ti",
"doc_count": 1
},
{
"1": {
"value": 112.84782409667969
},
"key": "V",
"doc_count": 1
},
{
"1": {
"value": 73.44461059570312
},
"key": "Zn",
"doc_count": 1
}
]
},
"key_as_string": "2019-06-06T00:00:00.000Z",
"key": 1559779200000,
"doc_count": 12
},
{
"3": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"1": {
"value": 7.688352108001709
},
"key": "Cr",
"doc_count": 1
},
{
"1": {
"value": 15.557666778564453
},
"key": "Cu",
"doc_count": 1
},
{
"1": {
"value": 1032.217529296875
},
"key": "Fe",
"doc_count": 1
},
{
"1": {
"value": 119.36128997802734
},
"key": "Ga",
"doc_count": 1
},
{
"1": {
"value": 6.439831256866455
},
"key": "Mg",
"doc_count": 1
},
{
"1": {
"value": 14.688204765319824
},
"key": "Mn",
"doc_count": 1
},
{
"1": {
"value": 82.25433349609375
},
"key": "Na",
"doc_count": 1
},
{
"1": {
"value": 41.55519485473633
},
"key": "Ni",
"doc_count": 1
},
{
"1": {
"value": 318.4305725097656
},
"key": "Si",
"doc_count": 1
},
{
"1": {
"value": 38.49009323120117
},
"key": "Ti",
"doc_count": 1
},
{
"1": {
"value": 113.08766174316406
},
"key": "V",
"doc_count": 1
},
{
"1": {
"value": 72.5600814819336
},
"key": "Zn",
"doc_count": 1
}
]
},
"key_as_string": "2019-06-07T00:00:00.000Z",
"key": 1559865600000,
"doc_count": 12
},
{
"3": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"1": {
"value": 7.434240341186523
},
"key": "Cr",
"doc_count": 1
},
{
"1": {
"value": 13.125242233276367
},
"key": "Cu",
"doc_count": 1
},
{
"1": {
"value": 963.45703125
},
"key": "Fe",
"doc_count": 1
},
{
"1": {
"value": 120.3698959350586
},
"key": "Ga",
"doc_count": 1
},
{
"1": {
"value": 9.350666999816895
},
"key": "Mg",
"doc_count": 1
},
{
"1": {
"value": 14.928293228149414
},
"key": "Mn",
"doc_count": 1
},
{
"1": {
"value": 73.5030746459961
},
"key": "Na",
"doc_count": 1
},
{
"1": {
"value": 43.860416412353516
},
"key": "Ni",
"doc_count": 1
},
{
"1": {
"value": 315.4002380371094
},
"key": "Si",
"doc_count": 1
},
{
"1": {
"value": 37.861541748046875
},
"key": "Ti",
"doc_count": 1
},
{
"1": {
"value": 111.94914245605469
},
"key": "V",
"doc_count": 1
},
{
"1": {
"value": 73.28097534179688
},
"key": "Zn",
"doc_count": 1
}
]
},
"key_as_string": "2019-06-08T00:00:00.000Z",
"key": 1559952000000,
"doc_count": 12
},
....
....
....
....
....