Canvas & .monitoring-es-6*

[rog_NB (Roger Buchwalder)]

Hello all

I'm tring to get a simple graph of our Elasticsearch's Document count over time.
The Canvas Layer has 3 elements:

• Time filter
• Area Chart
• Debug

Time filter's only one setting (Column in Display) points to "timestamp" (without the @)
I've set it to "Last 24 hours"
.
Debug and Area Chart has the same settings in "Data"

I've tried to do the following:

filters
| esdocs index=".monitoring-es-6*" fields="timestamp, node_stats.indices.docs.count" sort="timestamp, asc" query="_exists_:node_stats.indices.docs.count"
| pointseries x="timestamp" y="node_stats.indices.docs.count"
| plot defaultStyle={seriesStyle lines=1 fill=1}
| render

Area Chart:
On the X-axis you see only about 30 secs (instead of 24h) and on Y-axis starts the top number by 1.00 and the bottom number is -1.00. And of course, I don't see any line, it's a blank white area.

Debug:
I see here clearly a timestamp in format like "2019-01-01T00:00:00.000Z" plus below a number in the billiions for the filed node_stats.indices.docs.count

Can anyone let me know, how I do a nice graph of these docs'counts?

thanks
rog

[Nathan_Reese]

esdocs returns individual documents from elasticsearch. Only the first 100 documents are returned by default. Most likely, your results only cover the first 30 seconds.

Use the visualization application, which will display aggregation results, to visualize large data sets.

[rog_NB (Roger Buchwalder)]

I would like to know what the daily data growth is like and thus also check whether data arrive and are processed evenly. The time span should be around 60 days. I want to use canvas, because then I can include the CPU load, bandwidth and disk usage in the overview page.
Well... does this work with canvas?
Thanks
roger

[Nathan_Reese]

You can use the "Elasticsearch SQL" data source to display aggregation results in Canvas. Checkout the Grouping section in An Introduction to Elasticsearch SQL with Practical Examples - Part 2. SELECT AVG(FlightTimeHour) is pulling aggregation results.

With aggregations, you can group your data in elasticsearch and calculate metrics. Then, only those metrics get returned to the browser. This is a scalable solution that will allow the visualization of large data sets.

[rog_NB (Roger Buchwalder)]

Thank you Nathan

Now I'll see the time as I whish.
I've added now the Data Table element besides the debug element.
In both elements I see exact the same data. Even on the graphs I see the correct time stamp - but the line is still on zero.
This time I tried with element Line chart.

This is the request:

filters
| essql
query="SELECT node_stats.indices.docs.count, timestamp FROM ".monitoring-es-6*" WHERE node_stats.indices.docs.count > 1 AND timestamp > '2019-01-08T08:00:00.000Z' AND timestamp < '2019-01-08T08:15:00.000Z' ORDER BY timestamp ASC"
| pointseries x="timestamp" y="mean(node_stats.indices.docs.count)"
| plot defaultStyle={seriesStyle lines=3}
| render

Do you have any Idea, why my Graphs shows only zero's?
I've tried this also with different fields - but without luck - always zero...

thank you
rog

[rog_NB (Roger Buchwalder)]

Oh, well, the problem is solved!
I had to update kibana:

yum update kibana
---> Package kibana.x86_64 0:6.5.2-1 will be updated
---> Package kibana.x86_64 0:6.5.4-1 will be an update

Now I do see the graphs.

Anyway, I couldn't find any information in the change logs that would have put me up to it. I only noticed this when I tried to recreate the same configurations in the lab - the new version was already installed there.

greetings
rog