Rashid, thank you for the extensive write up but unfortunately this doesn't solve my problem.
The problem isn't so much that there is no (null) data between intervals, there is data but the way it's graphed when the interval falls below 1m is wrong.
What happens in the case of the netflow data I'm collecting:
- I start a large download.
- Netflow sends the records for this large download every minute or so (actual time depends on various factors).
- So for example every minute there is a record that has out_bytes 50MB.
- BUT, in the meantime other flows are still coming in, for example web browsing etc. These records might only show 50KB of usage.
- When the interval gets below 1m I get spikes. I did a quick test and at 30s intervals it gets really bad, up to 50s is OK'ish. This is with scale_interval(1s).
I'm not sure about the logic behind the auto interval or what is causing the problem but if I could somehow tell Kibana to NOT use intervals shorter than 1m it might solve my problem.
Is it possible to set a minimum auto interval?
edit: Similar to this. https://github.com/elastic/kibana/issues/3787
Looks like that feature is not implemented yet though.