Ok @flash1293, thank you, very much. That really explained a lot to me, and I think that is not explicitly expressed in the documentation..
So to conclude with my question, I would like to sum up with what was struggling me and how did I test it.
To begin with, the first time I dug into this, I understood that roll up indices would just "compress" my raw data, and it would be available to be queried again. But I thought that the query could be done in the "Discovery" section as any other, missing the part of the documentation that says it is possible to do queries across roll up indices via API only
Searching both historical rollup and non-rollup data
The Rollup Search API has the capability to search across both "live", non-rollup data as well as the aggregated rollup data.
Without that in mind, in my head the logic was that roll up indices should have:
- the same amount of documents in the roll up index as the raw index but with a much smaller storage size
- data could be queried and filtered in the "Discover" section
All of that is WRONG, because:
- the amount of documents should be different from the raw index, cause you are setting a time bucket which will be the minimum range of time to your new roll up index, so for example in a 10 minutes bucket are saving a "set of documents" together from the raw index (they are aggregated by the time bucket interval you set).
- data can not be filtered and queried from the Discover section (thanks to @flash1293 for pointing that out), but you could do it from the API if you needed to
So even though I have this documents count:
When I create the same visualization with different indices source in, the output would be the same:
The only thing that I couldn't modify in the green visualization, is setting a timestamp per less than 10 minutes, cause that is the minimum time interval when i created the roll up index in this example.
Thank you so much!