Indexing question.
I have rolling indexes (daily). Each index is capturing logging
information. I have a new requirement that adds in some analytics
information. The information is not log specific, it's just more
information that needs to be captured for reporting purposes.
That data comes in pre-aggregated.
A Rolling index on that analytics data seems to be wasteful to me since
it's generally 1-ish records per day. So I'm thinking I add another
mapping/type to my rolling logging index and bundle the analytics data with
it. Is that crazy?
If I need to run a report I would most likely need to run a date histogram
across 30 indexes, for a monthly analytics report for example, still crazy?
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Hello Adam,
On Mon, Jul 29, 2013 at 11:13 PM, Adam Venturella aventurella@gmail.comwrote:
Indexing question.
I have rolling indexes (daily). Each index is capturing logging
information. I have a new requirement that adds in some analytics
information. The information is not log specific, it's just more
information that needs to be captured for reporting purposes.
That data comes in pre-aggregated.
A Rolling index on that analytics data seems to be wasteful to me since
it's generally 1-ish records per day. So I'm thinking I add another
mapping/type to my rolling logging index and bundle the analytics data with
it. Is that crazy?
Not crazy, although it might be confusing, since those are not log-related.
If you want to separate, you can have just an index with the aggregated
data. Not sure how you want to query that data (together with logs or
not?), so maybe a separate index would be crazy 
If I need to run a report I would most likely need to run a date histogram
across 30 indexes, for a monthly analytics report for example, still crazy?
If you're talking about the aggregated data, 30 indices would mean 30-ish
records. So it should be fast no matter what. If it's about logs, it might
work. Although, if indices are small, you might be better off having weekly
ones instead of daily, or something like that.
Best regards,
Radu
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.