Is it problematic to create a large number of indices?
Because the memory drain on my machine is growing with the number of documents I've placed in ES (I've logged some OutOfMemory errors lately), I am working on a purging strategy that is based on expiration dates. To create maximum flexibility in my purging schedule, I am creating a separate index for each calendar date and document source. Indices are named like "2011-11-08-twitter"; if twitter is scheduled for a weekly purge, then I would delete this index on Nov. 15.
I can see this means I'll have about 175 indices at any given time (25 sources * 7 days). And the number will grow as I add more sources. I considered subdividing the indices into separate types according to source name, but then I'd be forced to purge using the DELETE BY query, which others have advised against. I want to simply delete an entire index.
Do I have anything to worry about?