Sorry for the rapid-fire posts, but here's another topic I'd like to
bring up. As I'm working on a rapidly-evolving application with
ElasticSearch as a central component, we often need to make mapping
changes that cannot be applied directly via a push mapping.
Accordingly, we use index aliases (a most excellent feature, I must
say) to set up a migration framework.
Essentially, let's say I have a semantic index 'my_index'. At any
given time, 'my_index' will be aliased to a real index with a
timestamp appended, e.g. 'my_index-123456789'. If we need to push a
mapping change that can't be done in-place, we create a new index, say
'my_index-1319658279'. We perform a scan search over the old index,
writing the documents to the new index as we go. When that's done, we
re-alias 'my_index' to the newly created index, and delete the old
This works great, but it's not especially fast, even using bulk
writes. My guess is that a lot of the overhead is just moving data
over the wire via HTTP. Since the migration involves no business
logic, it seems reasonable that ES could expose functionality to do
the same thing locally.
Essentially, what I have in mind is API endpoints that expose:
- Copy a document from one index to another
- Copy the entire contents of one index to another
I'm happy to explore writing this sort of functionality as a plugin,
but I'm wondering if anyone has any thoughts on the idea.