There are some unclear things in the documentation
At some point it says:
Note that multiple indices and/or types are allowed only for reading
But the next paragraph is titled "Dynamic/multi resource writes" and states that
For writing, elasticsearch-hadoop allows the target resource to be resolved at runtime by using patterns (by using the {} format), resolved at runtime based on the data being streamed to Elasticsearch. That is, one can save documents to a certain index or type based on one or multiple fields resolved from the document about to be saved.
Is seems like these two statements contradict each other. I'm still not sure is it possible to write to multiple indices or not.
Secondly, the example with timestamp also looks unclear. I think about resource as 'index/type' (please fix me if I'm wrong). In the example
# index the documents based on their date
es.resource.write = my-collection/{@timestamp:YYYY.MM.dd}
timestamp is a type right? But usually we separate indices by time, not data types. This seems wrong. I would expect timestamp to be a part of index:
# index the documents based on their date
es.resource.write = my-collection.{@timestamp:YYYY.MM.dd}/{media_type}
Is it valid resource? Will it work as expected (i.e. write to multiple indices)?
If it will work, then it would be nice to have an ability to explicitly pass an index for a document through it metadata (alongside with id)