Writing to multiple indices and documentation about it


(Ov7a) #1

There are some unclear things in the documentation

At some point it says:

Note that multiple indices and/or types are allowed only for reading

But the next paragraph is titled "Dynamic/multi resource writes" and states that

For writing, elasticsearch-hadoop allows the target resource to be resolved at runtime by using patterns (by using the {} format), resolved at runtime based on the data being streamed to Elasticsearch. That is, one can save documents to a certain index or type based on one or multiple fields resolved from the document about to be saved.

Is seems like these two statements contradict each other. I'm still not sure is it possible to write to multiple indices or not.

Secondly, the example with timestamp also looks unclear. I think about resource as 'index/type' (please fix me if I'm wrong). In the example

# index the documents based on their date
es.resource.write = my-collection/{@timestamp:YYYY.MM.dd}

timestamp is a type right? But usually we separate indices by time, not data types. This seems wrong. I would expect timestamp to be a part of index:

# index the documents based on their date
es.resource.write = my-collection.{@timestamp:YYYY.MM.dd}/{media_type}

Is it valid resource? Will it work as expected (i.e. write to multiple indices)?

If it will work, then it would be nice to have an ability to explicitly pass an index for a document through it metadata (alongside with id)


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.