Es 6.x not support multi types in index, but elasticsearch-hadoop docs not update

in this guide:

it recommend user to use es 6.x version which not support multi type, but still guide user to set multi-type in one index

val game = Map("media_type"->"game","title" -> "FF VI","year" -> "1994")
val book = Map("media_type" -> "book","title" -> "Harry Potter","year" -> "2010")
val cd = Map("media_type" -> "music","title" -> "Surfing With The Alien")

sc.makeRDD(Seq(game, book, cd)).saveToEs("my-collection/{media_type}")  

of cause, above code will fail:

rg.apache.spark.util.TaskCompletionListenerException: Found unrecoverable error [] returned Bad Request(400) - Rejecting mapping update to [my-collection] as the final mapping would have more than 1 type: [game, book]; Bailing out..
	at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:138)
	at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:116)

I moved your question to #elasticsearch-and-hadoop.

I agree. It's a bug (documentation bug). Would you like to open an issue in project?

I think you should call:

sc.makeRDD(Seq(game, book, cd)).saveToEs("my-collection-{media_type}/doc")

Agreed, we'll update the docs.

I have open an issue #1100 on

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.