EDIT: Whoops, did not realize the single type indexes were ES 5.6!
We're trying to write an RDD (with Spark Streaming) to Elasticsearch with just an index and no type, however we keep getting:
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: invalid pattern given my_index/
However, when I add a type, it works fine. I am calling
saveToEs like this:
EsSparkStreaming.saveToEs(directStream, "my_index/my_type", Map("es.nodes"->"localhost", "es.index.auto.create" -> "true"))
From what I can remember, regular Spark batch jobs can do just an index but Streaming doesn't seem to support that. This is my dependency in my POM in case it matters:
<dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch-spark-20_2.11</artifactId> <version>5.5.0</version> </dependency>
Thanks in advance!