I am using the high level rest client (6.6.0) and an embedded node (for system testing my code). I know the embedded node is not supported but this is not a "live" system. It is a node that is spun up in my system tests then destroyed at the end.
I can start the node, create an index and index a number of documents.
I then try and run a "delete_by_query" command (using the low level client as the high level client is adding unsupported query parameters).
I get an error response with the exception as "invalid_type_name_exception" and the reason being "Document mapping type cannot start with '_' found: [_delete_by_query]"
As you know you can not run elasticsearch embedded as this blog post says.
Note that to run integration tests (not unit tests) you would probably prefer running that in something close to a production environment, like a real elasticsearch server instance. I shared some ideas about integration testing in this thread: In memory testing with RestHighLevelClient
Also this sample project shows how to use Elasticsearch Test Classes:
I will take a look further into the elasticsearch test cases as that looks interesting.
I like to write integration tests around an embedded client in my code to give me a level of assurance before testing against other environments. It gives quicker feedback IMO.
So i set up my code to use the Elasticsearch test classes and the nodes that they fire up. However i still get the same errors when running delete by query.
Is this the same problem as when using an embedded node and the module is not loaded?
What i would find really useful about having an embedded node is i can write a system level test for my code (using junit for example) which tests my code against a lifelike system without having to move out of my IDE. Then I can TDD my code to make that test pass, using all the features offered within the IDE (debugging etc). Once that passes i can then move to further integration testing using other environments.
Is this pattern possible using the docker for maven solution?
That's what I'm doing and what I proposed in my previous answer.
But you can't debug elasticsearch itself. Which I don't think you need. I never needed to debug a server.
I can see that would work, but to me it brings in an external dependency on Docker into my development process, and I'm running on Windows which makes things more complicated.
Also if you were pushing those tests through a CI pipepline your build agents would then also need Docker.
Is there anyway I can identify the module that provides delete_by_query and load it? Could I just find the plugin version of it and load that?
I think that jars for modules are not published anywhere so you would need to manually build them, add to your repository manager and use.
Speaking of CI, you can also install Docker on the machine where your ci is running, have an elasticsearch service running somewhere in your network or use an external service like cloud.elastic.co.
This is what I tried to showcase there:
(There are multiple branches - look at the last one. Note that the code is a bit old).
Oh I see. As it might be required by the High Level Rest Client for the Reindex API, it is also published.
I don't know TBH how long it will stay published. If the HLClient only uses at some point new classes for reindex request and response objects, I think it will not be published anymore. But that's just my thoughts.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.