I have an Elasticsearch cluster that is currently on version 6.3.2 and we are not yet ready to upgrade it (hopefully that comes soon!).
According the ES documentation, I should be able to use the Java clients from newer versions as long as I'm on the same major version:
The client must have the same major version (e.g.
2.x, or5.x) as the nodes in the cluster. Clients may connect to clusters which have a different minor version (e.g.2.3.x) but it is possible that new functionality may not be supported. Ideally, the client should have the same version as the cluster.
For a variety of reasons, my client code is currently using version 6.6.2 of the REST high-level client. Since both client and cluster share the major version of 6.x, I would expect things to work so long as my client doesn't try to use any functionality that is newer than 6.3.2. If I am misinterpreting the above statement on version compatibility, please let me know.
Unfortunately, I am not finding these versions to be compatible when I introduce the use of a composite aggregation. The client code is inserting a "missing_bucket": false entry into the composite aggregation sources. I believe that the missing bucket feature was only added in Elasticsearch 6.4, and my 6.3.2 cluster is rejecting it with the error:
org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=illegal_argument_exception, reason=[date_histogram] unknown field [missing_bucket], parser not found]
Is this a violation of the above-referenced client compatibility policy? What are my options for best working around this? Do I need to change all of my client code back to use 6.3 until I can get a full upgrade of the cluster planned? Are there other workarounds I can consider in the meantime?