API backward compatibility policy?

Elastic Stack Elasticsearch
1

Hello,

What is the policy on ES client-server compatibility when it comes to
upgrades?
Example:

  • I have ES cluster version X.0.0 and am using its Java API to talk to it
  • There is a new version of ES out, version X.1.0
  • I (want to) upgrade the ES cluster to version X.1.0 (or Y.0.0 when it
    comes out)

At this point do I need to start using newer ES jars and change my
application because the API has changed?
Or can my app keep using the same Java API and the same old ES jars?
Of course, I can see if things compile, if there are deprecation warnings,
if tests pass, etc.

But I'm wondering more about the policy or goal - if I upgrade from X.0.0
to X.1.0 or from X.0.0 to Y.0.0, is the goal to guarantee apps using X.0.0
version of the Java API don't break?

How about the HTTP API?

Thanks,
Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html

--

2

Hi,

good questions, Otis!

I can only make some guesses from my own experience.

The ES version scheme how I interpret it is composed of three numbers

X.Y.Z

where

X has been always 0 (for "feature incomplete"?, hopefully soon 1 for
"feature complete"?)
Y a number denoting cluster and index format change (but not necessarily
API changes?)
Z patch number, not changing cluster/index format

So, when Z changes, an ES cluster won't need a migration.

For client/server and API compatibility, migration can be required at each
ES release, even on "minor" releases. Since 0.19.5, when compression was
introduced, TransportClient requires the Lucene jar of the Elasticsearch
cluster. So Lucene version changes are propagating now to all systems
having an ES Java client installed. They need to match. ES 0.19.9
introduced Lucene 3.6.1 e.g.

REST API (HTTP, for scripting client languages, Perl/Ruby/Python/etc and
curl) has been backward compatible (fortunately?), newer versions of ES
should always be usable from existing script language clients. Upgrades
should only required to use new features (but will this assumption always
be safe?).

I would love to see different subversion schemes under the ES main version
to enable a better planning of migration paths, for example

  • cluster version (cluster discovery, for node migration)
  • index format version (for data migration)
  • client / server version (for client migration)
  • REST API version (for script language client migration)

It would be very helpful for customer support if in each ES release notes
such subversions could be listed.

Best regards,

Jörg

On Sunday, August 26, 2012 11:02:06 PM UTC+2, Otis Gospodnetic wrote:

Hello,

What is the policy on ES client-server compatibility when it comes to
upgrades?
Example:

  • I have ES cluster version X.0.0 and am using its Java API to talk to it
  • There is a new version of ES out, version X.1.0
  • I (want to) upgrade the ES cluster to version X.1.0 (or Y.0.0 when it
    comes out)

At this point do I need to start using newer ES jars and change my
application because the API has changed?
Or can my app keep using the same Java API and the same old ES jars?
Of course, I can see if things compile, if there are deprecation warnings,
if tests pass, etc.

But I'm wondering more about the policy or goal - if I upgrade from X.0.0
to X.1.0 or from X.0.0 to Y.0.0, is the goal to guarantee apps using X.0.0
version of the Java API don't break?

How about the HTTP API?

Thanks,
Otis

Search Analytics - Cloud Monitoring Tools & Services | Sematext
Scalable Performance Monitoring - Sematext Monitoring | Infrastructure Monitoring Service

--

3

Largely Jorg answer is spot on. on 0.Y.Z, Y changes might require full cluster restarts and have same version (Java) client work with them. Z changes with same Y can coexist on the same cluster. As for API changes, Y might change the API (also for HTTP/REST), but special effort is always taken in trying to keep backward support.

On Aug 27, 2012, at 10:48 AM, Jörg Prante joergprante@gmail.com wrote:

Hi,

good questions, Otis!

I can only make some guesses from my own experience.

The ES version scheme how I interpret it is composed of three numbers

X.Y.Z

where

X has been always 0 (for "feature incomplete"?, hopefully soon 1 for "feature complete"?)
Y a number denoting cluster and index format change (but not necessarily API changes?)
Z patch number, not changing cluster/index format

So, when Z changes, an ES cluster won't need a migration.

For client/server and API compatibility, migration can be required at each ES release, even on "minor" releases. Since 0.19.5, when compression was introduced, TransportClient requires the Lucene jar of the Elasticsearch cluster. So Lucene version changes are propagating now to all systems having an ES Java client installed. They need to match. ES 0.19.9 introduced Lucene 3.6.1 e.g.

REST API (HTTP, for scripting client languages, Perl/Ruby/Python/etc and curl) has been backward compatible (fortunately?), newer versions of ES should always be usable from existing script language clients. Upgrades should only required to use new features (but will this assumption always be safe?).

I would love to see different subversion schemes under the ES main version to enable a better planning of migration paths, for example

  • cluster version (cluster discovery, for node migration)
  • index format version (for data migration)
  • client / server version (for client migration)
  • REST API version (for script language client migration)

It would be very helpful for customer support if in each ES release notes such subversions could be listed.

Best regards,

Jörg

On Sunday, August 26, 2012 11:02:06 PM UTC+2, Otis Gospodnetic wrote:
Hello,

What is the policy on ES client-server compatibility when it comes to upgrades?
Example:

  • I have ES cluster version X.0.0 and am using its Java API to talk to it
  • There is a new version of ES out, version X.1.0
  • I (want to) upgrade the ES cluster to version X.1.0 (or Y.0.0 when it comes out)

At this point do I need to start using newer ES jars and change my application because the API has changed?
Or can my app keep using the same Java API and the same old ES jars?
Of course, I can see if things compile, if there are deprecation warnings, if tests pass, etc.

But I'm wondering more about the policy or goal - if I upgrade from X.0.0 to X.1.0 or from X.0.0 to Y.0.0, is the goal to guarantee apps using X.0.0 version of the Java API don't break?

How about the HTTP API?

Thanks,
Otis

Search Analytics - Cloud Monitoring Tools & Services | Sematext
Scalable Performance Monitoring - Sematext Monitoring | Infrastructure Monitoring Service

--

--