I installed Elasticsearch 6.0.0 and Kibana 6.0.0 on a new machine and tried to run a C# project built with NEST 5.5.0 on it. I then got the following error when trying to index a 2nd type against the same index:
Rejecting mapping update to [index_name] as the final mapping would have more than 1 type: [2nd type, 1st type]"}]
However, the same project runs fine with Elasticsearch 5.5.2.
My questions are:
Is there a related breaking change in Elasticsearch 6.0.0?
Should I get and build with NEST 6.0.0? If so, how should I get it?
I issued the "Install-Package NEST" command in the Nuget Package Manager in Visual Studio, but still only NEST 5.5.2 was available. Does that mean the NEST compatible with Elasticsearch 6.0.0 is not available yet?
So because I have to use NEST, should I still stick to Elasticsearch 5.5.2 for the time being?
NEST 6.0.0 is actively being worked on, with a view to releasing in the next couple of weeks (so long as no showstoppers are found).
Yes, a single index created in 6.0 can only have a single type within it.
If you can't wait for NEST 6.0.0 to be officially released to Nuget, you can grab a canary CI build from MyGet. Canary builds contain the latest in master* and automatically start from the next minor version of the version found in that branch.
That's correct. NEST 5.x is not compatible (not tested or supported) with Elasticsearch 6.x. Major versions of the client are compatible with major versions of Elasticsearch, so it's recommended to keep the client up to date with the latest release for the major version you're using. The client is backwards binary compatible within a major version.
This is entirely up to you, but if you are starting greenfield, you may want to start with Elasticsearch 6.0 since there are many improvements to take advantage of.
I have a question what do you mean by a single index created in 6.0 can only have a single type with in it? Do you mean a single index can only have documents that have one type like logstash sends syslog documents into index A and types them syslog. I also send a document to index A but of type dog, but it won't work?
@forloop Hey Russ, I just sent a message via NuGet around this subject and some of my questions are answered above. I'd love to go directly to 6.0 (we're coming from 1.7.5) and I've done all the mapping changes (ie. single index per type, etc.) and successfully used "reindex from remote" w/ co-located instances of each ES version. I've even tested code changes using the NEST 5.5 and at least what we use in our web app is compatible enough query DSL-wise that it seems to be working quite nicely. However, we need to use "file" based scripts in our WebJobs which I could easily convert to stored scripts w/ 6.0 but I need to make a decision for our mid-January sprint roll. Do I try to use ES 5.x for our upgrade or wait for NEST 6.0 to be released? Just looking for guidance upon which we could make our decision.
Hey @Thomas_Doman, sounds like you're doing all the right things to make the transition from Elasticsearch 1.x to 6.0!
That's a tricky decision to make and ultimately I can't make it for you () , but let me see if I can provide some further details to better inform how you might wish to tackle it.
We are very near to being happy enough with the .NET client to make a public release, and are looking at a weeks time frame as opposed to months.
One of the challenges faced as a strongly typed client is that major versions are the only opportunity that there really is to make breaking changes, so we want to ensure that we have made all of the appropriate changes before making the first public release of a new major version. What I mean by breaking changes here includes:
Ensuring that the client APIs align with the Elasticsearch APIs and any associated changes to the public surface
Correcting any bugs from the previous major version that may have been remedied by patched workarounds
Our ultimate goal is to set users up for success with Elasticsearch when using the .NET client, and to make the transition from one major version to the next as smooth as possible. For the 6.x client in particular, we are looking at internalizing the use of JSON.Net as the serializer for a number of reasons, including
Removing the dependency on JSON.Net and thus avoiding conflicts with regards to major versions when incorporating NEST into other projects already using JSON.Net.
Creating a clear boundary between serialization conventions applied to NEST types and any conventions that you may have for your own types
Lays the foundations to look at alternative serialization libraries/techniques in the future e.g. async streams, ArrayPool<T>, Span<T>, etc.
If you're eager to start building against the 6.x client, then the MyGet canary releases are available to be able to start doing so. The canary packages there have advanced considerably since the alpha release on nuget, so using those would put you in much better stead for the official release. I should point out that as per all prereleases, we do not recommend using them in production.
Two questions worth asking are what the expected lifetime of the project that you're working on will be, and once in production, how easy will it be to upgrade Elasticsearch and consuming applications. I think a decision on how to proceed would be well informed by the answers.
@forloop Yep, that's all I could ask for and more, thank you very much, I appreciate your timely and thorough response.
I appreciate the caution w/ major version and breaking changes so we'll patiently but anxiously await the 6.x release. Definitely makes sense WRT the Newtonsoft library. In order to use NEST 5.5, we had to move from version 9 to version 10 which probably isn't a bad thing for us but indeed, another moving part that I could see making sense to eliminate.
As for our situation, we produce a SaaS offering (degreed.com) which dramatically increases our flexibility. That said, we'd love to go direct to 6.0 but having done the exercise so far, we're also in a pretty good position now if we had to go w/ 5.x first. I just need to verify that the "reindex from remote" functionality I've used so far is supported in 5.x. The fanciest things I've done so far are some _source "excludes" and one index where I needed a script to transform a boolean field from 0's and 1's to false and true. Oh, we've also used the "whitelist" for the 1.7.5 ES instance (ie. localhost) to allow the co-located destination ES 6.0 instance to re-index. I'm assuming it'll work but I'll try that out tomorrow so we know if that's an option. Is 5.6.4 compatible w/ NEST 5.5? Also assuming it is.
Thanks again, the Windows/.NET support you guys have given over the years has been exceptional.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
) , but let me see if I can provide some further details to better inform how you might wish to tackle it.
await the 6.x release. Definitely makes sense WRT the Newtonsoft library. In order to use NEST 5.5, we had to move from version 9 to version 10 which probably isn't a bad thing for us but indeed, another moving part that I could see making sense to eliminate.