Removal of types in es 6


(Ninad Pradhan) #1

Am I the only one feeling that es keeps making these fundamental level changes affecting everyone?

If they thought the analogy of tables and databases was a bad one, why did it take them like a thousand releases to know this. Architecturally speaking this is so poor not to have thought of this earlier.

When was the last time in an established db like oracle or any such database, or any such system did we have to remodel our data like this? Granted these are very different, but just for arguments sake Im giving this example.

This is absolutely ridiculous!!


(Mark Walkom) #2

Do you have suggestions for how we can approach these types of changes?


(David Pilato) #3

If they thought the analogy of tables and databases was a bad one, why did it take them like a thousand releases to know this.

Because you learn from your community and customers. And this could take time to make a decision.

Architecturally speaking this is so poor not to have thought of this earlier.

What would have been the consequence of thinking of that one or two years before? Same "pain" for existing users I'm afraid.
But as far as I recall we have been advertising that a lot the last years that using multiple types should be avoided. At least at conferences, at our trainings and on this forum.

When was the last time in an established db like oracle...

Oracle has been there for 40 years I think. And indeed this is pretty much stable now.

Note that you can still use the same "type" model you were using before the change. As explained in this blog post: https://www.elastic.co/blog/removal-of-mapping-types-elasticsearch

For example, if you had:

POST index/type1
{
  "foo": "bar",
  "type1": {
    "a": "text"
  }
}
POST index/type2
{
  "foo": "bar",
  "type2": {
    "a": 1234
  }
}

You can now index it as:

POST index/_doc
{
  "type": "type1",
  "foo": "bar",
  "type1": {
    "a": "text"
  }
}
POST index/_doc
{
  "type": "type2",
  "foo": "bar",
  "type2": {
    "a": 1234
  }
}

And you will have a similar behavior as what you had. But now you can understand exactly the consequences of having multiple types per index.

This is absolutely ridiculous!!

Even though I understand your frustration, I don't think it's ridiculous. It has been a hard decision to take to be honest but sometimes you must take decisions instead of letting users shooting a bullet in foot. I'm sorry you have to suffer from it and we will be happy to help you on this forum to move forward.


(Yeikel) #4

Do you have suggestions for how we can approach these types of changes?

I am not sure if the idea of "deprecation warnings" exists in ES, but in technologies such as Java, some type of warning is issued in a lower version when the plan is to remove a feature in a future version.

I think that having a warning about that in ES 5.0 at index time could have prevented people from continuing using it. The same could be done with other features.

The only time we discovered this change is when we went to the release notes of 6.0 when we starting planning to upgrade from 5.0. We built our search API around 5.0 and the idea of using multiple types and now we need to go through a major refactoring phase of our code if we wish to upgrade.


(David Pilato) #5

Did you try with elasticsearch 5.6 latest?
I think that warnings should be there.


(Yeikel) #6

I did not know this existed. Thank you


(swarmee.net) #7

I'm not sure if its of any help to others during their transition to single type indexes.

But it is still possible to create an alias across multiple indexes. So if you split your data by type into multiple indexes you can then alias the indexes together to search like it was one index.


(Ninad Pradhan) #8

Thanks for getting back. I just gave this as an example, also I certainly disagree that it would elasticsearch the community and customers experience for the something as fundamental as types, I just feel its overwhelming for a startup like ours to keep making such huge changes every year just for one component like elasticsearch, we rather have something that doesn't make us change our code so often and focus on our business problems. Yes oracle is very exaggerated example I gave just to drive the point. Also es has been around for about 8 years now, thats not very like just a year or something. I hope elasticsearch takes into account these frustrations and improves on it, so on every release customers dont have to make code changes.


(Ninad Pradhan) #9

Also not to mention, the support of type is being deprecated in es 7 and not supported in 8, so I don't know if its a good idea to use it now and change everything again. Better to bite the bullet and remove it now.


(David Pilato) #10

Probably because you don't imagine all the implications. But API wise that's the only to make that happen without making more mess.

BTW deprecation starts from 5.6 IIRC.
6.x can read 5.6 with multiple types but you can't create new index with multiple types in 6.x.
7.x will add support for ignoring the type parameter IIRC.
8.x will remove definitely the type parameter from all APIs.


(system) #11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.