Why types have been removed in ES 6.0?

[MaKro (Marek K)]

Hi.

I'm wondering why types have been removed completely in ElasticSeach 6.0 ?
I have read that they have been removed, because users were using elasticsearch like databases and also there were namespace issues in mappings. I understand that, and I agree that it's not the best situation.
However types had one big advantage - common score calculations.

In more than one of my projects, we used ElasticSearch for site search. We had different types of content, which was searchable. Size of the data wasn't big, it was less than 100 000 documents, so in the world of search engines it's not a big ammount of data (one document was a one article, so you can imagine, that data which was kept in single document was relatively small). It won't grow in next few years, I'm pretty sure of that.

So, because of the size of the data, we were using single index, with single shard and two replica shards. Each document had a concrete type, and we performed search to search across all types (or almost all, nevermind).
Results were sorted by score mainly, which is quite normal I think.
Thanks to having everything in one shard (in one index), scores were calculated exactly in the same way, because ammount of the data was exactly the same, so scores has been calculated in same way for each documents.

n ES 6.0 to achieve the same situation, we would need to add custom field 'type', as it is suggested in docs.
If I would pick ES 6.0 way to structure the data, I assume that I would need to split each type to separate index, and because of that my scores would not match the reality. For example I have 10000 documents with type article, 500 of them match phrase 'london' multiple times. and I have 100 documents with type car, 10 of them match phrase 'london' the same ammount of times. Because there's only 100 documents score will be completely different.

I'm wondering why decision about removing types has been made, because from my point of view it could stay, and it should be clearly told what are limitations and issues with using types. I'm not a search engine pro, so probalby I don't know something that you guys know.

[dadoonet (David Pilato)]

@MaKro

we would need to add custom field 'type', as it is suggested in docs.

That's basically what types are providing nowadays. So you can totally reproduce the old behavior with a type field instead of a _type field.

That way you can understand exactly the internal behavior of types. For example that's why having 2 fields with the same name but different settings in different types can not be supported.