Documents with same IDs but of different types in the same index in ES6?

Elastic Stack Elasticsearch
1

Hey there,
Sorry guys, but that mapping types removal in ES6 is quite a pain in the ass to say the least!
As it's proposed in the docs I chose the "custom type field" option to store the different document types in the same index (creating the separate index for every document type is not an option for me), but there's a problem with that approach: documents of different types and with the same IDs overwrite each other. I tried making unique IDs artificially by concatenating document type and ID values, but no luck, because I couldn't map the ID field to anything other than integer or long type so it fails to save the documents this way. The data is coming from MySQL database, so IDs are regular integers. Am I missing something?

2

But how were you doing before to push your documents to the right type?

I mean instead of

PUT index/type1/1
{}
PUT index/type2/1
{}

Can't you do this?

PUT index/doc/type1_1
{}
PUT index/doc/type2_1
{}
3

As I said I tried this, but it didn't work, because I couldn't map ID field to the "text" type, somehow it ends up as integer or long in the mapping which is really weird.

4

Why? I mean what are you using to collect data from MySQL and index into elasticsearch?

5

It's for analytics purposes.

6

I said "What" and not "Why".

What is the tool/language/framework you are using to send data from your DB to elasticsearch?

7

Ah, right, sorry. Laravel 5.5, sleimanx2/plastic package with
elasticsearch-php 6.0, ES 6.2.2. I haven't tried the low level way
interaction yet though (like curl), will try that tomorrow.

8

Okay, it appears to be an issue in the sleimanx2/plastic package. For some reason it doesn't allow adding "text" or "keyword" field types to the mapping. Investigating further.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.