Data Loss or a chance

Elastic Stack Elasticsearch
1

Hi all,

my colleague mixed up the mappings in ES 2.1.0 for the same idnex. Since our server crashed a week ago we are not able to restart our server with the old data as ES throws the error:

. . . is defined as an object in mapping [. . . ] but this name is already used for a field in other types . . .

(for a million of entries)

Now I am in a dilemma: all proposed solutions are based on POST and GET using localhost:9200. But I can not start on ES on localhost:9200.

Although I tried by connection with Java Code using the "org.elasticsearch.*" Package. I end in the same result.

In this forum I did not find similiar solutions I am searching for.

Question: is there a way to read out the data? I just need the entries (reproducing the json) Maybe with Lucene? Or do I have to name it "Data Loss forever"

Any links, blogs, whatever are welcome.

Thanks in advance,

Henner

2

This is really unclear.

Could you explain exactly what you did and what you are trying to do ?
Don't you have a source of truth somewhere? I mean where are the data coming from?

3

Hi David,

thanks for your reply.

Just to explain: my colleague produced conflicting field mappings as descripted hier

As also described "Elasticsearch will not start in the presence of conflicting field mappings." This is exactly what happens: our server crashed a week ago, I tried to start ES with ./bin/elasticsearch, but ES complains about the mappings:

. . .
java.lang.IllegalArgumentException: [payload.response] is defined as an object in mapping [passformLogs] but this name is already used for a field in other types
. . .

Without a running ES I can not e.g. delete an index with

curl -XDELETE 'http://localhost:9200/myindex/'

or make all other proposed stuff.

At the moment we moved the old data folder to data_folder_bak and started a new one. But we need access to the old data.

Same for trying to read the old data using Java: it crashes the same way.

So the question is: is there any chance to reconstruct the old data.

Any links, blogs, whatever are very welcome.

Hope it is more clear,

Henner

4

That's what I don't get. How did he do that?
Did he send a "wrong" mapping that has been accepted by elasticsearch?

I mean that it should not be possible.

5

Hi David,

to be honest: I don't know how he got it to work. As I read it was possible to do such mappings in older ES versions, but since version 2.x it is not possible anymore. Fact is: it is hard to reproduce how it happened.

More interesting is: how can I get it to work.

Still looking for a solution, e.g. curl 'http://localhost:9200/_cluster/state/routing_table?pretty=true' shows UNASSIGNED Shards:

. . .
"0" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "EMwwUPoDR-GB6UdZ2fETEQ",
"relocating_node" : null,
"shard" : 0,
"index" : "eventlog",
"version" : 1985,
"allocation_id" : {
"id" : "X8eoBYYnQiKyinEZWQqP1Q"
},
"unassigned_info" : {
"reason" : "ALLOCATION_FAILED",
"at" : "2016-07-27T12:08:21.799Z",
"details" : "failed to update mappings, failure IllegalArgumentException[Mapper for [payload] conflicts with existing mapping in other types[Can't merge a non object mapping [payload.response] with an object mapping [payload.response], Can't merge a non object mapping [payload.request] with an object mapping [payload.request]]]"
}
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 0,
"index" : "eventlog",
"version" : 1985,
"unassigned_info" : {
"reason" : "REPLICA_ADDED",
"at" : "2016-07-27T11:36:01.810Z"
}
} ]
}

. . .

E.g. trying to fix UNASSIGNED SHARD like described on https://t37.net/how-to-fix-your-elasticsearch-cluster-stuck-in-initializing-shards-mode.html (command: curl -XPOST 'localhost:9200/_cluster/reroute' -d '{ "commands" : [ { "allocate" : { "index" : "eventlog", "shard" : "1", "node" : "Red Ronin", "allow_primary" : true } } ] }')

stops with the unassigned error.

Maybe there are a few more commands I can check.

Thanks for any advice,

Henner