Want to figure that how the mapping of an index gets changed on one env

Elastic Stack Elasticsearch
1

Hello,
I want to figure out that is there any chances that the mapping of the index may change.

As the ES queries n all was running fine on one env but sudden I'm getting error while searching.
And I checked that and got to know that while searching in the query the date is in string format(in the query) and in the mapping the parameter is in long so that's why while searching I'm getting numberformatException that's why I'm not able to get the data.
Ok now, that is ok. Got that the mapping is not proper but it was working fine previously then how the mapping get's changed...?
Is there any possibility of that..?
Want to know that.

2

It depends on how are you creating your indices.

Are you using time based indices?

If you are using time based indices, like creating one indice per day or month for example, and you do not have a template with your mappings, than the mapping of a field can change.

When you use dynamic mapping Elasticsearch will try to infer the mapping for a field on the first time it receives a document with the specific field.

3

No, not using the dynamic mapping.
The mapping is set to the index while creating the index.
But still trying to figure out that how that mapping get's changed.
Means on one env the mapping is right and on other env the mapping is different.
That's why on one env the ES search query is running and on other env not getting the data of the search due to numberformatexception

4

You need to provide some evidence like what is the mapping in the both environments, what is the expected mapping and how you are creating the mapping for the specific field.

Something is not right because mappings will not change, if you are setting the mapping while creating the index, this mapping will not change, so something was not correctly setting or the specific field is using dynamic mapping.

But as mentioned, you need to provide evidence of your issue.

5

yes.
This is the mapping of the index of the env which is not working
{
"apiman_metrics": {
"mappings": {
"properties": {
"apiDuration": {
"type": "long"
},
"apiEnd": {
"type": "long"
},
"apiId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"apiOrgId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"apiStart": {
"type": "long"
},
"apiVersion": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"attributes": {
"type": "object"
},
"bytesDownloaded": {
"type": "long"
},
"bytesUploaded": {
"type": "long"
},
"error": {
"type": "boolean"
},
"errorMessage": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"failure": {
"type": "boolean"
},
"failureCode": {
"type": "long"
},
"method": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"requestDuration": {
"type": "long"
},
"requestEnd": {
"type": "long"
},
"requestStart": {
"type": "long"
},
"resource": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"responseCode": {
"type": "long"
},
"responseMessage": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}

And this is the mapping of the other env of which the mapping is working

{
"apiman_metrics": {
"mappings": {
"properties": {
"apiDuration": {
"type": "long"
},
"apiEnd": {
"type": "date"
},
"apiId": {
"type": "keyword"
},
"apiOrgId": {
"type": "keyword"
},
"apiStart": {
"type": "date"
},
"apiVersion": {
"type": "keyword"
},
"attributes": {
"type": "object"
},
"bytesDownloaded": {
"type": "long"
},
"bytesUploaded": {
"type": "long"
},
"clientId": {
"type": "keyword"
},
"clientOrgId": {
"type": "keyword"
},
"clientVersion": {
"type": "keyword"
},
"contractId": {
"type": "keyword"
},
"error": {
"type": "boolean"
},
"errorMessage": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"failure": {
"type": "boolean"
},
"failureCode": {
"type": "long"
},
"failureReason": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"method": {
"type": "keyword"
},
"planId": {
"type": "keyword"
},
"remoteAddr": {
"type": "ip"
},
"requestDuration": {
"type": "long"
},
"requestEnd": {
"type": "date"
},
"requestStart": {
"type": "date"
},
"resource": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"responseCode": {
"type": "long"
},
"responseMessage": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"user": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}

here in the above mappings of an apiman_metrics index
the one which is working in that
"requestEnd": {
"type": "date"
},
"requestStart": {
"type": "date"
},

is in date

and
in the mapping of env which is not working
in that it is
"requestEnd": {
"type": "long"
},
"requestStart": {
"type": "long"
},

that's why it is giving exception as in the query the data is in string format
but previously it working on that env as well but suddenly it stopped and during analysis got this that mapping is different as I compared both env's

6

Yeah, but how are you setting the mappings? You didn't share how are you creating the index with the mappings already set.

The mapping is set to the index while creating the index.

Are you doing this in your application? How are you doing this?

7

The indices are created long back ago and there is no change in that means while creating index at that time only the mapping is set.
Currenlty there is no change in the index or it's mapping.

8

Then the mappings were created different on each environment, Elasticsearch mapping does not change after an index is created.

The mappings can be created using a template, making a request to create them or dynamically when Elasticsearch receives data for a field for the first time, if there is no mapping already for this same field, but after the mapping for a field is created, it will not change.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.