Sorting on Integer field whose values are not the same length

Elastic Stack Elasticsearch
1

Hello All,
My elastic search index has about 325K records in it of data I've crawled over time and is no longer available publicly. I have an integer field (So i can use range) containing the items price. I'm having an issue when sorting as I get 1, 111, 21, etc. as my results and obviously not ordered by the actual value representation. I've researched this and it seems the little bit of information I've been able to find recommends to reindex the data with leading zeros to get all the values to the same length. Is there a simpler way to do this and if not has anyone written a script that could be used to bulk update this field in every record?

2

@bswingle for integers, you should not be seeing sorts that take length into consideration: it's an integer vs text-based sort. Are you sure the field is mapped as an integer? Sharing your mapping could help confirm: GET <index_name>/_mapping

if you are trying to change an existing field type, the easiest way is to do a re-index operation in combination with an Ingest pipeline. This Stack Overflow answer shows how it'd work for a field re-name. In your case, the Convert or Script processors might be the most appropriate choice.

1 Like
3

Mike,
It appears you are correct it looks like my mapping is text so i'll need to run that re-index process. I'll take a look at that example and come back if I have any questions. Thanks for the quick reply and all the guidance!

"price": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
},
"fielddata": true
}

Thanks
Brian

4

This worked perfectly but I ended up not needing the convert but here was my what I used with the convert in case it helps someone else!

PUT _ingest/pipeline/my_rename_pipeline
{
"description" : "describe pipeline",
"processors" : [
{
"convert": {
"field": "price",
"type": "integer"
}
}
]
}

POST _reindex
{
"source": {
"index": "myindex"
},
"dest": {
"index": "mynewindex",
"pipeline": "my_rename_pipeline"
}
}

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.