Painless sorting removing documents from results

[santiaguzz]

Hi there;

I'm trying to query three different indexes at a time, one of them have two mappings, as follows:

price: {
          type: 'float'
        },
        offer_price: {
          type: 'float'
        }

The next one have only one important mapping for prices:

 price: {
          type: 'float'
        }

And the last one have no prices mapped. when I query elasticsearch for the three indexes at a time the results are properly returned and filtered. However I would like to sort the results the following way: If there is an "offer_price" present take this value, else if there is a "price" present take this value, otherwise return 0. I'm trying to use the following painless script to that purpose:

 _script: {
          type: 'number',
          script: {
            lang: 'painless',
            source:
              "if(!doc['offer_price'].empty && doc['offer_price'].value != 0) { return doc['offer_price'].value; } else { if(!doc['price'].empty) { return doc['price'].value; } else { return 0; }}"
          },
          order: "asc"  //or "desc" (this works fine for the returned docs)
        }

When running the query only documents with the "offer_price" field mapped are returned, documents from the two other indexes are missing in the return set. I tried with a ternary operator with only one statement with no luck. Can anyone point me in the right direction please?

Thanks in advance.

Szz.

EDIT: Tried with:
_script: {
type: 'number',
script: {
lang: 'painless',
source:
"if(doc['offer_price'] != null && doc['offer_price'].value != 0) { return doc['offer_price'].value; } else { if(doc['price'] != null) { return doc['price'].value; } else { return 0; }}"
},
order: "asc"/"desc"
}

With no luck neither.

P.D: using elastic search 7.2.0

[rjernst (Ryan Ernst)]

If you check the response from elasticsearch, do you see a section about shard responses? Something like this:

"_shards":{
  "total" : 3,
  "successful" : 3,
  "skipped" : 0,
  "failed" : 0
},

In that, do you have failed shards? Based on your description, I will bet painless is blowing up with an exception like "No field found for [offer_price] in mapping" and causing all the requests to shards for your two indexes that do not have offer_price to fail. You need to check if doc has the field in question in the mappings, not check for null. To do this, use doc.containsKey('offer_price').

[santiaguzz]

Hi Ryan;

Thanks for your response, as you said there are failed shards in the response, didn't realize so since there is no error thrown!

shards:
{ total: 3,
successful: 1,
skipped: 0,
failed: 2, [...] }

Using doc.containsKey('offer_price') as you suggested the documents are properly returned and sorted.

Thanks again!