Runtime field not appearing in search results

I'm trying to set up a run time field to calculate the total of an e-commerce purchase. But there is something wrong with the way I'm declaring run time field, and it might have something to do with nested objects.

First I set up the index , the mapping and the runtime field like this:

PUT purchase
PUT purchase/_mapping
{
  "properties": {
    "items": {
      "type": "nested",
      "properties": {
        "product_id": {
          "type": "long"
        },
        "quantity": {
          "type": "long"
        },
        "product": {
          "properties": {
            "name": {
              "type": "keyword"
            },
            "product_id": {
              "type": "long"
            },
            "price": {
              "type": "long"
            }
          }
        }
      }
    },
    "purchase_date": {
      "type": "date"
    }
  },
  "runtime": {
    "items.total": {
      "type": "long",
      "script": {
        "source": "emit(doc['items.quantity'].value * doc['items.product.price'].value)"
      }
    }
  }
}

Then I post one test record lie this:

POST purchase/_doc
{
  "purchase_date": "2023-01-01",
  "items": [
    {
      "quantity": 2,
      "product_id": 1,
      "product": {
        "product_id": 1,
        "name": "pencil",
        "price": 1
      }
    },
    {
      "quantity": 4,
      "product_id": 2,
      "product": {
        "product_id": 2,
        "name": "stapler",
        "price": 5
      }
    }
  ]
}

When I do a GET purchase/_search, I get a result like this:

{
          "purchase_date": "2023-01-01",
          "items": [
            {
              "quantity": 2,
              "product_id": 1,
              "product": {
                "product_id": 1,
                "name": "pencil",
                "price": 1
              }
            },
            {
              "quantity": 4,
              "product_id": 2,
              "product": {
                "product_id": 2,
                "name": "stapler",
                "price": 5
              }
            }
          ]
        }

I was expecting a field order_items.total to be a sibling field to order_items.quantity. I've tried quite a few things but nothing seems to get the order_items.total field to appear.

What did I do wrong?

Hi @learningelastic

Read this answer, maybe dont work to nested fields.

Doc:

Use the fields parameter on the _search API to retrieve the values of runtime fields. Runtime fields won’t display in _source , but the fields API works for all fields, even those that were not sent as part of the original _source .

Thanks, you might be right that runtime fields don't work on nested objects. I just ran this search query:

GET purchase/_search
{
  "fields":["items.total"]
}

And I got this result:

{
  "error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "org.elasticsearch.server@8.6.1/org.elasticsearch.index.fielddata.ScriptDocValues.throwIfEmpty(ScriptDocValues.java:92)",
          "org.elasticsearch.server@8.6.1/org.elasticsearch.index.fielddata.ScriptDocValues$Longs.get(ScriptDocValues.java:110)",
          "org.elasticsearch.server@8.6.1/org.elasticsearch.index.fielddata.ScriptDocValues$Longs.getValue(ScriptDocValues.java:105)",
          "emit(doc['items.quantity'].value * doc['items.product.price'].value)",
          "                          ^---- HERE"
        ],
        "script": "emit(doc['items.quantity'].value * doc['items.product.price'].value)",
        "lang": "painless",
        "position": {
          "offset": 26,
          "start": 0,
          "end": 68
        }
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "purchase",
        "node": "VYLFHFZ3RtOtR_P8dlnx3w",
        "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "script_stack": [
            "org.elasticsearch.server@8.6.1/org.elasticsearch.index.fielddata.ScriptDocValues.throwIfEmpty(ScriptDocValues.java:92)",
            "org.elasticsearch.server@8.6.1/org.elasticsearch.index.fielddata.ScriptDocValues$Longs.get(ScriptDocValues.java:110)",
            "org.elasticsearch.server@8.6.1/org.elasticsearch.index.fielddata.ScriptDocValues$Longs.getValue(ScriptDocValues.java:105)",
            "emit(doc['items.quantity'].value * doc['items.product.price'].value)",
            "                          ^---- HERE"
          ],
          "script": "emit(doc['items.quantity'].value * doc['items.product.price'].value)",
          "lang": "painless",
          "position": {
            "offset": 26,
            "start": 0,
            "end": 68
          },
          "caused_by": {
            "type": "illegal_state_exception",
            "reason": "A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"
          }
        }
      }
    ]
  },
  "status": 400
}

If I change my mapping to delete the "type":"nested" line, then I run the same

GET purchase/_search
{
  "fields":["items.total"]
}

Then I do see

"fields": {
          "items.total": [
            2
          ]
        }

Which seems like elasticsearch just picked ONE record from the items array and multiplied product.price with the quantity.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.