Why do I get null pointer errors when sorting by a scripted field and increasing the size in a terms aggregation?

[patrickian]

I got a weird error while sorting with a scripted field. I have something like

"agg_one": {
                  "terms": {
                    "field": "application.keyword",
                    "size": 100,
                    "order": {
                      "_count": "desc"
                    }
                  },
                  "aggs": {
                    "total_time": {
                      "sum": {
                        "script": {
                          "inline": "doc['action_end'] - doc['action_start']",
                          "lang": "expression"
                        }
                      }
                    },
                    "threads": {
                      "terms": {
                        "field": "thread.keyword",
                        "size": 100,
                        "order": {
                          "total_time": "desc"
                        }
                      },
                      "aggs": {
                        "total_time": {
                          "sum": {
                            "script": {
                              "inline": "doc['action_end'] - doc['action_start']",
                              "lang": "expression"
                            }
                          }
                        }
                      }
                    }
                  }
                }

Under the threads aggregation, it's sorted by total_time, which is computed by script. If I do it like that, I get null pointer errors. Now, when I change it to sort by _count: "desc", it works. What could the error there be?

Also, in another instance, If i change the size from 100 to 1000 in the threads aggregate, It also throws null pointer errors. Again, why could that be? I have aggregates that have less than 100 but it still works fine when the size is set to 100. Advanced thanks to anyone who could shed some light as to why I'm getting these errors. Thank you.

Elasticsearch version is 5.5.0 by the way.

[rjernst (Ryan Ernst)]

Do you see a stack trace for the null pointer in your logs? If not, can you run your query with error_trace=true (see https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#common-options-error-options).

[patrickian]

I added "missing: 0" in the total_time aggregations and the error went away. I guess there were records not having the total_time aggregation?