# Using Debug.explain kills data nodes. (8.11.1)

**URL:** https://discuss.elastic.co/t/using-debug-explain-kills-data-nodes-8-11-1/348690
**Category:** Elasticsearch
**Tags:** painless, runtime-fields
**Created:** [December 6, 2023, 3:06am UTC](https://discuss.elastic.co/t/using-debug-explain-kills-data-nodes-8-11-1/348690 "2023-12-06T03:06:58Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![ong-ar](https://avatars.discourse-cdn.com/v4/letter/o/779978/32.png) [@ong-ar](https://discuss.elastic.co/u/ong-ar)
#### Post date: [December 6, 2023, 3:06am UTC](https://discuss.elastic.co/t/using-debug-explain-kills-data-nodes-8-11-1/348690/1 "2023-12-06T03:06:58Z")

</div>

### Elasticsearch Version

Version: 8.11.1, Build: rpm/6f9ff581fbcde658e6f69d6ce03050f060d1fd0c/2023-11-11T10:05:59.421038163Z, JVM: 21.0.1

### Installed Plugins

### Java Version

openjdk version "21.0.1"

### OS Version

6.1.61-85.141.amzn2023.x86\_64

### Problem Description

I often use map\_script for statistics, but when I use Debug.explain, the data nodes emit the message below and all die.

If removing `Debug.explain`, it works normally.

It wasn't a problem when using version 8.6.2, but since upgrading, I'm seeing that.

I need help.

```auto
[ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [data-03] fatal error in thread [elasticsearch[data-03][search_worker][T#14]], exiting
org.elasticsearch.painless.PainlessExplainError: null
        at org.elasticsearch.painless.api.Debug.explain(Debug.java:23) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute( ...:66) ~[?:?]
        at org.elasticsearch.search.aggregations.metrics.ScriptedMetricAggregator$1.collect(ScriptedMetricAggregator.java:124) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.aggregations.LeafBucketCollector.collect(LeafBucketCollector.java:86) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.query.QueryPhaseCollector$CompositeLeafCollector.collect(QueryPhaseCollector.java:306) ~[elasticsearch-8.11.1.jar:?]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:277) ~[lucene-core-9.8.0.jar:?]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:236) ~[lucene-core-9.8.0.jar:?]
        at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:45) ~[elasticsearch-8.11.1.jar:?]
        at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38) ~[lucene-core-9.8.0.jar:?]
        at org.elasticsearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:536) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:460) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.internal.ContextIndexSearcher.lambda$search$4(ContextIndexSearcher.java:375) ~[elasticsearch-8.11.1.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.11.1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.lang.Thread.run(Thread.java:1583) ~[?:?]

```

### Steps to Reproduce

1. create index

```auto
PUT /test_index?pretty
{
    "settings" : {
        "number_of_shards" : 1,
        "number_of_replicas" : 0
    },
    "mappings" : {
        "properties" : {
            "tags" : { "type" : "keyword" },
            "updated_at" : { "type" : "date" }
        }
    }
}

```

1. index a document

```auto
PUT test_index/_doc/1
{
  "tags": [
    "opster",
    "elasticsearch"
  ],
  "date": "01-01-2020"
}

```

1. search

```auto
GET test_index/_search
{
   "aggs": {
    "test":{
      "scripted_metric":{
        "init_script": """
          state.funding = new HashMap();
        """,
        "map_script": """ 
          def source = params._source;
          Debug.explain(source);
          return;
          
        """,
        "combine_script":"""
          return state;
          
        """,
        "reduce_script": """ 
          def result = new HashMap();
          
          return result;
         """
      }  
    }
  },
  "size":0
}

```

1. check

```auto
GET test_index/_search

```

If you use `Debug.explain` in map\_script, an error message will appear, but if you search another index afterwards, the search will not work. The reason is that the data nodes are dead with the below message.

### Logs (if relevant)

- map\_script result

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "runtime error",
        "painless_class": "java.util.LinkedHashMap",
        "to_string": "{tags=[opster, elasticsearch], date=01-01-2020}",
        "java_class": "java.util.LinkedHashMap",
        "script_stack": [
          """Debug.explain(source);
          """,
          " ^---- HERE"
        ],
        "script": " ...",
        "lang": "painless",
        "position": {
          "offset": 65,
          "start": 51,
          "end": 84
        }
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "test_index",
        "node": "o39edlW3TC6VvO9Iu0bH8w",
        "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "painless_class": "java.util.LinkedHashMap",
          "to_string": "{tags=[opster, elasticsearch], date=01-01-2020}",
          "java_class": "java.util.LinkedHashMap",
          "script_stack": [
            """Debug.explain(source);
          """,
            " ^---- HERE"
          ],
          "script": " ...",
          "lang": "painless",
          "position": {
            "offset": 65,
            "start": 51,
            "end": 84
          },
          "caused_by": {
            "type": "painless_explain_error",
            "reason": null
          }
        }
      }
    ]
  },
  "status": 400
}

```

- elasticsearch log

```auto
[ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [master-1] fatal error in thread [elasticsearch[master-1][search_worker][T#5]], exiting
org.elasticsearch.painless.PainlessExplainError: null
        at org.elasticsearch.painless.api.Debug.explain(Debug.java:23) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute( ...:66) ~[?:?]
        at org.elasticsearch.search.aggregations.metrics.ScriptedMetricAggregator$1.collect(ScriptedMetricAggregator.java:124) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.aggregations.LeafBucketCollector.collect(LeafBucketCollector.java:86) ~[elasticsearch-8.11.1.jar:?]
        at org.apache.lucene.search.MatchAllDocsQuery$1$1.score(MatchAllDocsQuery.java:63) ~[lucene-core-9.8.0.jar:?]
        at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:45) ~[elasticsearch-8.11.1.jar:?]
        at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38) ~[lucene-core-9.8.0.jar:?]
        at org.elasticsearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:536) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:460) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.search.internal.ContextIndexSearcher.lambda$search$4(ContextIndexSearcher.java:375) ~[elasticsearch-8.11.1.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.11.1.jar:?]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.11.1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.lang.Thread.run(Thread.java:1583) ~[?:?]

```

- system log

```auto
systemd-entrypoint: ERROR: Elasticsearch exited unexpectedly, with exit code 1

```

---

<div class="post-metadata">

### Author: ![ong-ar](https://avatars.discourse-cdn.com/v4/letter/o/779978/32.png) [@ong-ar](https://discuss.elastic.co/u/ong-ar)
#### Post date: [December 6, 2023, 3:35am UTC](https://discuss.elastic.co/t/using-debug-explain-kills-data-nodes-8-11-1/348690/2 "2023-12-06T03:35:44Z")

</div>

I also tested it on Centos 7 and it was the same.

enviroment: 1 node (master + data)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 3, 2024, 3:36am UTC](https://discuss.elastic.co/t/using-debug-explain-kills-data-nodes-8-11-1/348690/3 "2024-01-03T03:36:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
