# Replication of field using scripted field on Kibana causes error

**URL:** <https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653>\
**Category:** Kibana\
**Created:** [November 2, 2016, 2:52am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653 "2016-11-02T02:52:36Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [November 2, 2016, 2:52am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/1 "2016-11-02T02:52:36Z")

</div>

I am trying to replicate a field using the scripted field in Kibana (5.0 GA).

However, it causes below error when I search.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/e/ea2dca91d28daf0fa01ba32a0fd3207f7a5233d1.PNG)

Below is my scripted field definition.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/a/a8c4ddaf7b9bed6f2a975ac572a1fd76af4dc594.PNG)

"type" is the filed automatically generated by filebeat so it is a string type.

I have set security level to true but made no difference.

```
script.inline: true
script.stored: true
script.file : true

```

Do I have any grammer mistake in the painless language?

---

<div class="post-metadata">

**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\
**Post date:** [November 2, 2016, 5:54am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/2 "2016-11-02T05:54:21Z")

</div>

Using 5.0 in conjunction with filebeat, I was able to setup the same scripted field in your example without any issue.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/3/33a91b5b5c2246b83c7759f2bd6dca75318fbcaf.png)

Let's see what the error is which is causing the results to fail. Unfortunately, in this state, it's not the easiest to access. We can either inspect the request, which in Chrome is available in View \> Developer \> Developer Tools and on the Network tab.

Or, we can always make the request manually. For example if you have curl available, create a fail called `request.json` with the following:

```json
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "foo": {
      "script": {
        "inline": "doc['type'].value",
        "lang": "painless"
      }
    }
  }
}

```

Then from the same directory, run `curl -d @request.json "http://localhost:9200/filebeat-*/_search" | python -m json.tool`.

If Elasticsearch is running at the different location, you will need to update that.

Another option, is to view the Elasticsearch logs for the error.

---

<div class="post-metadata">

**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\
**Post date:** [November 2, 2016, 5:55am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/3 "2016-11-02T05:55:51Z")

</div>

Can you also provide me the mappings for the filebeat index?

For example, `http://localhost:9200/filebeat-*/_mapping`

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [November 2, 2016, 10:59am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/4 "2016-11-02T10:59:24Z")

</div>

I do not have a filebeat mapping imported because I import the logs from **logstash**. All the logs go through logstash in our environment and index using the dynamic template _logstash_ .

Below is my current mapping definition. The mapping definition was too long to post so I reproduced on other machine with same error.

```
{
  "syslog-messages": {
    "mappings": {
      "messages": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "facility": {
            "type": "long"
          },
          "facility_label": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "hoge": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "host": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "logsource": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "message": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "pid": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "priority": {
            "type": "long"
          },
          "program": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "severity": {
            "type": "long"
          },
          "severity_label": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "tags": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "test": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "timestamp": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [November 4, 2016, 4:23am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/5 "2016-11-04T04:23:14Z")

</div>

I got the result of the query.

[root@BS-POC-Test-Env y-watanabe]# curl -d @request.json "[http://localhost:9200/syslog-messages/\_search](http://localhost:9200/syslog-messages/_search)" | python -m json.tool  
% Total % Received % Xferd Average Speed Time Time Time Current  
Dload Upload Total Spent Left Speed  
100 1295 100 1130 100 165 95957 14011 --:--:-- --:--:-- --:--:-- 100k  
{  
"\_shards": {  
"failed": 1,  
"failures": [  
{  
"index": "syslog-messages",  
"node": "mnn9dEZNSVi9epNPq4BqAw",  
"reason": {  
"caused\_by": {  
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [host] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.",  
"type": "illegal\_argument\_exception"  
},  
"lang": "painless",  
"reason": "runtime error",  
"script": "doc['host'].value",  
"script\_stack": [  
"org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:335)",  
"org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:111)",  
"org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:87)",  
"org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:84)",  
"java.security.AccessController.doPrivileged(Native Method)",  
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:84)",  
"doc['host'].value",  
" ^---- HERE"  
],  
"type": "script\_exception"  
},  
"shard": 0  
}  
],  
"successful": 4,  
"total": 5  
},  
"hits": {  
"hits": [],  
"max\_score": 1.0,  
"total": 1868  
},  
"timed\_out": false,  
"took": 2  
}

---

<div class="post-metadata">

**Author:** ![tbragin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tbragin/32/45166_2.png) [@tbragin](https://discuss.elastic.co/u/tbragin)\
**Post date:** [December 13, 2016, 8:24pm UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/6 "2016-12-13T20:24:08Z")

</div>

Curious, have you resolved your issue?

In case you are still working with Painless for scripted fields in Kibana, there is now a blog available with examples and best practices: [https://www.elastic.co/blog/using-painless-kibana-scripted-fields](https://www.elastic.co/blog/using-painless-kibana-scripted-fields)

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [December 14, 2016, 1:38am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/7 "2016-12-14T01:38:28Z")

</div>

@tbragin

No . Though my current environment is 100% same with my original post but I still get the Courier fetch error.

Below error was reproduced in v5.1.1.  
Query was done through below route.

`Kibana(v5.1.1) -> tribe node(v5.1.1) -> data node(v5.1.1)`

Logs come in to data node through below route.

`filebeat(v5.0.2-1) -> logstash(v5.0.2-1) -> datanode(v5.1.1)`

I have no **xpack** in front.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/7/7dfb6ccc40008c53351bbbdfe472f4a849a55b78.PNG)

Nothing special with the scripted field definition really.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/e/eadc318e9aa76192a7d672e32b1817a994fa329e.PNG)

There was no error in the "Developer Tool" in my Chrome Browser.

Let me turn on the DEBUG log on the tribe and data node side.

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [December 14, 2016, 2:03am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/8 "2016-12-14T02:03:45Z")

</div>

Below is the error I get in my tribe node.

```
[2016-12-14T10:55:01,361][DEBUG][o.e.a.s.TransportSearchAction] [node-client] [160257] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [node-3][10.1.0.14:9300][indices:data/read/search[phase/fetch/id]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:181) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:128) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:163) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:502) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.action.search.SearchTransportService$13.messageReceived(SearchTransportService.java:356) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.action.search.SearchTransportService$13.messageReceived(SearchTransportService.java:353) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1385) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.1.1.jar:5.1.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
Caused by: java.lang.IllegalArgumentException: No field found for [sometype] in mapping with types []
        at org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:80) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.painless.Executable$Script.execute(doc['sometype'].value='somevalue' @ <inline script>:5) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:123) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:163) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:502) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.action.search.SearchTransportService$13.messageReceived(SearchTransportService.java:356) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.action.search.SearchTransportService$13.messageReceived(SearchTransportService.java:353) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1385) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) ~[elasticsearch-5.1.1.jar:5.1.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.1.1.jar:5.1.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
        at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_111]

```

It seems that this is the root cause.

**Caused by: java.lang.IllegalArgumentException: No field found for [sometype] in mapping with types []**

Do you need a field defined to use the scripted field?

---

<div class="post-metadata">

**Author:** ![tbragin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tbragin/32/45166_2.png) [@tbragin](https://discuss.elastic.co/u/tbragin)\
**Post date:** [December 16, 2016, 5:10am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/9 "2016-12-16T05:10:39Z")

</div>

Yes, the field you are trying to locate in doc\_values must exist in the mapping for the scripted field to work.

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [December 22, 2016, 10:06am UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/10 "2016-12-22T10:06:12Z")

</div>

@tbragin

I finally got it working.

I made field mapping for the field which will be created in scripted field.  
Below example is **f2** for the target field.

```
[root@elk-analytic elasticsearch]# curl -XGET localhost:9200/someindex/_mapping?pretty
{
  "someindex" : {
    "mappings" : {
      "sometype" : {
        "properties" : {
          "date" : {
            "type" : "date"
          },
          "f1" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "f2" : {
            "type" : "integer"
          },
          "f3" : {
            "type" : "long"
          }
        }
      }
    }
  }
}

```

**Scripted field definition.**

 ![](https://us1.discourse-cdn.com/elastic/original/2X/9/9bd2ffc555ec03a30fd49848e6c7a3fc77236a69.PNG)

**Result**

 ![](https://us1.discourse-cdn.com/elastic/original/2X/5/56160d39e3fbd4c8dc1134c386464f05dcc39462.PNG)

However, I had below warning when I first created the scripted field. Can this be ignored?

 ![](https://us1.discourse-cdn.com/elastic/original/2X/d/dbf6d86f87d77dfd00c368dc841bc7706913dfe9.PNG)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:33pm UTC](https://discuss.elastic.co/t/replication-of-field-using-scripted-field-on-kibana-causes-error/64653/11 "2017-07-06T13:33:37Z")

</div>