java.lang.IllegalArgumentException: Fielddata when upgrading to 7.0.0

Hi! I run 6.6.0 in several envs ok and today was trying to test a 7.0.0 setup of elastic APM before upgrading those envs but I'm facing some problems, hope anyone can help.

My setup is as follows (done with 6.6.0 successfully):

apm-servers -> logstash -> elasticsearch

As stated in

For this configuration, you must load the index template into Elasticsearch manually because the options for auto loading the template are only available for the Elasticsearch output.

So what I do is:
1- export the template definition in the apm-server:

apm-server export template > apm-template-6.6.0.json

2- in logstash, pipeline for apm looks as follow:

input {
  beats {
    port => 5044

output {
  elasticsearch {
    hosts           => [ 'my-elasticsearch:9200' ]
    index           => "apm-%{[@metadata][version]}-%{[processor][event]}-%{+YYYY.MM.dd}"
    manage_template => true
    template_name   => "apm"
    template        => "/etc/logstash/templates/apm-template-6.6.0.json"

While doing same steps for 7.0.0 I cannot see the services in APM in kibana and elasticsearch is logging following errors:

"Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.",
"at org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder( ~[elasticsearch-7.0.0.jar:7.0.0]",
"at org.elasticsearch.index.fielddata.IndexFieldDataService.getForField( ~[elasticsearch-7.0.0.jar:7.0.0]"

Any idea?

Your process and configuration look great except you'll need to update your template to use the 7.0.0 template export - there were significant field changes introduced in 7.0.0 that account for the issue you have encountered.

Hi @gil thanks for your response.
Yes that's what I mean, I repeated the steps including the export of template after installing apm-server 7.0.0 but now getting above errors.
note I'm not reusing the elasticsearch cluster but starting up a new one completely

OK I saw something in elasticsearch that doesn't seem correct:

{"type": "server", "timestamp": "2019-04-22T15:51:02,122+0000", "level": "INFO", "component": "o.e.c.m.MetaDataIndexTemplateService", "": "docker-cluster", "": "bc34ff2855d8", "cluster.uuid": "v6EAYWCoSBe04WwYtt7skg", "": "aDQ5gEwRQXq3lWZX5f9w-Q", "message": "adding template [apm] for index patterns [logstash-*]" }

so seems is not pushing the apm template correctly for index patterns apm-* but applying default logstash-*

however my pipeline definition is clear:

output {
elasticsearch {
hosts => [ <%={ |item| "'#{item}'" }.join(', ') -%> ]
index => "apm-%{[@metadata][version]}-%{[processor][event]}-%{+YYYY.MM.dd}"
manage_template => true
template_overwrite => true
template => "/etc/logstash/templates/<%= @template_file %>"
template_name => "apm"

and the template exported from apm-server starts like:

  "index_patterns": [
  "mappings": {
    "_meta": {
      "beat": "apm",
      "version": "7.0.0"
    "date_detection": false,

What am I doing wrong?


ok seems I'm facing

Good find. I've confirmed locally that ilm_enabled => false can work around this - is that an option for you?

yes just confirmed myself as well that disabling ilm works.
Yep I'll use that option while is fixed properly.

1 Like

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.