How to enable size-mapper plugin / _size field via RestHighLevelClient

Elastic Stack Elasticsearch
1

Hey Folks,

I'm trying to add the _size field to an existing index via RestHighLevelClient

and failing to see there I could put the following

"mappings": {
    "_size": {
      "enabled": true
    }
  }

the Dev Tools equivalent is

curl -X PUT "localhost:9200/my-index-000001?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "_size": {
      "enabled": true
    }
  }
}
'

trying the same for an existing index:


PUT existing/_mapping
{
  "mappings": {
    "_size": {
      "enabled": true
    }
  }
}

will result in

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Root mapping definition has unsupported parameters:  [mappings : {_size={enabled=true}}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [mappings : {_size={enabled=true}}]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "Root mapping definition has unsupported parameters:  [mappings : {_size={enabled=true}}]"
    }
  },
  "status" : 400
}
2

Did you install the plugin on all nodes and restarted the cluster?

3

Yes I did and it works when doing it via the Devtools console.
I just don't know

  • How to do it for an existing index
  • How to translate it to a mappings.json to do it programmatically via
private val elasticsearchClient: org.elasticsearch.client.RestHighLevelClient
elasticsearchClient.indices().putIndexTemplate(
            PutComposableIndexTemplateRequest()
                .name(elkConfig.templateName)
                .indexTemplate(indexTemplate),
            RequestOptions.DEFAULT
        )

which consumes a mapping.json which works for normal index templates but I'm unable to add the "_size" field:

{
  "mappings": {
    "_doc": {
      "_size": {
        "enabled": true
      }
    }
  }
4

Could you try with:

PUT existing/_mapping
{
  "_size": {
    "enabled": true
  }
}
5

@dadoonet I noticed the plugin wasn't activated on our cluster. Also a restart of all nodes didn't work

6

So the plugin has been deployed on the cluster now?

7

hard to tell
the following request fails:

PUT test_size_plugin
{
  "mappings": {
    "_size": {
      "enabled": true
    }
  }
}

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Root mapping definition has unsupported parameters:  [_size : {enabled=true}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [_size : {enabled=true}]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "Root mapping definition has unsupported parameters:  [_size : {enabled=true}]"
    }
  },
  "status" : 400
}

GET /_cat/plugins?v=true&s=component&h=name,component,version,description

While looking through the JSON, I noticed that the plugin seems to be activated on the “resources” level, but not on the “cluster_topology” level.

$.resources.elasticsearch.plan.elasticsearch.enabled_built_in_plugins = [“mapper-size”]

The plugin shows up in the plan definition of elasticsearch but not in the node definition:

  "resources": {
    "elasticsearch": [
      {
        "region": "aws-eu-central-1",
        "ref_id": "main-elasticsearch",
        "plan": {
          "cluster_topology": [
            {
              "zone_count": 2,
              "elasticsearch": {
                "user_plugins": [],
                "system_settings": {
                     ...
                },
                "user_bundles": [],
                "enabled_built_in_plugins": []    # <- The plugin seems to be missing here and in the other nodes
              },
...
8

So it has not been deployed. You need to add the plugin to all nodes and restart all nodes first.

Until GET /_cat/plugins?v shows it, you won't be able to use the plugin.

9

We did that, even tried twice

10

But what did you do exactly? I'm lost :slight_smile:

11

Oh sorry:

  • enable the size mapper plugin in the deployment of our elastic.co instance
  • restart the whole deployment
  • try to query enabled plugins => it’s missing
  • try to enabled the mapping for “_size” fields => fails

We tried the steps twice (with additional restart after disabling the plug-in before adding again)

12

@dadoonet Is this a setting you can add to an existing index holding data?

13

@Christian_Dahlqvist it fails for new and existing Index with data

14

What is the output of the cat plugins API?

Are you able to set it in an index template and create a new index based on that?

15

I'm not sure you can but first I want to make sure the plugin is working. :blush:

16

You are running on cloud.elastic.co? Is that correct?

If so, did you save the settings when you added the plugin? I have been confused myself in the past by the UI.

17

Yes :+1:t3:
The plug-in activation is also visible the the deployment activity summary.

However it seems like the plug-in is not activated on all nodes, see the snippet of “Equivalent API JSON” which represents enabling the feature via UI

18

It’s missing in the cat output as well

19

So I'd suggest to open a ticket with the support team.

1 Like
20

Yes did that … research is ongoing