Silence deprecation log without restarting the cluster

Elastic Stack Elasticsearch
#1

Hi,

We had a log4j settings that has set the logging level of org.elasticsearch.deprecation to ERROR that we set from the _cluster/settings endpoint. Ever since we upgraded to 6.8.6, that settings stopped working and the cluster logs a huge number of deprecation warnings. Is there anything that changed from 6.1 to 6.8? We recently upgraded from 6.1.0 to 6.8.0

    curl -X PUT "http://localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
      {
        "transient": {
          "logger.org.elasticsearch.deprecation": "ERROR"
        }
      }'

I think 90% of the warning comes from these 2:
Deprecated field [_version_type] used, expected [version_type] instead
and
Accessing variable [_agg] via [params._agg] from within a scripted metric agg init script is deprecated in favor of using [state].

Until we make a code change, we'd like to silence the warnings since it's quite noisy and impacts our bill for logging to sumo logic

Thanks,
Khoa

#2

Hello,
Your setting was transient and it gets reset once you restart all the nodes.

Can you try to do a GET _cluster/settings and see if the setting is still there?

If not, please set:

PUT _cluster/settings
{ "persistent": { "logger.org.elasticsearch.deprecation": "error" } }

Be aware this really important when you will move to 7. So please re-enable it as soon as you can.

#3

I already tried setting it both as transient and persistent actually. Our staging environment has it enabled and I just want to silence it in production

#4

Khoa,

Do you mean, the persistent setting does not work as well?

#5

No, neither persistent nor transient setting works. At this stage, I think it's probably a Elasticsearch bug

#6

Can you share the output of the Elasticsearch settings with GET _cluster/settings?include_default, the log4j2.properties and tell if you're running on Docker?

Thanks

#7

I've tried to reproduce this on 6.8.6. Each time I this against a brand-new 6.8.6 cluster ...

curl 'http://localhost:9200/_mtermvectors' -H 'Content-type: application/json' --data-binary $'{"docs":[{"_type":"testtype","_version_type":"external","_id":"testing_document","_index":"testidx","version":1}]}'

... I see this line in the deprecation log file ...

[2020-05-16T08:51:11,172][WARN ][o.e.d.c.ParseField       ] [node-0] Deprecated field [_version_type] used, expected [version_type] instead

I have confirmed that these logs are indeed correctly suppressed by all three methods mentioned above:

  • change logger.deprecation.level = warn to logger.deprecation.level = error in log4j2.properties (and restarting the node)

  • setting a transient setting using curl -XPUT 'http://localhost:9200/_cluster/settings' -H 'Content-type: application/json' --data-binary $'{"transient":{"logger.org.elasticsearch.deprecation":"ERROR"}}' (no restart needed)

  • setting a persistent setting using curl -XPUT 'http://localhost:9200/_cluster/settings' -H 'Content-type: application/json' --data-binary $'{"persistent":{"logger.org.elasticsearch.deprecation":"ERROR"}}' (no restart needed)

My best guess at the moment is that these log messages are coming from a different cluster from the one whose config you are adjusting.

1 Like
#8

Thanks David and Luca. The settings was there before and after the upgrade. It's just doesn't seem to do anything. I tried setting it to ERROR multiple times but it's not effective.

However, I just found that resetting "logging.org.elasticsearch.deprecation": null AND setting it to ERROR again from the cluster setting did the job. But it definitely didn't work without setting it to null first. Maybe somehow the rolling restart upgrade invalidate the setting and caused Elasticsearch to not picking up the change from PUT /_cluster/settings because it's thinking the changes are the same with what it currently have?

I'll mark this as resolved because my problem has been solved. I've linked the log of GET _cluster/settings and my log4j2.properties, just in case Elasticsearch team wants to dig deeper

log4j2.properties

_cluster/settings

_cluster/settings?include_default contains quite a bit sensitive info like IPs and such so I'm not sharing it here. However, if you can tell me what settings you need, I'll fetch it

Thanks

#9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.