Anonymous Access Unexpectedly Allowed

Elastic Stack Elasticsearch
1

Hello -

We deployed a trial version of an ElasticSearch 6.5.3 and Kibana stack from the Azure marketplace and everything was running fine. Everything was configured properly, and we were using a username & password to connect ElasticSearch to Kibana using https://clusterIP:9200.

Then one day we discovered that https://clusterIP:9200 was allowing anonymous connections! In talking to the application team, they do not remember making any configuration changes. My Ops team was able to confirm that it doesn't look like the security configuration was changed since the initial build.

Does any of this make sense or sound possible? Is it possible that the trial license expires on 6.5.3 that could inadvertently enable anonymous access? If so, that seems like a huge security hole.

The only reason I think this may be license-related is because when I hit the https://clusterIP:9200/security/role endpoint, I receive the following 403:

    {
        "error": {
            "root_cause": [{
                    "type": "security_exception",
                    "reason": "current license is non-compliant for [security]",
                    "license.expired.feature": "security"
                }
            ],
            "type": "security_exception",
            "reason": "current license is non-compliant for [security]",
            "license.expired.feature": "security"
        },
        "status": 403
    }
2

Please see here for what happens when the license expires. It does not automatically open the cluster up.

FYI 6.5 is not far off EOL - Elastic Product End of Life Dates | Elastic. If you must use 6.X then use 6.8, otherwise the latest is 7.6. As a bonus, Security is free in 6.8 and above! :slight_smile:

3

I agree that we did not expect it to open up the cluster, but we can't figure out any other explanation. As far as I can tell, no changes were made to the config files after the trial was started, and yet we can hit the following search API and return records unauthenticated:

https://clusterIP:9200/_search?q=*NAME*

{
    "took": 9857,
    "timed_out": false,
    "_shards": {
        "total": 160,
        "successful": 160,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 16065835,
        "max_score": 1.0,
        "hits": {<<suppressed>>}
	}
}

And it definitely returns results without prompting for a username or password. We checked the documentation for enabling anonymous access and do not see any of the required settings in the elasticsearch.yml. Here's our config file

elasticsearch.yml

cluster.name: "<<suppressed>>"
node.name: "<<suppressed>>"
path.logs: /var/log/elasticsearch
path.data: /datadisks/disk1/elasticsearch/data
discovery.zen.ping.unicast.hosts: ["<<suppressed>>"]
node.master: true
node.data: true
discovery.zen.minimum_master_nodes: 1
network.host: [_site_, _local_]
node.max_local_storage_nodes: 1
node.attr.fault_domain: 0
node.attr.update_domain: 0
cluster.routing.allocation.awareness.attributes: fault_domain,update_domain
xpack.license.self_generated.type: trial
xpack.security.enabled: true
bootstrap.memory_lock: true

Is there anywhere else besides the settings documented in this reference article for enabling anonymous access? If not then I'm really concerned this is a bug in 6.5 when a trial license somehow rolls back to a basic. Here's a sample of what is returned from xpack license endpoint (which also returns unauthenticated):

https://clusterIP:9200/_xpack/license

{
  "license" : {
    "status" : "active",
    "uid" : "<<suppressed>>",
    "type" : "basic",
    "issue_date" : "2019-02-27T18:20:06.581Z",
    "issue_date_in_millis" : 1551291606581,
    "max_nodes" : 1000,
    "issued_to" : "<<suppressed>>",
    "issuer" : "elasticsearch",
    "start_date_in_millis" : -1
  }
}
4

Is there any .security indices in the cluster? That would indicate that someone setup a native realm, and it might contain your answer.

Otherwise one of the folks that do our security engineering might be able to pop in with some ideas.

5

It looks like you have a basic license in the cluster, not a trial license. It would seem someone has reverted to a basic license at some point, which does not include security in Elasticsearch 6.5 (only 6.8 onwards). This does not happen automatically but can if I recall correctly be done through Kibana.

I would recommend you upgrade to 6.8 as security is available for free there with the Basic license.

1 Like
6

Ahh good catch!

7

Thanks for the quick reply. But where is it picking up the 'basic' license if we have the following configured in our elasticsearch.yml?

xpack.license.self_generated.type: trial
xpack.security.enabled: true

Are you saying our theory holds water that after trial expired it automatically rolled back to 'basic' thus turning off authentication and making the search API open to unauthenticated sessions?

8

It does not automatically roll back to a basic license, but this rollback can be triggered through Kibana by a superuser. It seems like someone did this on the 27th of February.

9

Ah, I see... Would that be a logged operation in Kibana?

10

It should definately be logged in Elasticsearch if you have logs going back that far. It could possibly also be logged by Kibana.

Have you had users mainly log in as superusers?

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.