So seriously, what permissions do beats need?

Elastic Stack Beats
, , ,
1

So contrary to the docs even this lot doesn't work:

POST /_security/api_key
{
  "name": "{beat_default_index_prefix}_localhost",
  "role_descriptors": {
    "{beat_default_index_prefix}_writer": {
      "cluster": ["monitor", "read_ilm", "manage_ingest_pipelines"],
      "index": [
        {
          "names": ["{beat_default_index_prefix}-*"],
          "privileges": ["view_index_metadata", "create_doc", "manage"]
        }
      ]
    }
  }
}

And surely "manage" is way too much privilege just to send logs to elasticsearch?

Errors seem to be 403 Forbidden:

error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/aliases/get] is unauthorized for API key id [blah] of user [elastic]"}]

I thought "manage" covered the admin/aliases privilege?

If I use a username: elastic/password: blah, then audit/journal/file/metric -beats all work fine, but I'd rather use API keys and not the superuser password on clients!

Also, where can I store the TLS certificates, as unless I save a copy of them in each beat/kibana/elasticsearch directory, they can't be read (despite the full path having 755 perms). I noticed something about dropped permissions in the docs, but that's not dropped permissions as any user can read them, seems more like a chroot?

Also, also.... As the beats seem to log to journald on systemd linux boxes, you end up with most of your logs just being about beats and not the interesting stuff like syslog, pam, auditd etc. Can you make the beats all log to a file/directory which can be ignored?

Finally, given that most of my beat configs are identical, is there a notion of a #include file or something, the new "meta beat" seems like overkill for my use case, but a single config file (and cert location!) would be useful.

2

Hi there ! I don't see the role descriptor you posted here in the documentation you linked to. Maybe you were referring to another documentation page ?

Agreed, but I think we don't suggest manage in the first place.

I thought "manage" covered the admin/aliases privilege?

It does but you shouldn't need it eitherway.

view_index_metadata should allow the indices:admin/aliases/get action, so I'm wondering whether the actual prefix you are using ( masked here with {beat_default_index_prefix} ) is not actually correct?

Also, where can I store the TLS certificates

For elasticsearch, it needs to be in a directory within the elasticseach configuration directory as this is the only dir that elasticsearch has access to read from .

Configure logging | Filebeat Reference [8.11] | Elastic

Nothing that I'm aware of , but I'll let other folks chime in.

3

the description was copied and pasted from that page, i added "manage" which did nothing and was suggested on a github issue.

ok this works for all of the beats now, however i had to add manage_ingest_pipelines or i get 403 errors about [cluster:admin/ingest/pipeline/put], plus its for the elastic superuser, so next have to figure out how to make a minimal user/role setup for filebeat clients:

POST /_security/api_key
{
  "name": "filebeat_localhost",
  "role_descriptors": {
    "filebeat_writer": {
      "cluster": ["monitor", "read_ilm", "manage_ingest_pipelines"],
      "index": [
        {
          "names": ["filebeat-*", "auditbeat-*", "journalbeat-*", "metricbeat-*"],
          "privileges": ["view_index_metadata", "create_doc"]
        }
      ]
    }
  }
}

As for TLS certificates, somehow all of the beats now seem fine with having ca.crt in /etc/elasticsearch/certs/, sure they weren't before, maybe that was just kibana or maybe it was /etc/pki/certs/ they didn't like.

This page seems to state that to override systemd logging you have to create a unit file (!!!) containing environment variables, so I assume setting logging.files won't work?

Also having a lot of trouble finding a common set of TLSv1.2 ciphers that works for 5601/9200/9300, seems to be a problem due to the mix of node and java8. The defaults work ok, but would like more control.

4

The API key is limited by the role descriptor you specify in there, so it doesn't matter that it is created by the elastic user, it has none of the privileges that the elastic user has, other than the ones that are explicitly described above. See the docs also.

It's always preferable to give the actual error when something is failing, rather your interpretation of it, as it allows others to provide actionable suggestions. If it didn't work before and it works now, probably something changed in the configuration now. What isn't working now and what is the error?

Again, this is unfortunately not detailed enough for anyone to assist you. It's always nice to give some additional details so that folks can come back with suggestions. What ciphers did you try and didn't work for both elasticsearch and kibana.

I guess so, since you are running with systemd, I'd trust these docs.

5

ok thanks for the info about API key roles @ikakavas

as for TLS, arguably the best TLSv1.2 ciphersuite common to all 5601/9300/9200 and the JVM is TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 however i get this error when i try to configure it (with the gnutls or openssl syntax for underscore/hyphen):

messages:Jun 28 21:37:08 elastic systemd-entrypoint[3187]: Likely root cause: java.lang.IllegalArgumentException: none of the ciphers [ECDHE-RSA-AES256-GCM-SHA384, ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384] are supported by this JVM

Java11 probably isn't an option, has to be java-1.8.0-openjdk, so rules out TLSv1.3 for the moment.

6

Not sure what you mean with

common to all 5601/9300/9200 and the JVM

9300 and 9200 are ports that elasticsearch uses and elasticsearch runs in the JVM so that's one thing. Node is the second ( kibana ) and golang is the third as beats need to support the ciphers too.

See here for a list of cipher names that you can use with Java 8 in the JVM.

here is the list of ciphers that filebeat (golang) supports.

and here is the list of ciphers that openssl (which Node.js uses internally for TLS) supports.

GCM based ciphers are not enabled in Java 8 by default because the performance of GCM in Java 8 is bad, see add AES-GCM cipher to default SSL suites · Issue #29695 · elastic/elasticsearch · GitHub for some additional context. However, you can explicitly enable them as you did above, could it be that you are running with an old version of java 1.8 ? What is the output of java -version on your machine?

Note that

ECDHE-RSA-AES256-GCM-SHA384, ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384

are not valid cipher names for the elasticsearch configuration.

7

Thanks, yes 3 different naming conventions was part of the problem, I think these are actually equivalent (although disabled by default?):

OpenSSL/node: ECDHE-ECDSA-AES128-SHA256
ES/Java:      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
Filebeat/go:  ECDHE-ECDSA-AES-128-CBC-SHA256

Without Galois you're basically limited to AES128-CBC (far from ideal) otherwise you end up with SHA1 digests.

java -version returns (from latest java-1.8.0-openjdk for RHEL 8.2):

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

nmap shows this lot for the default ciphers, which seems to show GCM is enabled:

PORT     STATE SERVICE  REASON
5601/tcp open  esmagent syn-ack
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM_8 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM (rsa 2048) - A
|       TLS_RSA_WITH_ARIA_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM_8 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM (rsa 2048) - A
|       TLS_RSA_WITH_ARIA_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|_  least strength: A

9200/tcp open  wap-wsp  syn-ack
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|_  least strength: A

So TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 would be the best non-GCM suite available but the docs list it as not supported by filebeat/golang.

8

I actually just verifed, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 is available in Java 8 and works fine with elasticsearch. That, in connection with your error message being:

cause: java.lang.IllegalArgumentException: none of the ciphers [ECDHE-RSA-AES256-GCM-SHA384, ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384] are supported by this JVM

(see the missing , between ECDHE-RSA-AES256-GCM-SHA384 and TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 ? ) makes me think this was a misconfiguration and not a JVM limitation.

Try setting

xpack.security.http.ssl.cipher_suites: ["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"]
xpack.security.transport.ssl.cipher_suites: ["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"]

it should work just fine.

1 Like
9

that works (only did it for http as transport is localhost only so http) and filebeat seems to be able to connect still.

Also set xpack.security.http.ssl.supported_protocols: [ TLSv1.2, TLSv1.3 ] in elasticsearch.yml for if I ever do move to java11.

9200/tcp open   wap-wsp  syn-ack
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|     compressors: 
|       NULL
|     cipher preference: indeterminate
|     cipher preference error: Too few ciphers supported
|_  least strength: A

in kibana.yml setting server.ssl.cipherSuites: ["ECDHE-RSA-AES256-GCM-SHA384"] gives me:

PORT     STATE  SERVICE  REASON
5601/tcp open   esmagent syn-ack
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|     compressors: 
|       NULL
|     cipher preference: indeterminate
|     cipher preference error: Too few ciphers supported
|_  least strength: A

And firefox shows i'm connecting using only that suite and only TLSv1.2

just got to figure out how to use api keys instead of uid/pwd with rsyslog now and i'm done!

Thanks so much for all the help and a great product.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.