I'm new to Elastic and am currently enabling SSL/TLS across our cluster. I've currently configured applications that ingest to Elasticsearch to use the built-in superuser 'elastic' for authentication. This is less than ideal due to the additional permissions on the user that aren't required for simple ingestion. I would like to create application-specific users that have the minimum required permissions to simply ingest data into Elasticsearch (e.g. if using Logstash, I would like a logstash_ingest application-user that authenticates the pipelines for elasticsearch ingestion, or if using NiFi, I would like a nifi_ingest application-user).
Elastic has a good bit of information on this topic for LogStash but it can be used for other writers too. This can be reduced depending on your usecase.
I've previously used the linked documentation for Logstash pipeline ingestion into Elasticsearch, however, I was encountering authentication errors. Before implementing the permissions in the documentation, ingestion was authenticated via the elastic built-in superuser. When implementing the logstash_internal user, Logstash was no longer able to successfully ingest into my Elastisearch cluster. Immediately reverting it back to the elastic superuser fixed the authentication issues.
Would there be any settings or permissions not listed in Configuring Security in Logstash docs that would cause authentication errors? For more reference, these are HTTP 401 errors seemingly generated by each Elasticsearch node.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
