We are endeavoring to build a large ELK deployment for a large US university and I'm participating in the working group devoted to security and RBAC. I believe our planned architecture is fairly common:
Beats -> Kafka -> Logstash -> Elastic
While there's some potential security around Elastic via Shield, there's no end-to-end security model afaict, which would be desirable for our deployment. The JWT [1] suite of standards seems well oriented to providing for both message-level security and confidentiality at each component interface above. JSON Web Tokens are naturally capable of nesting, allowing each component to sign or encrypt the message as it passes through, allowing the end recipient (Elastic) to verify the integrity of data at each state of the ingestion pipeline. The nesting of message-level security effectively provides for end-to-end security. (The trust model is a non-insignificant detail, but it's out of scope for the present discussion. Assume a way to distribute public keys among all components in the pipeline.)
I'm curious whether there's been any interest or consideration of end-to-end security in the past. I'm also curious what folks think of JWT as a potential solution.
JWA – JSON Web Algorithms, describes cryptographic algorithms used in JOSE
JWK – JSON Web Key, describes format and handling of cryptographic keys in JOSE
JWS – JSON Web Signature, describes producing and handling signed messages
JWE – JSON Web Encryption, describes producting and handling encrypted messages
JWT – JSON Web Token, describes representation of claims encoded in JSON and protected by JWS or JWE
and there is OpenID Connect 1.0, which is related to a pre-release version of JOSE
Yes, I think this can be a solution. Elasticsearch offers a flexible plugin architecture that permits a full replacement of the HTTP layer. But the solution should be only open source and not behind closed doors like Shield, where you are left to yourself in the decision to trust the implementation or not.
My personal preference for an OpenID implementation is the Google OAuth Java client.
JOSE is the parent standard, thanks for clarifying. I want to clarify that I am not suggesting to implement OpenID Connect as an authentication technology, but rather to leverage JSON Web Tokens (JWTs) to wrap existing Beats JSON messages to provide for message-level security that could cover the entire logging pipeline. I see this as entirely orthogonal to Shield; possibly complimentary, but certainly has to have broader scope.
This is an interesting idea! I'm not entirely sure which potential problem you're trying to solve, though.
Today, it's already possible to use TLS to encrypt traffic between the various components, and I believe you can also use PKI to ensure that upstream systems are receiving data from systems they trust, at all layers.
What additional functionality would this JWT-message-signing provide that you don't get from certificates and PKI?
If the HTTP request contains a JWT in a header (or url parameter, although thats not recommended) Search Guard will check the signature and, if it is valid, extract the subject and roles/group membership from the token.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.