# Encryption In Elasticsearch

**URL:** https://discuss.elastic.co/t/encryption-in-elasticsearch/360778
**Category:** Elasticsearch
**Created:** [June 4, 2024, 11:26am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778 "2024-06-04T11:26:40Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [June 4, 2024, 11:26am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/1 "2024-06-04T11:26:40Z")

</div>

HI Team,

We need to implement Encryption In Elasticsearch. As we know there are two types of encryption.

1. Encryption in Transit.
2. Encryption while data at REST.

Please help us how to achieve the same.

Thanks,  
Debasis

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 4, 2024, 11:40am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/2 "2024-06-04T11:40:56Z")

</div>

Encryption in Transit is done by configuring Elasticsearch to use TLS on both the transport and http endpoints.

The documentation for it starts [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/manually-configure-security.html) and there are also plenty of posts already on the forum about it.

Encryption at REST is not done by Elasticsearch, but by the operating system as explained in this [post](https://discuss.elastic.co/t/how-should-i-encrypt-data-at-rest-with-elasticsearch/96), for linux normally you just enable _dm-crypt_ on your server.

---

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [June 5, 2024, 6:43am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/3 "2024-06-05T06:43:33Z")

</div>

> [@leandrojmp](#):
>
> Encryption at REST is not done by Elasticsearch

Thanks @leandrojmp for response. Since encryption at REST depends on third party tool (OS) so is it avliable if we use basic version of Elasticsearch.

Thanks,  
Debasis

---

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [June 6, 2024, 10:01am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/4 "2024-06-06T10:01:53Z")

</div>

@leandrojmp As further discussion with DEV team get to know that, they want to encrypt the specific fields in Elasticsearch not the entire record which received from the customer.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 6, 2024, 12:45pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/5 "2024-06-06T12:45:27Z")

</div>

> [@Debasis\_Mallick](#):
>
> As further discussion with DEV team get to know that, they want to encrypt the specific fields in Elasticsearch not the entire record which received from the customer.

This needs to be done during ingestion, you can do it in both Logstash or using an Ingest pipeline.

Logstash has a `fingerprint` filter that can generate a hash of some string and Elasticsearch has a `fingerprint` processor that does the same thing.

But keep in mind thaty ou cannot search for the value befure you execute the fingerprint, only by the hash created.

For example, if you have this:

`fieldName`: `unencrypted value`, after you run the `fingerprint` filter or processor you can have `fieldName`: `random-hash`.

You cannot search `fieldName` for `unencrypted value`, but you can search using `random-hash`.

---

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [June 12, 2024, 6:31am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/6 "2024-06-12T06:31:52Z")

</div>

> [@leandrojmp](#):
>
> You cannot search `fieldName` for `unencrypted value`, but you can search using `random-hash`.

Thanks @leandrojmp for response. As you mentioned user need to search on `random-hash` value but how the end user or application will know the hash value of the actual value which is send by the customer.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 12, 2024, 8:05am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/7 "2024-06-12T08:05:36Z")

</div>

The application will need to hash the search term using the same algorithm used when it was indexed. Naturally this will only work for exact matches, although you can make it case insensitive by lowercasing before both indexing and querying

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 12, 2024, 11:54am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/8 "2024-06-12T11:54:19Z")

</div>

> [@Debasis\_Mallick](#):
>
> but how the end user or application will know the hash value of the actual value which is send by the customer.

You would need to do something like Christian mentioned, you will need to create the has of the entire value and search for it, it only works for exact matches.

For example, if you have a field named `sensitive_message` that needs to be encrypted and this field has this value _The quick brown fox jumps over the lazy dog_, you will create a hash of the entire string, and will only be able to search by the hash.

As mentioned:

> But keep in mind that you cannot search for the value before you execute the fingerprint, only by the hash created.

Which I mean is, if you generated a hash for a field, you cannot search on the uncrypted value, only the hash.

---

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [June 25, 2024, 6:29am UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/9 "2024-06-25T06:29:44Z")

</div>

@leandrojmp Could you please share any blog reference, how to use `fingerprint` processor in filebeat.  
I tried search in google did not find any blog reference how to use in file beat.

Thanks,  
Debasis

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [June 25, 2024, 12:54pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/10 "2024-06-25T12:54:50Z")

</div>

I do not know any blog post about it.

But have you checked the documentation [here](https://www.elastic.co/guide/en/beats/filebeat/current/fingerprint.html)?

You also have a fingerprint processor that can be used in an ingest pipeline in Elasticsearch, it is pretty similar and the documentation is [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/fingerprint-processor.html).

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 25, 2024, 12:59pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/11 "2024-06-25T12:59:08Z")

</div>

Although the fingerprint processor will replace fields with hashed values, remember that you will need to be able to exactly replicate this hash calculation in your application code if you want to be able to search of any of these hash values.

---

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [July 10, 2024, 12:02pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/12 "2024-07-10T12:02:40Z")

</div>

@Christian_Dahlqvist and @leandrojmp As you mentioned `fingerprint` but it is not same as of encryption. Please correct me if I am wrong , `fingerprint` will generate a hash value but encryption is a different thing.

Thanks,  
Debasis

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [July 10, 2024, 12:35pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/13 "2024-07-10T12:35:09Z")

</div>

> [@Debasis\_Mallick](#):
>
> Please correct me if I am wrong , `fingerprint` will generate a hash value but encryption is a different thing.

Well, hashing can be seem as a one-way encryption, some places refer to it in this way.

But since encryption means that you can also decrypt the data, then yes, they are different things.

In this context it is all you have, so the `fingerprint` filter will work as a one-way process to encrypt your data, but you cannot decrypt it.

---

<div class="post-metadata">

### Author: ![Debasis\_Mallick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debasis_mallick/32/123723_2.png) [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)
#### Post date: [July 10, 2024, 12:54pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/14 "2024-07-10T12:54:03Z")

</div>

Thanks @leandrojmp for confirmation. Then how we can achieve data encryption at REST apart from `dm-crypt` which is a native tool to OS.

Thanks,  
Debasis

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 10, 2024, 3:05pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/15 "2024-07-10T15:05:27Z")

</div>

I believe you need to handle this in your application/code. As far as I know there is no built in support within the stack.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [July 10, 2024, 3:09pm UTC](https://discuss.elastic.co/t/encryption-in-elasticsearch/360778/16 "2024-07-10T15:09:37Z")

</div>

> [@Debasis\_Mallick](#):
>
> Then how we can achieve data encryption at REST apart from `dm-crypt` which is a native tool to OS.

There is nothing in the stack that would allow you to do that as already mentioned.

If you want to be able to encrypt and decrypt data stored in Elasticsearch, you need to do that on your application or code.
