# Create a new column with the hash of an existing column

**URL:** <https://discuss.elastic.co/t/create-a-new-column-with-the-hash-of-an-existing-column/256779>\
**Category:** Elasticsearch\
**Created:** [November 26, 2020, 2:12pm UTC](https://discuss.elastic.co/t/create-a-new-column-with-the-hash-of-an-existing-column/256779 "2020-11-26T14:12:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![p1k4chu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/p1k4chu/32/79753_2.png) [@p1k4chu](https://discuss.elastic.co/u/p1k4chu)\
**Post date:** [November 26, 2020, 2:12pm UTC](https://discuss.elastic.co/t/create-a-new-column-with-the-hash-of-an-existing-column/256779/1 "2020-11-26T14:12:56Z")

</div>

I have an elasticsearch index to which I want to add a new column whose value is going to be the hash(sha256 or md5) of an existing column. Manually doing it isn't feasible as the index has more than 600,000,000 documents and doing it manually would take more than 2-3 months (according to a simple calculation done for 200,000 documents).

Any help is deeply appreciated.

---

<div class="post-metadata">

**Author:** ![egalpin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/egalpin/32/79772_2.png) [@egalpin](https://discuss.elastic.co/u/egalpin)\
**Post date:** [November 26, 2020, 8:30pm UTC](https://discuss.elastic.co/t/create-a-new-column-with-the-hash-of-an-existing-column/256779/2 "2020-11-26T20:30:55Z")

</div>

I think it should be clarified that Elasticsearch has no concept of columns, but rather _fields_. That's not intended to be pedantic but instead shared with the goal of enabling easier communication about features.

I think I understand what you're asking but please correct me if I'm wrong. Are you asking how you might add a new field whose value will be based on the value of another field? Do you need to do this on an ongoing basis for new data incoming, or only to apply this change to pre-existing data?

To apply to pre-existing data, I think the easiest approach might be [reindex using an ingest pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#reindex-with-an-ingest-pipeline) using a pipeline that includes a [script processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/script-processor.html) to produce a new field using a script like `ctx.my_field_sha = ctx.my_field.sha256()` (according to [this PR](https://github.com/elastic/elasticsearch/pull/59671) that `.sha256` method should be available).

If you need the hash value on an ongoing basis for new documents, I would strongly consider computing the hashed field in the client code. You could also use the same ingest pipeline mentioned above, but note that there are limitations with ingest pipelines such as [incompatibility with `update` API](https://github.com/elastic/elasticsearch/issues/17895#issuecomment-372729963).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 24, 2020, 8:31pm UTC](https://discuss.elastic.co/t/create-a-new-column-with-the-hash-of-an-existing-column/256779/3 "2020-12-24T20:31:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.