# Transforming Indexes

**URL:** <https://discuss.elastic.co/t/transforming-indexes/295932>\
**Category:** Elasticsearch\
**Tags:** transforms\
**Created:** [February 1, 2022, 10:51am UTC](https://discuss.elastic.co/t/transforming-indexes/295932 "2022-02-01T10:51:12Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arya\_Yellapu](https://avatars.discourse-cdn.com/v4/letter/a/db5fbb/32.png) [@Arya\_Yellapu](https://discuss.elastic.co/u/Arya_Yellapu)\
**Post date:** [February 1, 2022, 10:51am UTC](https://discuss.elastic.co/t/transforming-indexes/295932/1 "2022-02-01T10:51:12Z")

</div>

Transforming Indexes is considering the timestamp field under "\_source" while comparing the data. How to exclude this field while comparison?

Below is the query I am using to transform the data from two indexes:

PUT /\_transform/compare\_indexes?pretty  
{  
"source" : {  
"index" : [  
"index2",  
"index5"  
],  
"query" : {  
"match\_all" : { }  
}  
},  
"dest" : {  
"index" : "compare\_indexes"  
},  
"pivot": {  
"group\_by": {  
"name" : {  
"terms" : {  
"field" : "Name"  
}  
}

```
},
"aggregations" : {
  "compare" : { 
    "scripted_metric" : {
      "map_script" : "state.doc = new HashMap(params['_source'])", 
      "combine_script" : "return state", 
      "reduce_script" : """ 
        if (states.size() !=2 ) {
          return "document_mismatch"
        }
        else if (states.get(0).equals(states.get(1)))
        {
          return "match"
        }
        else {
          return "data_mismatch"
        }
        """
    }
  }
}

```

},  
"frequency": "1m",  
"sync": {  
"time": {  
"field": "@timestamp",  
"delay": "1h"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [February 1, 2022, 1:08pm UTC](https://discuss.elastic.co/t/transforming-indexes/295932/2 "2022-02-01T13:08:32Z")

</div>

> [@Arya\_Yellapu](#):
>
> `"map_script" : "state.doc = new HashMap(params['_source'])",`

You may remove some keys from the HashMap. Something like:

```auto
"map_script" : "state.doc = new HashMap(params['_source']);state.doc.remove('@timestamp');",

```

In addition, please edit the post to `preformat` the whole script using `</>` button.

---

<div class="post-metadata">

**Author:** ![Arya\_Yellapu](https://avatars.discourse-cdn.com/v4/letter/a/db5fbb/32.png) [@Arya\_Yellapu](https://discuss.elastic.co/u/Arya_Yellapu)\
**Post date:** [February 1, 2022, 1:10pm UTC](https://discuss.elastic.co/t/transforming-indexes/295932/3 "2022-02-01T13:10:56Z")

</div>

Thanks Tomo I will try that and let you know.  
Can you also help me like how to get the field which is not matching in the second if clause?  
Thanks in Advance.

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [February 1, 2022, 1:20pm UTC](https://discuss.elastic.co/t/transforming-indexes/295932/4 "2022-02-01T13:20:34Z")

</div>

I suppose you have to check by iterating keys and values of the hashmap.

---

<div class="post-metadata">

**Author:** ![Arya\_Yellapu](https://avatars.discourse-cdn.com/v4/letter/a/db5fbb/32.png) [@Arya\_Yellapu](https://discuss.elastic.co/u/Arya_Yellapu)\
**Post date:** [February 1, 2022, 1:21pm UTC](https://discuss.elastic.co/t/transforming-indexes/295932/5 "2022-02-01T13:21:25Z")

</div>

Thanks Tomo will let you know.

---

<div class="post-metadata">

**Author:** ![Arya\_Yellapu](https://avatars.discourse-cdn.com/v4/letter/a/db5fbb/32.png) [@Arya\_Yellapu](https://discuss.elastic.co/u/Arya_Yellapu)\
**Post date:** [February 1, 2022, 1:37pm UTC](https://discuss.elastic.co/t/transforming-indexes/295932/6 "2022-02-01T13:37:03Z")

</div>

This is working like a charm,  
Thanks Tomo.  
Any idea how can I make this transform continuous?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 1, 2022, 1:37pm UTC](https://discuss.elastic.co/t/transforming-indexes/295932/7 "2022-03-01T13:37:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.