# Trying to transform data from one index to another by applying pipeline in transform,But pipeline is executing only first half of it till Android filter

**URL:** <https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721>\
**Category:** Elasticsearch\
**Tags:** painless, transforms, ingest-pipeline\
**Created:** [April 25, 2023, 9:27am UTC](https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721 "2023-04-25T09:27:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chinmay\_Bhusate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chinmay_bhusate/32/100952_2.png) [@Chinmay\_Bhusate](https://discuss.elastic.co/u/Chinmay_Bhusate)\
**Post date:** [April 25, 2023, 9:27am UTC](https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721/1 "2023-04-25T09:27:39Z")

</div>

Sharing my transform alongwith Pipeline.

1. Transform

```auto
POST _transform/_preview
{
  "source": {
    "index": [
      "events.test"
    ]
  },
  "pivot": {
    "group_by": {
      "session_id": {
        "terms": {
          "field": "session_id"
        }
      },
      "device_id": {
        "terms": {
          "field": "device_id"
        }
      }
    },
    "aggregations": {
      "session_start": {
        "min": {
          "field": "server_time"
        }
      },
      "session_end": {
        "max": {
          "field": "server_time"
        }
      },
  
      "time_spent": {
        "bucket_script": {
          "buckets_path": {
            "start": "session_start.value",
            "stop": "session_end.value"
          },
          "script": """
            return (params.stop - params.start)/1000;
          """
        }
      }
    }
  },
  "frequency": "15m",
  "dest": {
    "index": "analysis.test",
    "pipeline":"transform_user"
    
  },
  "settings": {
    "max_page_search_size": 500
  }
}

2.Pipeline

PUT _ingest/pipeline/transform_user
{
  "processors": [
    {
      "grok": {
        "field": "session_id",
        "patterns": [
          "(?<param19>...)$"
        ]
      }
    },
    {
      "script": {
        "lang": "painless",
        "source": """
          if (ctx.param19 == 'and') 
          {
            ctx.param19 = 'Android';
          }
        """
      }
	  },
	  {
      "grok": {
        "field": "param20",
        "patterns": [
          "(?<param20>%{GREEDYDATA})$"
          ]
      }
    },
    {
     "script": {
        "source": """
		      ctx.user_type = "Free"
          if (ctx.param20 == "NearRenewalUser" || ctx.param20 == "NearRenewalUserPP"
         || ctx.param20 == "SubUser" || ctx.param20 == "SubUserPP"
         || ctx.param20 == "RenewedUser" || ctx.param20 == "RenewedUserPP"
         || ctx.param20 == "PremiumSubUserPP" || ctx.param20 == "PremiumSubUser")
          {
            ctx.user_type = 'Paid';
          }
        """
      }
    }
  ]
 }

```

Blockquote

---

<div class="post-metadata">

**Author:** ![przemekwitek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/przemekwitek/32/79526_2.png) [@przemekwitek](https://discuss.elastic.co/u/przemekwitek)\
**Post date:** [April 25, 2023, 9:42am UTC](https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721/2 "2023-04-25T09:42:18Z")

</div>

Hi,

1. Could you share Elasticsearch server logs from around the time you've experienced this issue?

2. Have you tested the pipeline separately from the transform, i.e.: by issuing a number of index requests with `?pipeline=transform_user` in the URL? This way we could see if the problem lies in the pipeline or in the transform.

3. Should there be a semicolon (`;`) after the line: `ctx.user_type = "Free"` ?

---

<div class="post-metadata">

**Author:** ![Chinmay\_Bhusate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chinmay_bhusate/32/100952_2.png) [@Chinmay\_Bhusate](https://discuss.elastic.co/u/Chinmay_Bhusate)\
**Post date:** [April 25, 2023, 9:54am UTC](https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721/3 "2023-04-25T09:54:55Z")

</div>

Hi My pipeline is showing results in destination index but only till ctx.param19 = 'Android ' , and for another part i.e

```auto
ctx.user_type = "Free";
          if (ctx.param20 == "NearRenewalUser" || ctx.param20 == "NearRenewalUserPP"
         || ctx.param20 == "SubUser" || ctx.param20 == "SubUserPP"
         || ctx.param20 == "RenewedUser" || ctx.param20 == "RenewedUserPP"
         || ctx.param20 == "PremiumSubUserPP" || ctx.param20 == "PremiumSubUser")
          {
            ctx.user_type = 'Paid';
          }
        """

```

It is showing all user\_type = Free . So that means it is not going inside if condition.

Kindly suggest any solution on this issue. ` `

This is the response for your refrence

```auto
"preview" : [
    {
      "device_id" : "0000000000000000",
      "session_id" : "16613182337680000000000000000_and",
      "session_start" : "2022-08-24 10:47:15",
      "platform" : "Android",
      "param20" : "Free",
      "session_end" : "2022-08-24 10:47:15",
      "time_spent" : 0.0
    },
    {
      "device_id" : "00000ca06f93d39e",
      "session_id" : "166959155804900000ca06f93d39e_and",
      "session_start" : "2022-11-28 23:14:34",
      "platform" : "Android",
      "param20" : "Free",
      "session_end" : "2022-11-28 23:14:34",
      "time_spent" : 0.0
    },

```

---

<div class="post-metadata">

**Author:** ![Chinmay\_Bhusate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chinmay_bhusate/32/100952_2.png) [@Chinmay\_Bhusate](https://discuss.elastic.co/u/Chinmay_Bhusate)\
**Post date:** [April 26, 2023, 5:35am UTC](https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721/4 "2023-04-26T05:35:14Z")

</div>

Hi .... any update on this?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 24, 2023, 5:35am UTC](https://discuss.elastic.co/t/trying-to-transform-data-from-one-index-to-another-by-applying-pipeline-in-transform-but-pipeline-is-executing-only-first-half-of-it-till-android-filter/330721/5 "2023-05-24T05:35:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.