# Continuous transform not working as expected

**URL:** https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834
**Category:** Elasticsearch
**Tags:** transforms
**Created:** [December 22, 2022, 11:58am UTC](https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834 "2022-12-22T11:58:54Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Dominik\_Solc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominik_solc/32/97844_2.png) [@Dominik\_Solc](https://discuss.elastic.co/u/Dominik_Solc)
#### Post date: [December 22, 2022, 11:58am UTC](https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834/1 "2022-12-22T11:58:54Z")

</div>

Hello,

I have problem that my continuous transform doesn't update as the index, which it takes data from, is getting new and new data.

The original index, let's name it `playwright-test-runs-data` (as i have it named), is getting json which consists of array of objects looking like this:

```auto
{
          "testName" : "Some test name",
          "numberOfRuns" : 1,
          "numberOfFails" : 0,
          "testStatus" : "expected",
          "environment" : "master",
          "browser" : "chromium",
          "testType" : "fullSuite",
          "dateTimestamp" : 1670927212470
}

```

On this `playwright-test-runs-data` i have created a Kibana index with a same name, where I added a field named `timestamp` with **date** type _(cause i wasn't able to successfully send a string that would match automatic date detection)_ that takes the `dateTimestamp` field and converts it into a date string format.

On this Kibana `playwright-test-runs-data` i run continuous transform with the `timestamp` field, as the field it checks for new data, which has this grouping and aggregations:

```auto
{
  "group_by": {
    "timestamp": {
      "date_histogram": {
        "field": "timestamp",
        "calendar_interval": "1m"
      }
    },
    "testName": {
      "terms": {
        "field": "testName.keyword"
      }
    },
    "browser": {
      "terms": {
        "field": "browser.keyword"
      }
    },
    "environment": {
      "terms": {
        "field": "environment.keyword"
      }
    },
    "testStatus": {
      "terms": {
        "field": "testStatus.keyword"
      }
    },
    "testType": {
      "terms": {
        "field": "testType.keyword"
      }
    }
  },
  "aggregations": {
    "numberOfFails.sum": {
      "sum": {
        "field": "numberOfFails"
      }
    },
    "numberOfRuns.sum": {
      "sum": {
        "field": "numberOfRuns"
      }
    }
  }
}

```

After i create and start the transform named "playwright-test-runs-transformed-data", it takes all data from the original index and transforms it all, but after that it doesn't update at all if the "playwright-test-runs-data" index gets new data, as can be seen in the screenshots

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/1/41de30e1ece7e1841dd8d249cd7e25728d1eb64d.jpeg)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/d/bd4d6c64686efbdd9fc25c4a0ad31607cd1ba63c.png)

Can you please help me find some mistake i made along the way or if I am missing some settings needed to be set?

If you will need more information, I will have no problem providing it if it will be possible.

---

<div class="post-metadata">

### Author: ![James\_Gowdy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_gowdy/32/74175_2.png) [@James\_Gowdy](https://discuss.elastic.co/u/James_Gowdy)
#### Post date: [December 22, 2022, 12:37pm UTC](https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834/2 "2022-12-22T12:37:17Z")

</div>

Hi,

Could you please share the full transform config?  
You can get it from the UI by expanding the row in the transforms table and selecting the JSON tab.

Thanks,  
James

---

<div class="post-metadata">

### Author: ![Dominik\_Solc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominik_solc/32/97844_2.png) [@Dominik\_Solc](https://discuss.elastic.co/u/Dominik_Solc)
#### Post date: [December 22, 2022, 2:56pm UTC](https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834/3 "2022-12-22T14:56:21Z")

</div>

Okay, that's what i found

```auto
value={
  "id": "playwright-test-runs-transformed-data",
  "version": "7.17.1",
  "create_time": 1670883112635,
  "source": {
    "index": [
      "playwright-test-runs-data*"
    ],
    "query": {
      "match_all": {}
    },
    "runtime_mappings": {
      "timestamp": {
        "type": "date",
        "script": {
          "source": "def timestamp = doc['dateTimestamp'].value;\nemit(timestamp);"
        }
      }
    }
  },
  "dest": {
    "index": "playwright-test-runs-transformed-data"
  },
  "frequency": "1m",
  "sync": {
    "time": {
      "field": "timestamp",
      "delay": "50s"
    }
  },
  "pivot": {
    "group_by": {
      "timestamp": {
        "date_histogram": {
          "field": "timestamp",
          "calendar_interval": "1m"
        }
      },
      "testName": {
        "terms": {
          "field": "testName.keyword"
        }
      },
      "environment": {
        "terms": {
          "field": "environment.keyword"
        }
      },
      "browser": {
        "terms": {
          "field": "browser.keyword"
        }
      },
      "testStatus": {
        "terms": {
          "field": "testStatus.keyword"
        }
      },
      "testType": {
        "terms": {
          "field": "testType.keyword"
        }
      }
    },
    "aggregations": {
      "numberOfRuns.sum": {
        "sum": {
          "field": "numberOfRuns"
        }
      },
      "numberOfFails.sum": {
        "sum": {
          "field": "numberOfFails"
        }
      }
    }
  },
  "settings": {
    "max_page_search_size": 500
  }
}

```

Is that correct?

---

<div class="post-metadata">

### Author: ![Dominik\_Solc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominik_solc/32/97844_2.png) [@Dominik\_Solc](https://discuss.elastic.co/u/Dominik_Solc)
#### Post date: [January 9, 2023, 2:20pm UTC](https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834/4 "2023-01-09T14:20:43Z")

</div>

Hello @James_Gowdy,

I hope i sent the right data you wanted. If you have a time, could you please have a look on the config?

Thanks,  
Dominik

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 6, 2023, 2:21pm UTC](https://discuss.elastic.co/t/continuous-transform-not-working-as-expected/321834/5 "2023-02-06T14:21:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
