[Transforms] How to add an aggregation field?

Elastic Stack Kibana
1

I'm using Transforms to pivot my log data for analytics.
I want to add another aggregation to the transform (obviously only for future runs). I don't see this option in the UI neither in the API.
Is it possible to add it?

If there's no official way, is the transform stored in a save object where I could edit it (as a workaround)?

2

I was able to do this by using DevTools:

  • Delete the old Transform
  • PUT the new one.

Let me know if there's a better way to achieve this. :pray:

3

The configuration of a transform that contains the group-by clauses and aggregations cannot be updated, only the parts documented in the link for the update API you shared. This has to do with the optimizations transforms run under the hood to work efficiently and consistently in continuous mode. Transforms run as a native Elasticsearch task so all the configuration is also stored directly as part of this in Elasticsearch, not as Saved Objects in Kibana (besides the related Data Views you can create for a transform's target index).

In a production environment the following workflow might work to avoid service interruptions:

  • Access the destination index for searches of the existing transform via an alias
  • Clone the transform, update the configuration, and specify a new destination index (all of this can be done via the UI, cloning is available in the transform list as part of the action items for each transform).
  • Once the new transform caught up to the most recent data, point the alias to the destination index of the new transform.
  • Finally, stop and delete the old transform (the UI also gives you options to clean up related destination indices and data views)

Hope that helps!

Best,
Walter

4

Thanks @walterra
Sounds like a well thought out strategy!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.