How to get a continuous transform to write to multiple indexes

When I load my data through logstash I specify an index name pattern, to split the index up by time.

I now have a continuous transform running, grouping by a 1 hour date histogram, against that data. Problem is it's writing it all to a single index. How do I do the equivalent of logstash putting date elements into the index name? sales_hourly-2020-02, sales_hourly-2020-03, sales_hourly-2020-04 etc...

Transform creates the destination index for you if you haven't. That means, you can create the destination index yourself or for this case use aliases and index life-cycle management.

1 Like

Thanks, looks like it'll work, even if it's not quite a smooth as the time based logstash naming (mostly a concern when bulk indexing historical data).

You've got a typo on this page: https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html

The sample template ( _template/timeseries_template) shown is incorrect. The settings section needs to be inside a template object.

{
  "index_patterns": ["timeseries-*"], 
  "template": {  
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1,
      "index.lifecycle.name": "timeseries_policy",      
      "index.lifecycle.rollover_alias": "timeseries"    
    }
  }  
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.