Kubernetes CRDs for index templates / datastreams with ECK?

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hi all,

I’m deploying Elastic via ECK and want to manage index templates / datastreams as code in Kubernetes manifests (Helm / GitOps).

I checked the ECK API docs but didn’t see any CRDs for this (or I missed it).

Question: Are there CRDs for index templates/datastreams ?

Thanks!

2

Yes! ECK Has CRDs for Index Templates

ECK provides the StackConfigPolicy CRD (available since ECK 2.6) that allows you to manage Elasticsearch settings declaratively as Kubernetes resources. This includes index templates.

Example: Managing Index Templates with StackConfigPolicy

apiVersion: stackconfigpolicy.k8s.elastic.co/v1alpha1
kind: StackConfigPolicy
metadata:
  name: my-stack-config
  namespace: elastic-system
spec:
  # Select which Elasticsearch clusters this policy applies to
  resourceSelector:
    matchLabels:
      environment: production
  
  elasticsearch:
    # Index Templates
    indexTemplates:
      # Component Templates (reusable building blocks)
      componentTemplates:
        my-mappings:
          template:
            mappings:
              properties:
                '@timestamp':
                  type: date
                message:
                  type: text
        my-settings:
          template:
            settings:
              number_of_shards: 2
              number_of_replicas: 1
      
      # Composable Index Templates (combine component templates)
      composableIndexTemplates:
        my-logs-template:
          index_patterns:
            - "logs-*"
            - "app-logs-*"
          composed_of:
            - my-mappings
            - my-settings
          priority: 500
          template:
            aliases:
              logs: {}
          _meta:
            description: "Template for application logs"
    
    # ILM Policies
    indexLifecyclePolicies:
      logs-policy:
        phases:
          hot:
            actions:
              rollover:
                max_size: 50gb
                max_age: 1d
          warm:
            min_age: 7d
            actions:
              shrink:
                number_of_shards: 1
          delete:
            min_age: 30d
            actions:
              delete: {}
    
    # Ingest Pipelines
    ingestPipelines:
      my-pipeline:
        description: "Add timestamp to documents"
        processors:
          - set:
              field: ingested_at
              value: "{{_ingest.timestamp}}"

Important Notes

  1. License Required: StackConfigPolicy requires an Enterprise license (or trial license)

  2. Elasticsearch Version: Requires Elasticsearch 8.6.1+ (uses the file-based settings feature internally)

  3. GitOps Compatible: The CRD works perfectly with Helm, Kustomize, ArgoCD, Flux, etc.

  4. Label Selector: Use resourceSelector to target specific Elasticsearch clusters

  5. No Datastream CRD: There's no dedicated CRD for datastreams, but datastreams are automatically created when you index data that matches an index template with data_stream: {} enabled

Documentation

1 Like