Problem with my personal index creation for platform logs

Elastic Stack Logstash
1

Hello, i'm trying to deploy the ECK with Beat (Filebeat) to ingest logs and Logstah to collect, for example platform logs or namespaces logs of Kubernetes cluster.
I'm using eck-operator 2.3.0 and Elastic stack in 8.3.3 inside a IBM Kubernetes Cluster in 1.23 version.
My Question is what is the the correct way to setup custom index in logstash output?
According to my understanding I need to create a index template directly in ES and use the same name in the output of the logstash. Is this not the right approach? Because I don't see the fields are index in the way I defined in the template. Below are steps I followed:
output to Elastic from Logstash:

    output {
      if ![kubernetes][namespace] {
        if [event][module] == "system" {
          elasticsearch {
            hosts => ["${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT:9200}"]
            # Default setting in Filebeat
            # index: "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
            index => "filebeat-%{[@metadata][version]}-%{+YYYY.MM.dd}"
            user => elastic
            password => "${ELASTICSEARCH_PASSWORD}"
            ssl => false
            ssl_certificate_verification => false
            cacert => "/usr/share/logstash/ca.crt"
            ecs_compatibility => disabled
            manage_template => true
            template => "/usr/share/logstash/default-template.json"
            template_name => "default"
            template_overwrite => true
          }
        }
      }
      else {
        elasticsearch {
          hosts => ["${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT:9200}"]
          index => "d-logging-logs-%{[kubernetes][namespace]}-%{+YYYY.MM.dd}"
          user => elastic
          password => "${ELASTICSEARCH_PASSWORD}"
          ssl => false
          ssl_certificate_verification => false
          cacert => "/usr/share/logstash/ca.crt"
          manage_template => true
          ecs_compatibility => disabled
          template => "/usr/share/logstash/default-template.json"
          template_name => "default"
          template_overwrite => true
        }
      }

default-template.json

    {
    "index_patterns": ["filebeat-*", "d-logging-logs-*"],
    "template": {
        "settings": {
        "index": {
            "number_of_shards": "1",
            "number_of_replicas": "1"
        }
        },
        "aliases": {},
        "mappings": {}
    }
    }

The topic here i only receive index from filebeat but i cant receive the other index

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.