How to Apply Custom Template to Logstash (8.x)

[inkweon7269]

I am studying Elasticsearch using Docker Compose and ELK. I am having an issue where the custom template is not being applied in logstash.conf. How can I fixed it?

csv-template.json

{
  "template": "csv",
  "order": "1",
  "settings": {
    "refresh_interval": "5s"
  },
  "mappings": {
    "properties": {
      "age": {
        "type": "double"
      },
      "name": {
        "type": "keyword"
      },
      "skill": {
        "type": "keyword"
      },
      "city": {
        "type": "keyword"
      }
    }
  }
}

file.csv

name,age,skill,city
Kim,100,JavaScript,Seoul
Park,200,Python,Busan
Lee,300,Java,Incheon

logstash.conf

input {
    beats {
        port => 5044
    }
    tcp {
        port => 50000
    }
    file {
        path => "/usr/share/logstash/input/file.csv"
        start_position => "beginning"
        sincedb_path => "NUL"
    }
}

filter {
    csv {
        separator => ","
        skip_header => "true"
        columns => ["name", "age", "skill", "city"]
    }
    mutate {
        remove_field => ["message", "@timestamp", "host", "@version"]
    }
}

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
        user => "elastic"
        password => "${LOGSTASH_INTERNAL_PASSWORD}"
        index => "csv"
        manage_template => true
        template => '/usr/share/logstash/config/csv-template.json'
        template_name => "csv"
        template_overwrite => true
    }
    stdout {
        codec => "rubydebug"
    }
}

docker-compse.yml

version: '3.7'
......
  logstash:
    build:
      context: logstash/
      args:
        ELASTIC_VERSION: ${ELASTIC_VERSION}
    volumes:
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro,Z
      - ./logstash/config/csv-template.json:/usr/share/logstash/config/csv-template.json:ro,Z
      - ./logstash/pipeline:/usr/share/logstash/pipeline:ro,Z
      - ./logstash/input:/usr/share/logstash/input:ro,Z
      - ./logstash/jar/postgresql-42.6.0.jar:/usr/share/logstash/postgresql.jar
......

Result

[stephenb (Stephen Brown)]

Hi @inkweon7269 Welcome to the community.

What version are you using?

You need to set the index patterns something like this shown here this is example

"index_patterns": [ "csv", "csv-*"],

Otherwise the template won't match the index being created... That's how the template gets applied

[inkweon7269]

Please respond. Thank you. We are using all versions as 8.x. (Elasticsearch, logstash, kibana)

I accessed the link you provided and made the changes, and it's working properly now. Thank you!