Criar index com logstash utilizando settings customizado

Elastic Stack Elasticsearch
1

Boa tarde pessoal, estou tentando criar um Indice no Elasticsearch (7.0.1) a partir do Logstash (7.3.1) quando tento colocar um template de configurações não funciona (quando subo com as configurações padrões funciona sem problema ).

Meu arquivo de configuração
input {

    file {
        path => "/home/temp/*.*"
        start_position => "beginning"
        type => "product"
    }
    
    jdbc {
        type => "product"
        jdbc_connection_string => "jdbc:postgresql://postgres_b2b:5432/b2b"
        jdbc_user => "***"
        jdbc_password => "***"
        jdbc_driver_class => "org.postgresql.Driver"
        schedule => "* * * * *"
        statement => "my query"
        tracking_column_type => "timestamp"
    }

}

filter {
    if [type] == "product" { 
        aggregate {
            task_id => "%{product_id}"
            code => "mappings"
            push_previous_map_as_event => true
            timeout => 3
        }
    }

}


output {
    if [type] == "product" { 
        elasticsearch {
             index => "product"
             document_type => "_doc"
             document_id => "%{product_id}"
             manage_template => true                             
             template => "/home/temp/template.json"
             template_name => "template"
             template_overwrite => true
        }
    }
}

Meu template

{
        "settings" : {
            "analysis" : {
                "analyzer" : {
                    "default-template" : {
                        "tokenizer" : "standard",
                        "filter" : ["asciifolding","lowercase"]
                    }
                }
            }
        }
    }

Erro que sobe no console quando tento iniciar a aplicação

logstashb2b      |   output {
logstashb2b      |     elasticsearch {
logstashb2b      |       # This setting must be a path
logstashb2b      |       # File does not exist or cannot be opened /home/temp/template.json
logstashb2b      |       template => "/home/temp/template.json"
logstashb2b      |       ...
logstashb2b      |     }
logstashb2b      |   }
logstashb2b      | [2019-10-01T09:37:05,324][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:87:in `config_init'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:60:in `initialize'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:232:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:48:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:30:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:242:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:140:in `buildOutput'", "org/logstash/execution/JavaBasePipelineExt.java:50:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:24:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:325:in `block in converge_state'"]}

É importante ressaltar que meu template está com chmod 777 então acredito que não deve ser problema de permissão.

Any tips, or something wrong in my configuration file?

Alguma sugestão?

Obrigado.

2

O problema era relativo ao localização do meu template.
Ele estava fisicamente em minha máquina, porém meu logstash/elasticsearch rodam em containers do docker.
Incluí o template no docker e funcionou normalmente =)

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.