EN: I have a problem with Logstash and would like help in trying to resolve.
PT: Eu tenho um problema com Logstash e gostaria de ajuda para tentar resolver.
EN: I'm trying to extract data from a relational database and send it to the elasticsearch.
PT: Estou tentando extrair dados de um baco de dados relacional e enviar para o elasticsearch.
EN: My Settings:
PT: Minhas configurações:
/etc/logstash/conf.d/01-main.conf
input {
tcp {
port => 5000
type => "logstash-tcp"
}
http {
port => 5043
type => "logstash-http"
response_headers => {
"Access-Control-Allow-Origin" => ""
"Content-Type" => "text/plain"
"Access-Control-Allow-Headers" => "Origin, X-Requested-With, Content-Type, Accept"
}
}
beats {
port => 5044
type => "logstash-beats"
}
gelf {
port => 12201
type => "logstash-gelf"
}
file {
path => "/var/log/elk"
type => "logstash-file"
start_position => "beginning"
}
jdbc {
jdbc_driver_library => "/etc/logstash/jdbc/ojdbc7.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:@IP:PORTA/SID"
jdbc_user => "username"
jdbc_password => "password"
jdbc_validate_connection => true
jdbc_fetch_size => 50000
schedule => " * * * * America/Sao_Paulo"
sql_log_level => "debug"
statement => "SELECT * FROM TTR"
type => "logstash-jdbc"
}
}
output {
# elasticsearch {
# hosts => [ "http://elasticsearch:9200" ]
# user => elastic
# password => "${elastic_password}"
# index => "logstash-6.5.4-%{+type}-%{+YYYY.MM.dd}"
# document_id => "%{id}"
# codec => json
# }
stdout {
codec => rubydebug
}
}
EN: Output to the above configuration:
PT: Saída para a configuração acima:
logstash[14695]: {
logstash[14695]: "ttrclsautvo" => "ABC",
logstash[14695]: "ttrcod" => 123,
logstash[14695]: "ttridcisncpmf" => 0,
logstash[14695]: "ttrclsprc" => "ABC",
logstash[14695]: "ttrdes" => "ABC",
logstash[14695]: "ttrclslog" => "ABC",
logstash[14695]: "ttridchab" => 1,
logstash[14695]: "ttrclsstyaut" => nil,
logstash[14695]: "ttrcodest" => 0,
logstash[14695]: "@version" => "1",
logstash[14695]: "@timestamp" => 2019-01-15T16:55:11.709Z,
logstash[14695]: "type" => "logstash-jdbc",
logstash[14695]: "ttrdathorcrg" => 2006-02-10T03:00:00.000Z,
logstash[14695]: "ttrdesres" => "ABC",
logstash[14695]: "ttrclsaut" => "ABC",
logstash[14695]: "ttrclstot" => "ABC",
logstash[14695]: "ttrtip" => 0,
logstash[14695]: "ttrclsttr" => "ABC",
logstash[14695]: "ttrclssty" => "ABC",
logstash[14695]: "ttrclsprcagd" => "ABC"
logstash[14695]: }
EN: When I send the output to elasticsearch, the following error occurs:
PT: Quando envio a saída para o elasticsearch, ocorre o seguinte erro:
logstash[14157]: Exception in thread "Ruby-0-Thread-10: :1" Exception in thread "Ruby-0-Thread-7: :1" java.lang.IllegalArgumentException: Illegal pattern component: t
logstash[14157]: at org.joda.time.format.DateTimeFormat.parsePatternTo(DateTimeFormat.java:566)
logstash[14157]: at org.joda.time.format.DateTimeFormat.createFormatterForPattern(DateTimeFormat.java:687)
logstash[14157]: at org.joda.time.format.DateTimeFormat.forPattern(DateTimeFormat.java:177)
logstash[14157]: at org.logstash.StringInterpolation.evaluate(StringInterpolation.java:48)
logstash[14157]: at org.logstash.Event.sprintf(Event.java:262)
logstash[14157]: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_sprintf(JrubyEventExtLibrary.java:175)
logstash[14157]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_2_dot_4_minus_java.lib.logstash.outputs.elasticsearch.common.RUBY$method$event_action_tuple$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/common.rb:68)
EN: Structure of the table in the relational database:
PT: Estrutura da tabela no banco de dados relacional:
Nome Nulo Tipo
------------- -------- -------------
TTRCOD NOT NULL NUMBER(8)
TTRDES NOT NULL VARCHAR2(40)
TTRDESRES NOT NULL VARCHAR2(15)
TTRTIP NOT NULL NUMBER(1)
TTRCODEST NUMBER(8)
TTRIDCISNCPMF NOT NULL NUMBER(1)
TTRCLSTTR VARCHAR2(120)
TTRCLSLOG VARCHAR2(120)
TTRCLSAUT VARCHAR2(120)
TTRCLSPRC VARCHAR2(120)
TTRCLSTOT VARCHAR2(120)
TTRCLSSTY VARCHAR2(120)
TTRCLSSTYAUT VARCHAR2(120)
TTRDATHORCRG DATE
TTRCLSPRCAGD VARCHAR2(120)
TTRCLSAUTVO VARCHAR2(120)
TTRIDCHAB NOT NULL NUMBER(1)
EN: I tried to configure with some answers that I found in the forum, but without success.
PT: Tentei configurar com algumas respostas que encontrei no fórun, porém sem sucesso.