java.lang.IllegalArgumentException: Illegal pattern component: t


(Marcos Oliveira Soares) #1

EN: I have a problem with Logstash and would like help in trying to resolve.
PT: Eu tenho um problema com Logstash e gostaria de ajuda para tentar resolver.

EN: I'm trying to extract data from a relational database and send it to the elasticsearch.
PT: Estou tentando extrair dados de um baco de dados relacional e enviar para o elasticsearch.

EN: My Settings:
PT: Minhas configurações:

/etc/logstash/conf.d/01-main.conf
input {
tcp {
port => 5000
type => "logstash-tcp"
}
http {
port => 5043
type => "logstash-http"
response_headers => {
"Access-Control-Allow-Origin" => ""
"Content-Type" => "text/plain"
"Access-Control-Allow-Headers" => "Origin, X-Requested-With, Content-Type, Accept"
}
}
beats {
port => 5044
type => "logstash-beats"
}
gelf {
port => 12201
type => "logstash-gelf"
}
file {
path => "/var/log/elk"
type => "logstash-file"
start_position => "beginning"
}
jdbc {
jdbc_driver_library => "/etc/logstash/jdbc/ojdbc7.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:@IP:PORTA/SID"
jdbc_user => "username"
jdbc_password => "password"
jdbc_validate_connection => true
jdbc_fetch_size => 50000
schedule => "
* * * * America/Sao_Paulo"
sql_log_level => "debug"
statement => "SELECT * FROM TTR"
type => "logstash-jdbc"
}
}

output {
# elasticsearch {
# hosts => [ "http://elasticsearch:9200" ]
# user => elastic
# password => "${elastic_password}"
# index => "logstash-6.5.4-%{+type}-%{+YYYY.MM.dd}"
# document_id => "%{id}"
# codec => json
# }
stdout {
codec => rubydebug
}
}

EN: Output to the above configuration:
PT: Saída para a configuração acima:

logstash[14695]: {
logstash[14695]: "ttrclsautvo" => "ABC",
logstash[14695]: "ttrcod" => 123,
logstash[14695]: "ttridcisncpmf" => 0,
logstash[14695]: "ttrclsprc" => "ABC",
logstash[14695]: "ttrdes" => "ABC",
logstash[14695]: "ttrclslog" => "ABC",
logstash[14695]: "ttridchab" => 1,
logstash[14695]: "ttrclsstyaut" => nil,
logstash[14695]: "ttrcodest" => 0,
logstash[14695]: "@version" => "1",
logstash[14695]: "@timestamp" => 2019-01-15T16:55:11.709Z,
logstash[14695]: "type" => "logstash-jdbc",
logstash[14695]: "ttrdathorcrg" => 2006-02-10T03:00:00.000Z,
logstash[14695]: "ttrdesres" => "ABC",
logstash[14695]: "ttrclsaut" => "ABC",
logstash[14695]: "ttrclstot" => "ABC",
logstash[14695]: "ttrtip" => 0,
logstash[14695]: "ttrclsttr" => "ABC",
logstash[14695]: "ttrclssty" => "ABC",
logstash[14695]: "ttrclsprcagd" => "ABC"
logstash[14695]: }

EN: When I send the output to elasticsearch, the following error occurs:
PT: Quando envio a saída para o elasticsearch, ocorre o seguinte erro:

logstash[14157]: Exception in thread "Ruby-0-Thread-10: :1" Exception in thread "Ruby-0-Thread-7: :1" java.lang.IllegalArgumentException: Illegal pattern component: t
logstash[14157]: at org.joda.time.format.DateTimeFormat.parsePatternTo(DateTimeFormat.java:566)
logstash[14157]: at org.joda.time.format.DateTimeFormat.createFormatterForPattern(DateTimeFormat.java:687)
logstash[14157]: at org.joda.time.format.DateTimeFormat.forPattern(DateTimeFormat.java:177)
logstash[14157]: at org.logstash.StringInterpolation.evaluate(StringInterpolation.java:48)
logstash[14157]: at org.logstash.Event.sprintf(Event.java:262)
logstash[14157]: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_sprintf(JrubyEventExtLibrary.java:175)
logstash[14157]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_2_dot_4_minus_java.lib.logstash.outputs.elasticsearch.common.RUBY$method$event_action_tuple$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/common.rb:68)

EN: Structure of the table in the relational database:
PT: Estrutura da tabela no banco de dados relacional:

Nome          Nulo     Tipo          
------------- -------- ------------- 
TTRCOD        NOT NULL NUMBER(8)     
TTRDES        NOT NULL VARCHAR2(40)  
TTRDESRES     NOT NULL VARCHAR2(15)  
TTRTIP        NOT NULL NUMBER(1)     
TTRCODEST              NUMBER(8)     
TTRIDCISNCPMF NOT NULL NUMBER(1)     
TTRCLSTTR              VARCHAR2(120) 
TTRCLSLOG              VARCHAR2(120) 
TTRCLSAUT              VARCHAR2(120) 
TTRCLSPRC              VARCHAR2(120) 
TTRCLSTOT              VARCHAR2(120) 
TTRCLSSTY              VARCHAR2(120) 
TTRCLSSTYAUT           VARCHAR2(120) 
TTRDATHORCRG           DATE          
TTRCLSPRCAGD           VARCHAR2(120) 
TTRCLSAUTVO            VARCHAR2(120) 
TTRIDCHAB     NOT NULL NUMBER(1)   

EN: I tried to configure with some answers that I found in the forum, but without success.
PT: Tentei configurar com algumas respostas que encontrei no fórun, porém sem sucesso.


#2

I think it is trying to interpret %{+type} as a time format. Does it work if you change it to

index => "logstash-6.5.4-%{type}-%{+YYYY.MM.dd}"

(Marcos Oliveira Soares) #3

EN: Thank you very much @Badger
EN: My whole focus was on the jdbc input and filter.

PT: Muito obrigado @Badger
PT: Todo o meu foco estava na entrada jdbc e filtro


(system) closed #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.