vipinkumar
(Vipin Kumar)
September 12, 2017, 5:22am
1
Hi All
I am working on csv files and in that file there is a column as DATE/TIME in which the date and time is coming in following format.
2017-09-12-0915
YYYY-mm-dd-HHmm
I tried to convert this to DATE/TIME to date_time type by using convert, but in Kibana its type is still string.
Below is my config file.
input {
beats {
port => "5043"
}
}
filter {
if[fields][document] == "ServerIf"
{
csv {
autodetect_column_names => true
separator => ","
convert => {
"DATE/TIME" => "date_time"
"EnquiryIn" => "integer"
"EnquiryOut" => "integer"
"BalanceUsed" => "integer"
"BalanceLeft" => "integer"
}
}
date {
match => ["DATE/TIME" , "yyyy-MM-dd HH:mm"]
target => "DATE/TIME"
}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}
Can anyone help me with this issue...How to convert string to date_time.
warkolm
(Mark Walkom)
September 12, 2017, 5:31am
2
That's not valid, there's no type like that.
That also doesn't match your pattern you originally posted;
vipinkumar:
YYYY-mm-dd-HHmm
vipinkumar
(Vipin Kumar)
September 12, 2017, 5:55am
3
Hi Mark
Correct me if I'm wrong, but as per this - https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping-date-format.html
I can use date_time instead of date right? In fact originally I tried using date_hour_minute, but it gave this error:
Pipeline aborted due to error {:exception=>#<LogStash::ConfigurationError: Invalid conversion types: date_hour_minute>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-csv-3.0.4/lib/logstash/filters/csv.rb:102:in register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:281:in
register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:292:in register_plugins'", "org/jruby/RubyArray.java:1613:in
each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:292:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:302:in
start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:226:in run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:398:in
start_pipeline'"]}
As for your second reply, yes you're right I'm actually trying to change the format YYYY-mm-dd-HHmm to yyyy-MM-dd HH:mm. Bases on your reply I guess this isn't the right approach. Could you kindly guide me.
I can use date_time instead of date right?
No. You're referencing the ES documentation which has nothing to do with Logstash.
The date pattern you need is YYYY-MM-dd-HHmm.
warkolm
(Mark Walkom)
September 12, 2017, 6:03am
5
system
(system)
Closed
October 10, 2017, 6:03am
6
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.