Convert to timestamp

[Pinno_Lin]

Hi,

I have time record in syslog and want to convert to timestamp

Agent_Local_Time="2018/5/14 上午 09:58:59"
Agent_Local_Time="2018/5/11 下午 09:52:20"

It's 12-hour system. The "上午" is A.M and "下午" is P.M

How to get correct time to @timestamp?

thanks.

[magnusbaeck (Magnus Bäck)]

The date filter only recognizes AM and PM so you'll have to replace "上午" with AM and "下午" with PM. Use a mutate filter's gsub option.

[Pinno_Lin]

Hi, magnusbaeck,

Thank for your reply. I test gsub option in test file. but it show the error. Logstash can't parse Chinese word?

input {
file {
path => "/u1/mailSyslog/Sample1"
start_position => "beginning"
sincedb_path => "/dev/null"
tags => ["LAB"]
codec => plain {
charset => "UTF-8"
}
}
}

filter {
if "LAB" in [tags] {
mutate {
gsub => [ "[dg][Agent_Local_Time]", "[上午]", "AM" ]
}
}

output {
if "LAB" in [tags] {
file {
path => "/tmp/lab.txt"
codec => rubydebug
}
}
}

[2018-05-15T09:47:04,023][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Createˆ]", "AM"\t]\n }\t\n}\n\noutput {\n\tif ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:inwith_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in block in converge_state'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:inblock in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:inconverge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:inblock in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

[magnusbaeck (Magnus Bäck)]

Yeah, that might be the problem. Are you sure you've saved the configuration file as UTF-8?

[Pinno_Lin]

Hi magnusbaeck,

I had use other ways to get result i want. but it's not smart.
Thank for your reply. it's give some idea to resolve.

grok {
match => {
"Agent_Local_Time" => "%{GREEDYDATA:AgentDate} %{GREEDYDATA:AgentDatedep} %{GREEDYDATA:AgentTime}"
}
}
if "上午" in [AgentDatedep] {
mutate {
add_field => { "EventTime" => "%{AgentDate} %{AgentTime} AM" }
}
}
if "下午" in [AgentDatedep] {
mutate {
add_field => { "EventTime" => "%{AgentDate} %{AgentTime} PM" }
}
}

            date {
                    match => ["EventTime", "yyyy/MM/dd hh:mm:ss a"]
                    timezone => "Asia/Taipei"
                    target => ["EventTime"]
            }