How to extract timestamp and log level from message and add it as a filed

Elastic Stack Logstash
1

Hi

I want to extract timestamp and log level from message and want to add it as a field can you please suggest some filter or grok pattern to do this
sample log :
2022-10-12 12:02:32,611 [DEBUG][pool-5-thread-14][com.hybris.service.workflow.worker.order.InitiateFulfilmentWorker][][] after obtaining lock

2

Dissect, faster

	 dissect {
      mapping => {
        "message" => "%{time} [%{level}][%{thread}][%{method}][%{field1}][%{field2}] %{logmsg}"
      }

Grok, should prefer for this case

    grok { 
       match => { "message" => "%{TIMESTAMP_ISO8601:time}%{SPACE}\[%{LOGLEVEL:loglevel}\]%{SPACE}\[%{DATA:tread}\]%{SPACE}\[%{DATA:method}\]%{SPACE}\[%{DATA:field1}\]%{SPACE}\[%{DATA:field2}\]%{SPACE}%{GREEDYDATA:logmsg}" }

    }

Convert to the date format

    date {
       match => ["timestamp", "yyyy/MM/dd HH:mm:ss,SSS" ]
       target => "@timestamp"
       remove_field => ["timestamp"]
    }
3

I am getting this error after using this dissect and grok filter

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:oms-server-logs, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"=>\" at line 16, column 9 (byte 286) after filter {\n     dissect {\n      mapping => {\n        \"message\" => \"%{time} [%{level}][%{thread}][%{method}][%{field1}][%{field2}] %{logmsg}\"\n      }\n    \n   grok ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:199:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:381:in `block in converge_state'"]}
4

Add 1 more }

dissect {
      mapping => {
        "message" => "%{time} [%{level}][%{thread}][%{method}][%{field1}][%{field2}] %{logmsg}"
      }
}
5

Thank you this dissect is working

6

match => ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]

It should be like this

1 Like
7

Yes, sorry my mistake, copy+paste.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.