How to add a timestamp from the previous line to the current line

Elastic Stack Logstash
1

In my log file, some of the lines don't have a timestamp. I want to assign the previous line's timestamp to the lines that don't have the timestamp.

I tried "memorize" but that's giving some errors.

Is there a way we can do this?

2

Welcome!

Which tool are you using to collect the logs?

3

I am new to ELK and doing some tests. I am copying the files to a folder manually and from there using logstash passing to the Elasticsearch

This is my logstash config input portion.

input {
  file {
    path => "C:/../../logwithexceptions22.log"
    start_position => "beginning" 
	
  }
}
4

I moved your question to #elastic-stack:logstash

Have a look at Aggregate filter plugin | Logstash Reference [7.15] | Elastic but I'm not sure if this could help.

That's weird that your logs are not coming with a timestamp for every line of log. May be you could fix the logs?

Could you share some sample lines of logs?

5 
10/01/2021 14:24:55.840 JST Thread-4 (22.002.01) INFORMATIVE: SubProcess "/deployment/abcdef/ConfigServer" appears to have been manually started
10/01/2021 14:24:55.847 JST Thread-4 (22.001.18) ACCOUNTING: Process /deployment/abcdef/ConfigServer state is changed to CMDL_MANUAL. Process has started manually or by request.
10/01/2021 14:25:00.836 JST main (20.002.03) CRITICAL: Cannot find configuration for abcdefg during AdminAgentServer.configAdminAgent(). 
com.hp.siu.utils.EntryNotFoundException: /deployment/abcdefg
	at com.hp.siu.utils.ConfigManager.readConfigTree(ConfigManager.java:276)

I want to add the stack trace of the logs(last 2 lines) with the previous line's timestamp. Otherwise, it gives the grok parse failure, or even if I handle it properly, it gets the date and time of the processing time for these lines.

6

You could use a multiline codec on the file input to combine the error and the stacktrace into a single event. Something like

codec => multiline {
    pattern => "^\d{2}/\d{2}/\d{4} "
    negate => true
    what => previous
    auto_flush_interval => 5
}
7

Thank you Badger.. It's working

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.