Hello Badger, Thanks but though I am now getting output, it's not line by line.
I read the doc link but didn't understand it much.
Now All lines get wrapped into 1 big line and there are duplicates in the same line. It's a bit wierd..
I tried with and also without the overwrite option.
input {
file {
path => "/tmp/lines.log"
start_position => "beginning"
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601}"
negate => true
what => "previous"
}
}
}
filter {
grok { pattern_definitions => { "MyTimestamp" => "\d{2}-%{WORD}-\d{4} %{TIME}" } match => { "message" => "%{MyTimestamp:timestamp}\s+%{GREEDYDATA:message}" } overwrite => [ "message" ] }
date {
match => [ "[@metadata][timestamp]", "dd-MMM-yyyy HH:mm:ss.SSS" ]
target => "@timestamp"
}
if "_grokparsefailure" in [tags] {
drop { }
}
if "_dateparsefailure" in [tags] {
drop { }
}
}
output {
file {
path => "/tmp/logstash.out"
}
}
Basically, if there are like 3 lines in a log file like below,
where let's say the 2nd line is a multiline of 100 lines or so,
then I would like to see 3 lines outputted as well.
The 2nd line should wrap up all the 100 lines below, until it reaches the 3rd line.
Some example input having 3 such lines:
12-Sep-2023 20:24:49.921 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin Match [Server/Service/Connector] failed to set property [debug] to [0]
12-Sep-2023 20:25:33.836 WARNING [Catalina-utility] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread [http-exec-something-3 url: /rest/filter/1.0/service/etc; user: someuser (id=[nnn12]) has been active for [71,125] milliseconds (since [9/12/23 7:20 AM]) to serve the same request for [https://some.url.com/rest/table-filter/] and may be stuck (configured thresh
old for this StuckThreadDetectionValve is [60] seconds). There is/are [1] thread(s) in total that are monitored by this Valve and may be stuck.
java.lang.Throwable
at java.base@11.0.17/java.net.SocketInputStream.socketRead0(Native Method)
at java.base@11.0.17/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base@11.0.17/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base@11.0.17/java.net.SocketInputStream.read(SocketInputStream.java:140)
+more lines
12-Sep-2023 20:26:36.565 WARNING [http-exec-6 url: /some/url/filter/1.0/; user: someone] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI https://some.url.com/rest/filter/1.0/...defmeb2ajfgv8usdfuj53b41d&_=1694389812926, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @ParamForm will work as expected. Resource methods consuming the request body by other means will not work as expected.
Could something like that be easily achieved if I try something else?
Thanks v much again.