Java log Parsing with Filebeat and Logstash

Elastic Stack Elasticsearch
1

So I have a java log that I want to consume using filebeat. When I test in the local env I succeed, but when I try to use the production env the log is not successfully parsed. Because in the local env I use logstash but in the production env I use filebeat.

Here is an example of the log

[9/6/24 10:10:57:038 WIB] 00055cd2 SystemOut     O 2024-09-06 10:10:57 HTransaction [ERROR] com.m24.shared.exception.UnsafeSessionCloseException: This not Valid Transaction
com.m24.shared.exception.UnsafeSessionCloseException: This not Valid Transaction
	at com.m24.shared.plugin.db.hibernate.HTransaction.released(HTransaction.java:54)
	at com.m24.shared.plugin.db.hibernate.HSession.close(HSession.java:113)
	at com.m24.front.ui.channeling.BayarAngsuranSyariahScreen.doPayment(BayarAngsuranSyariahScreen.java:1211)
	at com.m24.front.ui.channeling.BayarAngsuranSyariahScreen.doSave(BayarAngsuranSyariahScreen.java:498)
	at com.m24.front.ui.channeling.BayarAngsuranSyariahScreen$6$1.onClose(BayarAngsuranSyariahScreen.java:295)
	at org.vaadin.dialogs.DefaultConfirmDialogFactory$2.buttonClick(DefaultConfirmDialogFactory.java:152)

Logs that are successfully parsed to alstic are only

[9/6/24 10:10:57:038 WIB] 00055cd2 SystemOut     O 2024-09-06 10:10:57 HTransaction [ERROR] com.m24.shared.exception.UnsafeSessionCloseException: This not Valid Transaction
com.m24.shared.exception.UnsafeSessionCloseException: This not Valid Transaction

When message have tag the java message is not include.
ex:
at org.vaadin.dialogs.DefaultConfirmDialogFactory$2.buttonClick(DefaultConfirmDialogFactory.java:152)

This configuration Logstash and Filebeat:
My Logstash in Local Env:

type: filestream
id: Convert
enabled: true
paths:
- Convertnode1*/SystemOut.log
- Convertnode2*/SystemErr.log
fields:
application_name: convert
serverName: convert-currency
fields_under_root: true
multiline.pattern: '^/'
multiline.negate: true
multiline.match: after

Logstash

file {
path => "/var/log/logstash/dump_hasil/*.log"
start_position => "beginning"
sincedb_path => "/dev/null"
codec => multiline {
pattern => "\d{4}-\d{2}-\d{2}-\d{2}.\d{2}.\d{2}.\d{6}+\d{3}"
negate => true
what => "previous"

Please help

2

What version of filebeat ...

Looks like you are using filestream input so your multiline syntax is wrong , not using the parsers syntax see here and you would still need the correct pattern...

Example...

parsers:
- multiline:
    type: pattern
    pattern: '^\['
    negate: true
    match: after
3

Thank you it's work now