### Author:  [@shikha1](https://discuss.elastic.co/u/shikha1)
#### Post date: [March 7, 2022, 1:07pm UTC](https://discuss.elastic.co/t/combine-filebeat-logs-to-ship-to-elastic-search-based-on-unique-trace-id-without-logstash/298968/1 "2022-03-07T13:07:11Z")
I have configured filebeat for shipping logs of my spring boot application directly to Elasticsearch without configuration of logstash. I want to configure filebeat multiline regex in a way that it can combine all the logs in a single record having same transactionId. Can anyone please suggest ? Here the part of log configuration looks like in filebeat.yml:
```auto
filebeat.inputs:
- type: log
paths:
- E://filebeat//*.log
reload.enabled: true
reload.period: 300s
enabled: true
multiline:
type: pattern
pattern: '^\['
negate: true
match: after
Log Sample:
2022-03-07 07:08:44 [ERROR] | transactionId=6af42925-48df-4f49-95a4-aa0e43b152ed | c.example.utility.advice.LoggingAdvice - Method Signature: Boolean com.example.config.service.impl.ServiceImpl.testLogger() - Line No : 1345 - Exception : null
2022-03-07 07:08:50 [ERROR] | transactionId=51a2d445-574b-4992-b867-a7fc5ee4b473 | c.example.utility.advice.LoggingAdvice - Method Signature: Boolean com.example.config.service.impl.ServiceImpl.testLogger() - Line No : 1345 - Exception : null
2022-03-07 05:26:08 [ERROR] | transactionId=17f4f815-ee74-4658-a8d7-347fea6dd9d1 | o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause
java.lang.NullPointerException: null
at com.example.config.service.impl.ServiceImpl.testLogger(EventServiceImpl.java:1345) ~[classes/:na]
at com.example.config.service.impl.ServiceImpl$FastClassBySpringCGLIB$983a4fb2.invoke() ~[classes/:na]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.9.jar:5.3.9]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) ~[spring-aop-5.3.9.jar:5.3.9]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.9.jar:5.3.9]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.9.jar:5.3.9]
at
```
---
### Author:  [@Tetiana\_Kravchenko](https://discuss.elastic.co/u/Tetiana_Kravchenko)
#### Post date: [March 9, 2022, 1:18pm UTC](https://discuss.elastic.co/t/combine-filebeat-logs-to-ship-to-elastic-search-based-on-unique-trace-id-without-logstash/298968/3 "2022-03-09T13:18:40Z")
Hi @shikha1!
I think your multiline pattern is not correct. Could you try this configuration:
```auto
multiline:
type: pattern
pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2}'
negate: true
match: after
```
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [March 9, 2022, 1:26pm UTC](https://discuss.elastic.co/t/combine-filebeat-logs-to-ship-to-elastic-search-based-on-unique-trace-id-without-logstash/298968/4 "2022-03-09T13:26:30Z")
I think that the multiline pattern should be just:
```auto
pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
```
There is no `[` to escape, this pattern will get all lines that start with a date in the format `yyyy-MM-dd`.
---
### Author:  [@shikha1](https://discuss.elastic.co/u/shikha1)
#### Post date: [March 10, 2022, 5:43am UTC](https://discuss.elastic.co/t/combine-filebeat-logs-to-ship-to-elastic-search-based-on-unique-trace-id-without-logstash/298968/5 "2022-03-10T05:43:08Z")
Thanks Leandrojmp and Tetiana. This pattern seems to be working fine for me when I want to combine exception logs in my java application. However, I want to combine logs based on unique words such as trace id or transaction id in single records along with these exception logs. Is there any suggestion on that you can provide please ?
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [March 10, 2022, 1:50pm UTC](https://discuss.elastic.co/t/combine-filebeat-logs-to-ship-to-elastic-search-based-on-unique-trace-id-without-logstash/298968/6 "2022-03-10T13:50:34Z")
That's not possible with filebeat alone, if you want to aggregate lines based on some specific ID you will need to use Logstash to do that.
Logstash has the [aggregate filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html) to join lines based on a ID.
You would need to send your logs to logstash and then to Elasticsearch.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 7, 2022, 3:51pm UTC](https://discuss.elastic.co/t/combine-filebeat-logs-to-ship-to-elastic-search-based-on-unique-trace-id-without-logstash/298968/7 "2022-04-07T15:51:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.