I have to ingest a text file with multilines running filebeat 7.1.1.
But the format is different from anything I've had so far.
Here below is an example.
Each line starts with timestamp+loglevel+"[stderr]"+ "(componentIdentifier)" + text.
Any idea how to logically group these lines ?
The componentIdentifier is a good start but no idea how to remove timestamps from lines after the first one, also the timestamp is not the same on all lines from a multiine event as the decimal parts increased during the log.
I searched in filebeat and logstash doc but found nothing relevant or helping me.
Regards,
Jean-Marc
2020-12-17 06:29:47,533 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,533 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,533 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,533 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,533 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,533 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,534 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 06:29:47,534 ERROR [stderr] (default task-2051) Serializing ContextWrapModel for EntityStoreModel(JmxBeanOperation)
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) com.isencia.passerelle.edm.engine.RequestProcessingException: REQUE
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) - Context:
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.passerelle.edm.engine.impl.facade.DefaultRequestBro
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at bgc.dar.edm.soap.diagnostic.v3.service.DARDiagnosticPortTypeImp
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at sun.reflect.GeneratedMethodAccessor307.invoke(Unknown Source)
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMetho
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at java.lang.reflect.Method.invoke(Method.java:498)
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.commons.jaxws.AbstractServiceDelegate.callService(A
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.commons.jaxws.internal.WSServlet.doPost(WSServlet.j
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at javax.servlet.http.HttpServlet.service(HttpServlet.java:523)
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at org.eclipse.equinox.http.servlet.internal.HttpServiceRuntimeImp
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at org.eclipse.equinox.http.servlet.internal.registration.Endpoint
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at org.eclipse.equinox.http.servlet.internal.servlet.FilterChainIm
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at com.isencia.commons.jaxws.logging.ProviderLogger.doFilter(Provi
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at org.eclipse.equinox.http.servlet.internal.HttpServiceRuntimeImp
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at org.eclipse.equinox.http.servlet.internal.registration.FilterRe
2020-12-17 08:44:48,280 ERROR [stderr] (default task-2058) at org.eclipse.equinox.http.servlet.internal.servlet.FilterChainIm
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequ
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at io.undertow.server.Connectors.executeRootHandler(Connectors.jav
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextC
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueEx
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(En
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(Enhanced
2020-12-17 08:44:48,282 ERROR [stderr] (default task-2058) at java.lang.Thread.run(Thread.java:748)
2020-12-17 10:00:19,084 ERROR [stderr] (default task-2080) com.isencia.passerelle.edm.engine.RequestProcessingException: REQUE
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) - Context:
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.passerelle.edm.engine.impl.facade.DefaultRequestBro
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.passerelle.edm.engine.impl.facade.DecisionEngineFac
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at bgc.dar.edm.soap.diagnostic.v3.service.DARDiagnosticPortTypeImp
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at sun.reflect.GeneratedMethodAccessor307.invoke(Unknown Source)
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMetho
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at java.lang.reflect.Method.invoke(Method.java:498)
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.commons.jaxws.AbstractServiceDelegate.callService(A
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.commons.jaxws.internal.WSServlet.doPost(WSServlet.j
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at javax.servlet.http.HttpServlet.service(HttpServlet.java:523)
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at org.eclipse.equinox.http.servlet.internal.HttpServiceRuntimeImp
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at org.eclipse.equinox.http.servlet.internal.registration.Endpoint
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at org.eclipse.equinox.http.servlet.internal.servlet.FilterChainIm
2020-12-17 10:00:19,085 ERROR [stderr] (default task-2080) at com.isencia.commons.jaxws.logging.ProviderLogger.doFilter(Provi