Hi Elastic/Filebeat Team,
I am trying to configure filebeat along with logstash and Elasticsearch but not getting expected result in Kibana dashboard as logs are getting split into multiple line please find sample logs below:
DATA FLOW : Filebeat >> Logstash >> Elasticsearch >> Kibana
Logs Structure:
Hi Elastic Team,
I am trying to configure filebeat along with logstash and Elasticsearch but not getting expected result in Kibana dashboard as logs are getting split into multiple line please find sample logs and filebeat config below:
Logs Structure:
2019 Jan 20 23:53:40:360 GMT +0530 BW.xyz-xyz Info [BW-User] - Job-7851213-2 [Logger/LogToLocal.process/Log]: Error in: Services/xyz/Status02_02.process/Status02>Processes/time/Status02.process/qwerty.xyz>Processes/References/qwerty/qwerty.xyz.process/Invoke qwerty par Status check with request payload: <?xml version="1.0" encoding="UTF-8"?> <ns0:Status02Request xmlns:ns0="http://www.abc.com/IN/abc"><MessageHeader><head:MessageHeader xmlns:head="http://www.abc.com/IT/EAG/Common/Header_01" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.abc.com/IT/EAG/Common/Header_01" xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:v1="http://www.abc.com/IN/new/mysee/yiu/v1" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><head:From><1234</head:ConversationId><head:MessageId>1234</head:MessageId><head:BusinessObjectId>Acc</head:BusinessObjectId><head:Timestamp>2019-01-20T23:53:40+05:30</head:Timestamp></head:HeaderFields></head:MessageHeader></MessageHeader><MessageBody><ns0:Status02Request xmlns:ns0="http://www.abc.com/IN/new/mysee/Status/v2" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><TxnId>AVC1234</TxnId></ns0:Status02Request></MessageBody></ns0:Status02Request> and error message: An IOException was thrown while trying to execute the Http method and stack trace: Job-7851213 Error in [Processes/References/qwerty/qwerty.xyz.process/Invoke qwerty par Status check] An IOException was thrown while trying to execute the Http method at com.myapp.plugin.share.http.client.JakartaHttpTransportDriver$RequestExecutor.run(Unknown Source) at com.myapp.pe.util.ThreadPool$ThreadPoolThread.run(Unknown Source) caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at java.net.Socket.<init>(Socket.java:434) at java.net.Socket.<init>(Socket.java:286) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) at com.myapp.plugin.share.http.client.JakartaHttpTransportDriver$RequestExecutor.run(Unknown Source) at com.myapp.pe.util.ThreadPool$ThreadPoolThread.run(Unknown Source)
2019 Jan 20 23:54:40:360 GMT +0530 BW.xyz-xyz Info [BW-User] - Job-7851213-2 [Logger/LogToLocal.process/Log]: Error in: Services/xyz/Status02_02.process/Status02>Processes/time/Status02.process/qwerty.xyz>Processes/References/qwerty/qwerty.xyz.process/Invoke qwerty par Status check with request payload: < xmlns:ns0="http://www.abc.com/IN/abc"><MessageHeader><head:MessageHeader xmlns:head="http://www.abc.com/IT/EAG/Common/Header_01" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.abc.com/IT/EAG/Common/Header_01" xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:v1="http://www.abc.com/IN/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><head:From><head:Id></head:Id></head:From><head:HeaderFields><head:ConversationId>19012</head:ConversationId><head:MessageId>190</head;/ns0:></MessageBody></ns0:est> and error message: An IOException was thrown while trying to execute the Http method and stack trace: Job-7851213 Error in [Processes/References/qwerty/qwerty.xyz.process/Invoke qwerty par Status check] An IOException was thrown while trying to execute the Http method at com.myapp.plugin.share.http.client.JakartaHttpTransportDriver$RequestExecutor.run(Unknown Source) at com.myapp.pe.util.ThreadPool$ThreadPoolThread.run(Unknown Source) caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at java.net.Socket.<init>(Socket.java:434) at java.net.Socket.<init>(Socket.java:286) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) at com.myapp.plugin.share.http.client.JakartaHttpTransportDriver$RequestExecutor.run(Unknown Source) at com.myapp.pe.util.ThreadPool$ThreadPoolThread.run(Unknown Source)
Regards,
Abhi