How to configurate filebeats for muliline log file


(Swelltt0576) #1

hi, i have a file just like this:

2015-12-21 17:44:31,922 ERROR c.e.u.e.e.E2BService. [pool-2-thread-519] Connect to localhost:80 [localhost/127.0.0.1] failed: Connection refused
org.apache.http.conn.HttpHostConnectException: Connect to localhost:80 [localhost/127.0.0.1] failed: Connection refused
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.1.jar:4.5.1]
	at com.elinku.ums.et.e2b.E2BService.sendE2BRequest(E2BService.java:48) [ums-et-api-1.1.3-RELEASES.jar:?]
	at com.elinku.ums.core.MpQQService.processQ2ETextMo(MpQQService.java:160) [ums-core-service-1.1.3-RELEASES.jar:?]
	at com.elinku.ums.core.MpQQService.processQ2ERequest(MpQQService.java:128) [ums-core-service-1.1.3-RELEASES.jar:?]
	at com.elinku.ums.core.CoreService.processQ2E(CoreService.java:60) [ums-core-service-1.1.3-RELEASES.jar:?]
	at com.elinku.ums.core.Q2EProcessor.run(Q2EProcessor.java:32) [ums-core-service-1.1.3-RELEASES.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_66]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_66]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_66]
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_66]
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_66]
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_66]
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_66]
	at java.net.Socket.connect(Socket.java:589) ~[?:1.8.0_66]
	at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74) ~[httpclient-4.5.1.jar:4.5.1]
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) ~[httpclient-4.5.1.jar:4.5.1]
	... 17 more

when i use the default config to load the log into logstash , i find that there more than 1 message in the logstash.

it is clearly that filebeat provide multi-event for this log, but i want there event can be combined into one event.

how can i do?

thanks .


(Steffen Siering) #2

multiline support is about to be added to filebeat. For reference see: https://github.com/elastic/beats/issues/461 and https://github.com/elastic/beats/pull/570.

I think most recent logstash also has support for multiline.


(Swelltt0576) #3

i got, thank you very much


(system) #4