# Squid Module: Splitting multiple logs which are combined as a single message in Kibana **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 14, 2020, 4:12pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609 "2020-12-14T16:12:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ron\_g](https://avatars.discourse-cdn.com/v4/letter/r/7feea3/32.png) [@ron\_g](https://discuss.elastic.co/u/ron_g)\ **Post date:** [December 14, 2020, 4:12pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609/1 "2020-12-14T16:12:45Z")
We have the problem that all logs which are send from our Squid server over Filebeat with the Squid module are combined as single messages in Kibana. The message contains 5 up to 15 entrys which are in fact single log lines from the Squid server. The squid server is using the default log output format, but we tried different formats with no solution. Logformat squid: `> %ts.%03tu %6tr %>a %Ss/%03>Hs % 1607961779.062 7 0.0.0.0 TCP_MISS/206 5004 GET http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/xxxx? - HIER_DIRECT/0.0.0.0 application/octet-stream` We are NOT using any "multiline" options, just the default filebeat configuration with the Squid module which shipps everything directly to ES. We also tried to ship the logs from Squid to logstash but didn't find a solution for the multiple entries. And we want to use the Squid module at the end. Is there a more precise guide for the squid settings than this one: > **[Squid module | Filebeat Reference \[7.10\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.10/filebeat-module-squid.html)** Similar asked here: > [@Splitting multiple logs combined as a single message to multiple event log entries](https://discuss.elastic.co/t/splitting-multiple-logs-combined-as-a-single-message-to-multiple-event-log-entries/177756): > > I have installed filebeat to forward apache logs to logstash to parse and store it in elasticsearch so that i can view it in Kibana, but i'm facing the below issue Multiple logs are getting parsed as a single message - Example 1: - 64.77.63.8 - - [15/Apr/2016:03:28:42 -0400] "POST /WeblRunner?requestID=1878587528&splitID=0 HTTP/1.1" 200 70124 64.77.63.8 - - [15/Apr/2016:03:28:47 -0400] "POST /Web?requestID=1878587679&splitID=0 HTTP/1.1" 200 53487 64.77.63.8 - - [15/Apr/2016:03:28:48 -0400] "P… Moreover the filebeat syslog is full of this type of error messages: `> filebeat[38211]: 2020-12-14T16:57:15.980+0100#011ERROR#011[processor.javascript]#011console/console.go:54#011extract_page failed for 'www.google.com:443'` Which [processor.javascript] extract\_page process is the cause of this? This errors are in fact from the Squid module, of course not when using logstash. Example of the multiple logs per entry: ![kibananlog](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d245b5eeb9b942095b139bb26ab2b8824868ff8b.png) ---
**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\ **Post date:** [December 15, 2020, 8:23am UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609/2 "2020-12-15T08:23:05Z")
It smells like a bug and reproduced the other community user's case. Would you mind opening a Github issue for filebeat and copy a couple of faulty log lines? ---
**Author:** ![ron\_g](https://avatars.discourse-cdn.com/v4/letter/r/7feea3/32.png) [@ron\_g](https://discuss.elastic.co/u/ron_g)\ **Post date:** [December 16, 2020, 2:31pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609/3 "2020-12-16T14:31:35Z")
I have an update, after reading Squid documentation (again and again): > "...being UDP this module may drop packets when the network is under load or congested." So we changed the Squid log output from UDP to TCP and now there is only 1 instead of 5 up to 20 messages in a single entry. As an example of the load: we have around 15.000 - 20.000 entries per 5 minutes. But: The "[processor.javascript] extract\_page" error still exists. It writes an error message per log entry, so around 15-20k error messages in syslog per 5 minutes. Did you mean this error for the Github issue? Because I couldn't find a single entry regarding this error, no matter which search engine I use. ---
**Author:** ![Lee\_Hinman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_hinman/32/74973_2.png) [@Lee\_Hinman](https://discuss.elastic.co/u/Lee_Hinman)\ **Post date:** [December 17, 2020, 3:13pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609/4 "2020-12-17T15:13:29Z")
Hi. Great new that you got it working with TCP. The extract\_page error you are seeing should be a debug message, we accidentally left it as an error. This will be fixed in newer release. ---
**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\ **Post date:** [December 17, 2020, 3:24pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609/5 "2020-12-17T15:24:09Z")
> [@ron\_g](#): > > So we changed the Squid log output from UDP to TCP and now there is only 1 instead of 5 up to 20 messages in a single entry. Turns out that Squid can send multiple lines of logs in a single UDP packet: [http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-logging-to-UDP-logs-multiple-lines-at-the-same-time-td4685384.html](http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-logging-to-UDP-logs-multiple-lines-at-the-same-time-td4685384.html) While Beats' udp input treats each packet as a single message and doesn't split at newlines. I've created an enhancement request: [Filebeat udp input: Support line\_delimiter option · Issue #23195 · elastic/beats · GitHub](https://github.com/elastic/beats/issues/23195) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 14, 2021, 5:24pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609/6 "2021-01-14T17:24:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.