Trouble configuring Logstash to Squid Logs

Elastic Stack Elasticsearch
1

My Logstash (1.4.1) config to read Squid log is shown below:

input { file{ path => "/var/log/squid3/access.log" }}filter { grok
{ match => ["message","%{NUMBER:timestamp} \s+
%{NUMBER:request_msec:float} %{IPORHOST:src_ip}
%{WORD:cache_result}/%{NUMBER:response_status:int}
%{NUMBER:response_size:int} %{WORD:http_method}
(%{URIPROTO:http_proto}://)?%{IPORHOST:dst_host}(?::%{POSINT:port})?(?:%{URIPATHPARAM:uri_param})?
%{USERNAME:cache_user} %{WORD:request_route}/(%{IPORHOST:forwarded_to}|-)
%{GREEDYDATA:content_type}"] add_tag => ["squid"] }
date { match => [ "timestamp", "ISO8601" ] }}output {
elasticsearch { host => localhost } stdout { codec => rubydebug }}
I have tested the pattern (using Grok debugger) and it is ok. However,
Logstash does nothing. It doesn't produce any error message and shows
nothing when I use "stdout { }".

Can somebody advise me on how to troubleshoot? Many thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/12c57a63-af80-4e77-9251-e724d01ac824%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

While not a squid user myself, I do my debugging on logstash by
increasingly expantind the grok filter.
For example, try replacing all the content of you match with
{GREEDYDATA:the_message} and see if outputs something.

If it does start adding more filters:
%{NUMBER:timestamp}%{GREEDYDATA:the_message} and so on, until you find the
problem.

On Thursday, June 5, 2014 5:39:24 AM UTC-3, SG Chan wrote:

My Logstash (1.4.1) config to read Squid log is shown below:

input { file{ path => "/var/log/squid3/access.log" }}filter {
grok { match => ["message","%{NUMBER:timestamp} \s+
%{NUMBER:request_msec:float} %{IPORHOST:src_ip}
%{WORD:cache_result}/%{NUMBER:response_status:int}
%{NUMBER:response_size:int} %{WORD:http_method}
(%{URIPROTO:http_proto}://)?%{IPORHOST:dst_host}(?::%{POSINT:port})?(?:%{URIPATHPARAM:uri_param})?
%{USERNAME:cache_user} %{WORD:request_route}/(%{IPORHOST:forwarded_to}|-)
%{GREEDYDATA:content_type}"] add_tag => ["squid"] }
date { match => [ "timestamp", "ISO8601" ] }}output {
elasticsearch { host => localhost } stdout { codec => rubydebug }}
I have tested the pattern (using Grok debugger) and it is ok. However,
Logstash does nothing. It doesn't produce any error message and shows
nothing when I use "stdout { }".

Can somebody advise me on how to troubleshoot? Many thanks in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b70d3404-1a73-4215-a57e-98bdb4195d1b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.