# Squid logs (access logs)

**URL:** <https://discuss.elastic.co/t/squid-logs-access-logs/171274>\
**Category:** Logstash\
**Created:** [March 7, 2019, 10:37am UTC](https://discuss.elastic.co/t/squid-logs-access-logs/171274 "2019-03-07T10:37:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\
**Post date:** [March 7, 2019, 10:37am UTC](https://discuss.elastic.co/t/squid-logs-access-logs/171274/1 "2019-03-07T10:37:33Z")

</div>

I am trying to create index for a squid log....although the filebeat creates the pipeline for logstash but due to some error in my grok I am not able to segregate the logs..P.S help me..  
sample of my log file:

"  
551728235.295 866048 10.130.2.58 TCP\_TUNNEL/200 8429 CONNECT [play.google.com:443](http://play.google.com:443) artidesh HIER\_DIRECT/172.217.160.206 -  
1551728237.817 61121 10.27.2.174 TCP\_MISS/200 319 GET [http://su.ff.avast.com/R/A28KIGJmYzZlYWM5OGFkNDQzYWI5OWU5MmQ5ZThiY2RmMWVkEgQGBwIZGHgiAf8qBwgEEOS292syCggAELC492sYgAI465eMkAFCINs5qHO9YDIoqtdhpEPo6hYMAcz6Dpa\_oZsRDcsK0yfhSICDmBA=](http://su.ff.avast.com/R/A28KIGJmYzZlYWM5OGFkNDQzYWI5OWU5MmQ5ZThiY2RmMWVkEgQGBwIZGHgiAf8qBwgEEOS292syCggAELC492sYgAI465eMkAFCINs5qHO9YDIoqtdhpEPo6hYMAcz6Dpa_oZsRDcsK0yfhSICDmBA=) sumitk HIER\_DIRECT/77.234.45.65 application/octet-stream  
1551728238.234 109983 10.26.2.44 TCP\_TUNNEL/200 587 CONNECT [c.go-mpulse.net:443](http://c.go-mpulse.net:443) harish HIER\_DIRECT/104.81.21.87 -  
1551728238.997 60974 10.126.2.11 TCP\_MISS/200 319 GET [http://su.ff.avast.com/R/A3cKIGRkMDc1NTJkYTEwZDRlMGNiZjMwMWJkZDQxY2I2YzUxEgQCAwMZGHgiAf4qBwgEEK-492sqBwgDEKDT8GYyCggEEK-492sYgAo4spKckAFCIAWim9dpoLM1oNFpiMFNau5nnsWSKWVZZP3OPYtUxXoRSICDKA==](http://su.ff.avast.com/R/A3cKIGRkMDc1NTJkYTEwZDRlMGNiZjMwMWJkZDQxY2I2YzUxEgQCAwMZGHgiAf4qBwgEEK-492sqBwgDEKDT8GYyCggEEK-492sYgAo4spKckAFCIAWim9dpoLM1oNFpiMFNau5nnsWSKWVZZP3OPYtUxXoRSICDKA==) mukund HIER\_DIRECT/77.234.41.236 application/octet-stream"

my grok looks like:

input {  
beats {  
port =\> "5044"  
}  
}  
filter {  
grok {  
match =\> { "message" =\> "%{INT:timestamp}.%{INT}\s\*%{NUMBER:request\_msec:float} %{IPORHOST:src\_ip} %{WORD:cache\_result}/%{NUMBER:response\_status:int} %{NUMBER:response\_size:int} %{WORD:http\_method} (%{URIPROTO:http\_proto}://)?%{IPORHOST:dst\_host}(?::%{POSINT:port})?(?:%{DATA:uri\_param})? %{USERNAME:cache\_user} %{WORD:request\_route}/(%{IPORHOST:forwarded\_to}|-) %{GREEDYDATA:content\_type}"}  
}  
geoip {  
source =\> "dst\_ip"  
}

}

output {

elasticsearch {  
hosts =\> ["10.11.109.7:9200"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{+YYYY-MM}"  
document\_type =\> "%{[@metadata][type]}"  
}

}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 7, 2019, 3:45pm UTC](https://discuss.elastic.co/t/squid-logs-access-logs/171274/2 "2019-03-07T15:45:11Z")

</div>

Your pattern works for me...

```
   "request_msec" => 61121.0,
     "cache_user" => "sumitk",
         "src_ip" => "10.27.2.174",
    "http_method" => "GET",
   "forwarded_to" => "77.234.45.65",
   "content_type" => "application/octet-stream",
      "timestamp" => "1551728237",
"response_status" => 200,
  "response_size" => 319,
       "dst_host" => "su.ff.avast.com",
   "cache_result" => "TCP_MISS",
  "request_route" => "HIER_DIRECT",
        "message" => "1551728237.817 61121 10.27.2.174 TCP_MISS/200 319 GET http://su.ff.avast.com/R/A28KIGJmYzZlYWM5OGFkNDQzYWI5OWU5MmQ5ZThiY2RmMWVkEgQGBwIZGHgiAf8qBwgEEOS292syCggAELC492sYgAI465eMkAFCINs5qHO9YDIoqtdhpEPo6hYMAcz6Dpa_oZsRDcsK0yfhSICDmBA= sumitk HIER_DIRECT/77.234.45.65 application/octet-stream",
     "http_proto" => "http",
      "uri_param" => "/R/A28KIGJmYzZlYWM5OGFkNDQzYWI5OWU5MmQ5ZThiY2RmMWVkEgQGBwIZGHgiAf8qBwgEEOS292syCggAELC492sYgAI465eMkAFCINs5qHO9YDIoqtdhpEPo6hYMAcz6Dpa_oZsRDcsK0yfhSICDmBA="
```

---

<div class="post-metadata">

**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\
**Post date:** [March 8, 2019, 4:15am UTC](https://discuss.elastic.co/t/squid-logs-access-logs/171274/3 "2019-03-08T04:15:49Z")

</div>

But while creating Index in elasticsearch it ain't segregating.  
My elasticsearch output is:

|t message| |1551759655.972 103 10.31.2.143 TCP\_MISS/206 66117 GET [http://2.au.download.windowsupdate.com/c/msdownload/update/software/crup/2017/08/ppaddin-x-none\_dafe4123f452d9c1a822e3844dd317e342915b89.cab](http://2.au.download.windowsupdate.com/c/msdownload/update/software/crup/2017/08/ppaddin-x-none_dafe4123f452d9c1a822e3844dd317e342915b89.cab) - HIER\_DIRECT/104.118.7.49 application/vnd.ms-cab-compressed|  
I am really stuck at this point

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 5, 2019, 4:15am UTC](https://discuss.elastic.co/t/squid-logs-access-logs/171274/4 "2019-04-05T04:15:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.