# Parse HTTP requests to the NASA Kennedy Space Center WWW server in Florida

**URL:** <https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009>\
**Category:** Logstash\
**Created:** [May 7, 2019, 2:57pm UTC](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009 "2019-05-07T14:57:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jailbreakerSN](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jailbreakerSN](https://discuss.elastic.co/u/jailbreakerSN)\
**Post date:** [May 7, 2019, 2:57pm UTC](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009/1 "2019-05-07T14:57:51Z")

</div>

Hi,

I'm working on Nasa Kennedy Space Center web server data set and I want to monitore them in Kibana.  
So what I want is to collect them with Filebeat and parse them with Logstash.  
So is somebody know how to configure the grok filter in order to retrieve all meaningfull information, then I could create visualisation in kibana dashboard.  
here is the log format

// 3066-7.usi.edu - - [04/Aug/1995:10:13:34 -0400] "GET /images/launchmedium.gif HTTP/1.0" 200 11853  
// [tiber.gsfc.nasa.gov](http://tiber.gsfc.nasa.gov) - - [04/Aug/1995:10:13:36 -0400] "GET /elv/DELTA/delta.gif HTTP/1.0" 200 2244  
// [unifex.ksc.nasa.gov](http://unifex.ksc.nasa.gov) - - [04/Aug/1995:10:13:38 -0400] "GET / HTTP/1.0" 304 0  
[unifex.ksc.nasa.gov](http://unifex.ksc.nasa.gov) - - [04/Aug/1995:10:13:38 -0400] "GET /images/MOSAIC-logosmall.gif HTTP/1.0" 304 0  
// [unifex.ksc.nasa.gov](http://unifex.ksc.nasa.gov) - - [04/Aug/1995:10:13:38 -0400] "GET /images/ksclogo-medium.gif HTTP/1.0" 304 0

Need help please!!

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [May 7, 2019, 3:22pm UTC](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009/2 "2019-05-07T15:22:47Z")

</div>

Hi @jailbreakerSN,

If you download and extract (or install from package) Logstash you should find a file like  
`vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/httpd` in the Logstash folder. There are some ready made patterns that might work or would only need a bit of editing to work for you

```
# cat vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/httpd
HTTPDUSER %{EMAILADDRESS}|%{USER}
HTTPDERROR_DATE %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}

# Log formats
HTTPD_COMMONLOG %{IPORHOST:clientip} %{HTTPDUSER:ident} %{HTTPDUSER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)
HTTPD_COMBINEDLOG %{HTTPD_COMMONLOG} %{QS:referrer} %{QS:agent}

# Error logs
HTTPD20_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{LOGLEVEL:loglevel}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:message}
HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \[pid %{POSINT:pid}(:tid %{NUMBER:tid})?\]( \(%{POSINT:proxy_errorcode}\)%{DATA:proxy_message}:)?( \[client %{IPORHOST:clientip}:%{POSINT:clientport}\])?( %{DATA:errorcode}:)? %{GREEDYDATA:message}
HTTPD_ERRORLOG %{HTTPD20_ERRORLOG}|%{HTTPD24_ERRORLOG}

# Deprecated
COMMONAPACHELOG %{HTTPD_COMMONLOG}
COMBINEDAPACHELOG %{HTTPD_COMBINEDLOG}

```

There are also many other ready made patterns like

```
HOSTNAME \b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b)

```

Combining these should give you what you want.

This is not very pretty but is almost complete

```
// %{HOSTNAME:hostname} - - \[%{MONTHDAY:day}\/%{MONTH:month}\/%{YEAR:year}:%{TIME:time} %{DATA:timezone}\] \"%{DATA:http_method} %{URIPATH:uri_path} %{DATA:protocol}\/%{NUMBER:version}\" %{NUMBER:responce_code}

```

The end of the line and the optional bits need to be handled.

---

<div class="post-metadata">

**Author:** ![jailbreakerSN](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jailbreakerSN](https://discuss.elastic.co/u/jailbreakerSN)\
**Post date:** [May 7, 2019, 3:42pm UTC](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009/3 "2019-05-07T15:42:01Z")

</div>

hi @A_B,

Thank for the reply, it helps me a lot.  
I try it and it seems to be well parsed with the %{COMMONAPACHELOG} patterns.

Thank again for the tips!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 4, 2019, 3:44pm UTC](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009/4 "2019-06-04T15:44:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.