# Parse with Grok an HTTP communication

**URL:** <https://discuss.elastic.co/t/parse-with-grok-an-http-communication/153465>\
**Category:** Logstash\
**Created:** [October 22, 2018, 6:48pm UTC](https://discuss.elastic.co/t/parse-with-grok-an-http-communication/153465 "2018-10-22T18:48:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![camay123](https://avatars.discourse-cdn.com/v4/letter/c/7cd45c/32.png) [@camay123](https://discuss.elastic.co/u/camay123)\
**Post date:** [October 22, 2018, 6:48pm UTC](https://discuss.elastic.co/t/parse-with-grok-an-http-communication/153465/1 "2018-10-22T18:48:11Z")

</div>

I am trying to parse an HTTP header communication with grok/logstash, here is a sample comm:

```
GET /msdownload/update/v3/static/trustedr/en/authrootstl.cab?07f093d6b2ea8d03 HTTP/1.1
Connection: Keep-Alive
Accept: */*
User-Agent: Microsoft-CryptoAPI/10.0
Host: ctldl.windowsupdate.com

```

I have the following hacky way of doing it:

grok {  
match =\> {"client\_payload" =\> "(?\<request\_method\>HEAD|PUT|GET|POST|DELETE) %{DATA:request\_path} HTTP/%{NUMBER:request\_version}::(?\<channel\_rest\>._)User-Agent: %{DATA:request\_user-agent}::(?\<channel\_rest2\>._)" }  
}

Which gives me a few fields, but my problem relies in the fact that the order of the HTTP headers is not always the same, and sometimes User-Agent: will come before Host: and vice versa....

Basically I want to have the following values extracted:

HTTPMETHOD  
URIPATH  
HTTPVERSION  
USER-AGENT  
HOST

Any less hacky way of doing it ?

---

<div class="post-metadata">

**Author:** ![camay123](https://avatars.discourse-cdn.com/v4/letter/c/7cd45c/32.png) [@camay123](https://discuss.elastic.co/u/camay123)\
**Post date:** [October 23, 2018, 1:25am UTC](https://discuss.elastic.co/t/parse-with-grok-an-http-communication/153465/2 "2018-10-23T01:25:08Z")

</div>

I am trying with ruby, with this filter:

```
ruby {
	
	code => '
                if event.get("client_payload")
                        m = event.get("client_payload").match /(?<request_method>^(GET|POST|PUT|HEAD|DELETE)) (?<request_path>.*) HTTP\/(?<request_version>[0-9]{1}\.[0-9]{1})/

                        if m
                                event.set("request_path", m[:request_path])
                                event.set("request_method", m[:request_method])
                                event.set("request_version", m[:request_version])
                        end

						useragent = event.get("client_payload").match /User-Agent: (?<user_agent>.*?)(::|$)/
                        if useragent[:user_agent]
                                event.set("request_user_agent", useragent[:user_agent])
                        end

						host = event.get("client_payload").match /Host: (?<request_host>.*?)(::|$)/
                        if host[:request_host]
                                event.set("request_host", host[:request_host])
                        end
                end
		'
	}

```

However, I get alot of error in the logstash logs, such as:

[2018-10-22T21:22:58,991][ERROR][logstash.filters.ruby] Ruby exception occurred: undefined method `[]' for nil:NilClass

Just not certain where to go from here.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 20, 2018, 1:25am UTC](https://discuss.elastic.co/t/parse-with-grok-an-http-communication/153465/3 "2018-11-20T01:25:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.