# Parsing URL with Logstash (using ECS fields)

**URL:** <https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814>\
**Category:** Logstash\
**Created:** [October 30, 2019, 9:13am UTC](https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814 "2019-10-30T09:13:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vincent\_Maury](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent_maury/32/59973_2.png) [@Vincent\_Maury](https://discuss.elastic.co/u/Vincent_Maury)\
**Post date:** [October 30, 2019, 9:13am UTC](https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814/1 "2019-10-30T09:13:26Z")

</div>

Hi there  
i've been working on a way to parse url into ecs fields, and it seems to be working well.  
Hope it helps!

```
# Parser a url
filter {
    grok {
        match => [
            # match https://user:pwd@stuff.domain.com:8080/some/path?p1=v1&p2=v2#anchor
            "field", "%{URIPROTO:url.scheme}://(?:%{USER:url.username}:(?<url.password>[^@]*)@)?(?:%{IPORHOST:url.domain}(?::%{POSINT:url.port}))?(?:%{URIPATH:url.path}(?:%{URIPARAM:url.query}))?",
            # match stuff.domain.com:8080/some/path?p1=v1&p2=v2#anchor
            "field", "%{IPORHOST:url.domain}(?::%{POSINT:url.port})(?:%{URIPATH:url.path}(?:%{URIPARAM:url.query}))?",
            # match /some/path?p1=v1&p2=v2#anchor
            "field", "%{URIPATH:url.path}(?:%{URIPARAM:url.query})"
        ]
    }
    if "_grokparsefailure" not in [tags] {
        grok {
            match => {
                "url.query" => "^\?(?<url.query>[A-Za-z0-9$.+!*'|(){},~@%&/=:;_?\-\[\]<>]*)(?:#(?:%{WORD:url.fragment}))?"
            }
      overwrite => ["url.query"]
        }
        kv {
        source => "url.query"
        field_split => "&"
        target => "url.query.params"
        }
    }
}
```

---

<div class="post-metadata">

**Author:** ![Vincent\_Maury](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent_maury/32/59973_2.png) [@Vincent\_Maury](https://discuss.elastic.co/u/Vincent_Maury)\
**Post date:** [October 31, 2019, 11:57am UTC](https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814/2 "2019-10-31T11:57:49Z")

</div>

Here is a working example, also illustrating multiple matches (break\_on\_match true by default):

```
input {
  generator {
    count => 1
    message => 'https://user:pwd@stuff.domain.com:8080/some/path?p1=v1&p2=v2#anchor'
  }
}

# Parser a url
filter {
	grok {
		match => {
      message => [
  			# match https://user:pwd@stuff.domain.com:8080/some/path?p1=v1&p2=v2#anchor
  			"%{URIPROTO:url.scheme}://(?:%{USER:url.username}:(?<url.password>[^@]*)@)?(?:%{IPORHOST:url.domain}(?::%{POSINT:url.port}))?(?:%{URIPATH:url.path}(?:%{URIPARAM:url.query}))?",
  			# match stuff.domain.com:8080/some/path?p1=v1&p2=v2#anchor
  			"%{IPORHOST:url.domain}(?::%{POSINT:url.port})(?:%{URIPATH:url.path}(?:%{URIPARAM:url.query}))?",
  			# match /some/path?p1=v1&p2=v2#anchor
  			"%{URIPATH:url.path}(?:%{URIPARAM:url.query})"
      ]
		}
	}
	if "_grokparsefailure" not in [tags] {
		grok {
			match => {
				"url.query" => "^\?(?<url.query>[A-Za-z0-9$.+!*'|(){},~@%&/=:;_?\-\[\]<>]*)(?:#(?:%{WORD:url.fragment}))?"
			}
      overwrite => ["url.query"]
		}
		kv {
  		source => "url.query"
  		field_split => "&"
  		target => "url.query.params"
		}
	}
}

output {
  stdout {
    codec => json
  }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 28, 2019, 11:57am UTC](https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814/3 "2019-11-28T11:57:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.