# How to regex urls with Grok

**URL:** <https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011>\
**Category:** Logstash\
**Created:** [November 2, 2022, 3:20pm UTC](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011 "2022-11-02T15:20:52Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![lamdba](https://avatars.discourse-cdn.com/v4/letter/l/bc8723/32.png) [@lamdba](https://discuss.elastic.co/u/lamdba)\
**Post date:** [November 2, 2022, 3:20pm UTC](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011/1 "2022-11-02T15:20:52Z")

</div>

Hello there,

Sorry to bother but I'm slowly working my way to fully use and understand the stack and right now I'm blocked with a (I guess) simple problem with Logstash.

I parse multiple events through Logstash and everything is going fine, doing few operations on it and so for. But right now I'm a bit lost when I want to perform a regex on the events I parse.

I receive json events, in those event I have a field 'URL' with a complete url that contains some informations I would like to extract before going to the output.  
The entries received look like those:

```auto
{
	"id": "AVERYRANDOMID",
	"timeRef": "2022-10-25T10:45:05.000+02:00",
	"url": "https://www.mywebsite.com/the-name-of-my-page-12345678",
	"queryParams": {
		"utm_medium": "email",
		"utm_source": "newsletter",
		"utm_campaign": "MyCampaign"
	},
}
{
	"id": "ANOTHERRANDOMID",
	"timeRef": "2022-10-25T10:45:05.000+02:00",
	"url": "https://www.mywebsite.com/",
	"queryParams": {
	},
}
{
	"id": "INEEDANOTHERONE",
	"timeRef": "2022-10-25T10:45:05.000+02:00",
	"url": "https://www.mywebsite.com/4567890",
	"queryParams": {
	},
}
{
	"id": "LASTONEIPROMISE",
	"timeRef": "2022-10-25T10:45:05.000+02:00",
	"url": "https://www.mywebsite.com/the-name-of-my-page-12345678&fbclid=IwAR2Kf_HojrEdNy",
	"queryParams": {
		"utm_medium": "email",
		"utm_source": "newsletter",
		"utm_campaign": "MyOtherCampaign"
	},
}

```

At the end I would need, for those 4 entries, to retrieve the ID of the article -if present, from the URL. All rules I found about that ID :

- Always between 6 to 8 numbers
- Can follow the title of the article/page
- Can be directly on the root of the website (without the title
- Can be (ok, will be) followed by extra characters (shared from social network, from utm campaign, etc..)
- ID can also be absent from the url (because people can sometimes (ahah) use the tools incorrectly)

I have the feeling it can be done via logstash and I can avoid some processing after the output, but right now I'm kinda of stuck. And I'm trying to avoid completely Ruby on that one if that's possible.

Did I miss something? If someone could point me to the right direction, would be much appreciated. 🙂

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 2, 2022, 4:30pm UTC](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011/2 "2022-11-02T16:30:33Z")

</div>

You might be able to use something as simple as

```
grok { match => { "url" => "(?<articleId>\d{6,8})" } }

```

If you need it to be at the end of the URI path then maybe

```
grok { match => { "url" => "%{URIPROTO}://(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:%{URIPATH:[@metadata][uri]})?" } }    
grok { match => { "[@metadata][uri]" => "(?<articleId>\d{6,8})$" } }

```

---

<div class="post-metadata">

**Author:** ![lamdba](https://avatars.discourse-cdn.com/v4/letter/l/bc8723/32.png) [@lamdba](https://discuss.elastic.co/u/lamdba)\
**Post date:** [November 3, 2022, 9:46am UTC](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011/3 "2022-11-03T09:46:21Z")

</div>

Hello @Badger !

Thanks a million for taking the time. Your first idea was the right and work even better than I expected. Got everything I need with that Grok filter and a nice little add\_field got me all the data stored where I want them.

I had some difficulties to understand how Grok works/can help me but thanks to your solution I feel like I won 2 weeks of testing and swearing at my screen. Once again, thanks for the help. 🙂

Edit : And I just found out the add\_field is not even necessary. 😃  
Slowly groking.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 1, 2022, 9:47am UTC](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011/4 "2022-12-01T09:47:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.