### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 2, 2015, 6:56pm UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/1 "2015-12-02T18:56:24Z")
Is it possible to extract the domain only from a URL using lostash:
URL:
[e10.whatsapp.net](http://e10.whatsapp.net)
[mtalk.google.com](http://mtalk.google.com)
[teredo.ipv6.microsoft.com](http://teredo.ipv6.microsoft.com)
[extshort.weixin.qq.com](http://extshort.weixin.qq.com)
Domain
[whatsapp.net](http://whatsapp.net)
[google.com](http://google.com)
> **[Microsoft - Official Home Page](https://www.microsoft.com/en-us/)**
>
> At Microsoft our mission and values are to help people and businesses throughout the world realize their full potential.
[qq.com](http://qq.com)
---
### Author:  [@vtst2412](https://discuss.elastic.co/u/vtst2412)
#### Post date: [December 3, 2015, 5:32am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/2 "2015-12-03T05:32:06Z")
How about:
```
filter {
if [URL] {
ruby {
begin
event['Domain'] = event['URL'].match(/^.*\.((.*?)\.(.*?))$/)[1]
end
}
}
}
```
Might be able to do it with grok to, but I don't see an obvious way. It might require more brain power than just using ruby.
Cheers.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 3, 2015, 7:06am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/3 "2015-12-03T07:06:48Z")
Untested:
```auto
filter {
grok {
match => ["URL", "\.(?[^.]+\.[^.]+)$"]
}
}
```
Splitting the string on each period, grabbing the two last elements, and joining them back together should be a lot more efficient though.
---
### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 3, 2015, 8:33am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/4 "2015-12-03T08:33:13Z")
This will however take only the two last part or the url and return this. How do you then handle the other domains ending with more than one criteria e.g.
tachiarai.fukuoka.jp
blogspot.co.uk
wa.edu.au
[ap-southeast-2.compute.amazonaws.com](http://ap-southeast-2.compute.amazonaws.com)
I have found a very interesting site that updates all the domains on the internet on a regular bases, here is the link: [https://publicsuffix.org/list/public\_suffix\_list.dat](https://publicsuffix.org/list/public_suffix_list.dat)
Is it possible to use this file as a reference for the domains and then just add one more -1 to add the actual name of the site?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 3, 2015, 8:43am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/5 "2015-12-03T08:43:46Z")
> This will however take only the two last part or the url and return this.
Yes, of course. It wasn't obvious what you meant by domain.
> I have found a very interesting site that updates all the domains on the internet on a regular bases, here is the link: [https://publicsuffix.org/list/public\_suffix\_list.dat](https://publicsuffix.org/list/public_suffix_list.dat)
> Is it possible to use this file as a reference for the domains and then just add one more -1 to add the actual name of the site?
Sure, a custom filter plugin could easily do that.
---
### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 3, 2015, 10:36am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/6 "2015-12-03T10:36:25Z")
Could you kindly elaborate, how to go about doing this?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 3, 2015, 10:38am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/7 "2015-12-03T10:38:56Z")
Have a look at this page:
[https://www.elastic.co/guide/en/logstash/current/\_how\_to\_write\_a\_logstash\_filter\_plugin.html](https://www.elastic.co/guide/en/logstash/current/_how_to_write_a_logstash_filter_plugin.html)
---
### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 3, 2015, 6:48pm UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/8 "2015-12-03T18:48:02Z")
Thank you for the link, is there a forum also for ruby coding to seek assistance when getting stuck?
---
### Author:  [@vtst2412](https://discuss.elastic.co/u/vtst2412)
#### Post date: [December 3, 2015, 7:12pm UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/9 "2015-12-03T19:12:13Z")
Ruby is pretty straight-forward. Most of the folks here can probably provide you with assistance. Just read the [documentation](https://www.ruby-lang.org/en/documentation/).
---
### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 4, 2015, 5:59am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/10 "2015-12-04T05:59:32Z")
Thank you, will try and find a starting point somewhere.
---
### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 7, 2015, 10:04am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/11 "2015-12-07T10:04:42Z")
Hi All, a quick question, I am very new to the Plugin writing and Ruby, would it be possible for someone to provide some guidance. After going through the documentation and information it is still quite overwhelming. So what I would like to do a take the url and extract only the domain from this by comparing the public\_suffix\_list.dat ([https://publicsuffix.org/list/public\_suffix\_list.dat](https://publicsuffix.org/list/public_suffix_list.dat)) list to the url and minus one field. So if there is a url with www.bbc.co.uk comparing this to the public\_suffix\_list.dat file a hist would be on .co.uk then minus one to get bbc.co.uk.
Any assistance in getting started would be appreciated.
---
### Author:  [@RobT](https://discuss.elastic.co/u/RobT)
#### Post date: [December 7, 2015, 3:39pm UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/13 "2015-12-07T15:39:29Z")
Look at the second answer from here:
>
Here's the lib they reference:
> **[weppos/publicsuffix-ruby](https://github.com/weppos/publicsuffix-ruby)**
>
> publicsuffix-ruby - Domain name parser for Ruby based on the Public Suffix List.
Looks fairly straightforward, hopefully it works out for you.
---
### Author:  [@Hans](https://discuss.elastic.co/u/Hans)
#### Post date: [December 8, 2015, 2:26pm UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/14 "2015-12-08T14:26:25Z")
Thank you very much Robt, I will try this.
---
### Author:  [@Jeremy\_Colton](https://discuss.elastic.co/u/Jeremy_Colton)
#### Post date: [April 18, 2016, 9:30am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/15 "2016-04-18T09:30:56Z")
So the grok filter posted doesn't work for all domains?
I have a 'referer' entry in my nginx log (eg "[https://abc.storage.googleapis.com/app/desktop.html?a=1](https://abc.storage.googleapis.com/app/desktop.html?a=1)) that I want to extract just the domain from, excluding the protocol. From the confusing grok patterns I have this:
match =\> { "referer" =\> "%{HOST:referer\_domain}" }
But I see in the logstash log that this fails. So, please please how do I do this using a grok pattern?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 18, 2016, 5:41pm UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/16 "2016-04-18T17:41:06Z")
> match =\> { "referer" =\> "%{HOST:referer\_domain}" }
Try this:
```
match => { "referer" => "%{URIPROTO}://%{URIHOST:referer_domain}" }
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:01am UTC](https://discuss.elastic.co/t/extracting-domain-from-url/36219/17 "2017-07-06T05:01:50Z")