# Parse subdomain from fqdn in logstash

**URL:** https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535
**Category:** Logstash
**Created:** [May 20, 2021, 12:39pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535 "2021-05-20T12:39:07Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![tushar.bansal](https://avatars.discourse-cdn.com/v4/letter/t/e495f1/32.png) [@tushar.bansal](https://discuss.elastic.co/u/tushar.bansal)
#### Post date: [May 20, 2021, 12:39pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/1 "2021-05-20T12:39:07Z")

</div>

Hi,

I have several events coming in logstash with info of web link accessed.

example:  
DNS query is called for the name **[crl.sectigo.com](http://crl.sectigo.com)**, type 28  
DNS query is called for the name **[repo.maven.apache.org](http://repo.maven.apache.org)**, type 2  
DNS query is called for the name **[crl.maven.apache.random.org](http://crl.maven.apache.random.org)**, type 1

I parsed them using grok in a separate field.

example:  
[crl.sectigo.com](http://crl.sectigo.com)  
[repo.maven.apache.org](http://repo.maven.apache.org)  
[crl.maven.apache.random.org](http://crl.maven.apache.random.org)

I want to have one more field with just last 2 subdomains.

example:  
[sectigo.com](http://sectigo.com)  
[apache.org](http://apache.org)  
[random.org](http://random.org)

I tried many logics with grok etc. but of no use. Thing is I can get a random domain name, no clue with how many subdomains in it. I need to have only last 2 in a field.

---

<div class="post-metadata">

### Author: ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)
#### Post date: [May 20, 2021, 1:10pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/2 "2021-05-20T13:10:00Z")

</div>

Hi,

This grok pattern give you the result you want :  
`%{WORD:name}.%{WORD:TLD}$`  
With this pattern, you sai you want two words separated by a dot and they are placed at the end of your field (with the character $)

Cad.

---

<div class="post-metadata">

### Author: ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [May 20, 2021, 1:39pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/3 "2021-05-20T13:39:53Z")

</div>

You have to be careful with this because country tlds have an extra characters on the front of a domain. So `qbc.co` becomes `qbc.co.uk` and this logic would only return `co.uk`. I don't think logstash has a registered domain processor but filebeat and elasticsearch do. See [Registered Domain | Filebeat Reference [7.12] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/processor-registered-domain.html).

---

<div class="post-metadata">

### Author: ![cknz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cknz/32/9640_2.png) [@cknz](https://discuss.elastic.co/u/cknz)
#### Post date: [May 22, 2021, 11:26am UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/4 "2021-05-22T11:26:01Z")

</div>

The 'tld' filter exists precisely for this purpose. To do this correctly is very non-trivial and cannot be done algorithmically; you need to use the 'public suffix' data, which is what the 'tld' plugin (and others like it) use.

Cheers,  
Cameron

(PS. In case you come across a plugin I wrote called logstash-filter-dnssummary, I would suggest you stick with the 'tld' plugin, just because its more maintained; unless perhaps you care about Unicode and IDNA)

---

<div class="post-metadata">

### Author: ![tushar.bansal](https://avatars.discourse-cdn.com/v4/letter/t/e495f1/32.png) [@tushar.bansal](https://discuss.elastic.co/u/tushar.bansal)
#### Post date: [May 22, 2021, 1:13pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/5 "2021-05-22T13:13:31Z")

</div>

Thanks all for tld suggestion. Thing is, my logstash server does not have access to internet. And tld filter needs to be manually installed. It does not come with the bundle. Any suggestions?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 22, 2021, 3:16pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/6 "2021-05-22T15:16:04Z")

</div>

> [@tushar.bansal](#):
>
> Thing is, my logstash server does not have access to internet.

Use the [plugin manager](https://www.elastic.co/guide/en/logstash/current/offline-plugins.html).

---

<div class="post-metadata">

### Author: ![cknz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cknz/32/9640_2.png) [@cknz](https://discuss.elastic.co/u/cknz)
#### Post date: [May 22, 2021, 11:45pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/7 "2021-05-22T23:45:16Z")

</div>

Or if you only lack **direct** access to the internet, but can still go through a HTTP proxy, you could use it via a proxy (port 3128 is the default port for a Squid proxy; other proxies vary)

```auto
https_proxy=myproxy.example.com:3128 http_proxy=myproxy.example.com:3128 logstash_plugin install logstash-filter-tld

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 21, 2021, 11:46pm UTC](https://discuss.elastic.co/t/parse-subdomain-from-fqdn-in-logstash/273535/8 "2021-06-21T23:46:10Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
