How to match IP address to hostname without dns [Solved]

Elastic Stack Logstash
1

I've been attempting to deploy ELK and I'm trying to figure out how to translate IP addresses into hostnames. Currently feeding in syslog from some switches and host field comes up as IP address, not a problem but I'd quite like to have hostname listed as well.

The DNS filter appears to do this but the issue is we don't use DNS, does anyone have any input on the best way to do this? ideally using /etc/hosts as the data source.

2

https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html will get you heading in the right direction.

3

If you have all your entries in /etc/hosts, then you can just setup a local caching DNS server e.g. on ubuntu you can use dnsmasq, which by default is configured to resolve from/etc/hosts entries.

Assuming your logstash instance with the filter will run on ubuntu, it's as simple as "apt-get install dnsmasq", no further tuning required.

1 Like
4

Thanks yuphing, I've not used dnsmasq before. it worked perfectly but I'm having a little issue with my config of the dns filter. specifically this bit

reverse => [ "source_host", "field_with_address" ]

so my config reads

filter {
  if [type] == "syslog" {
    dns {
      add_field => { "hostname" => "" }
      reverse => [ "hostname", "host" ]
      add_tag => [ "dns_lookup" ]
    }
  }
}

which results in

"message" => "<189>cli[12801]: [cli.NOTICE]: user admin: CLI exiting\n",
"@version" => "1",
"@timestamp" => "2015-06-19T15:14:12.482Z",
          "type" => "syslog",
          "host" => "172.20.0.161",
          "tags" => [
  [0] "_grokparsefailure_sysloginput"
],
      "priority" => 0,
      "severity" => 0,
      "facility" => 0,
"facility_label" => "kernel",
"severity_label" => "Emergency"

but if I change it to

      reverse => [ "host" ]

then it works perfectly, all be it not how I want...

{
"message" => "<189>cli[12944]: [cli.NOTICE]: user admin: CLI exiting\n",
"@version" => "1",
"@timestamp" => "2015-06-19T15:19:45.124Z",
"type" => "syslog",
"host" => [
    [0] "172.20.0.161",
    [1] "IB1"
],
"tags" => [
    [0] "_grokparsefailure_sysloginput",
    [1] "dns_lookup"
],
"priority" => 0,
"severity" => 0,
"facility" => 0,
"facility_label" => "kernel",
"severity_label" => "Emergency",
"hostname" => ""
}

I can't see where I'm going wrong :frowning:

5

I think the documentation's example is a bit misleading. You reverse an array of IP addresses, i.e. every string in there will be reversed, then depending on the action, either replaced or appended with the looked up hostname.

Not, as it would seem on first read, the string in the first position being replaced with the hostname of the IP address in the second position.

In my use case, as I'm doing netflow parsing, I reverse both source and destination host IPs by first creating then copying the IP address into the hostname strings, then using:

reverse => [ "hostname_source", "hostname_dest" ]
action => replace

(As an aside, my reverse DNS lookup for external hostnames slows down my ELK stack's ability to ingest the inputs too much, even with DNS caching, so I ended up only doing internal host looksup)

1 Like
6

Ah! it all make sense now.

I had not figured out the reason but I managed to achieve what I wanted by doing the following:

filter {
  if [type] == "syslog" {
    mutate {
      add_field => { "hostname" => "%{host}" }
    }
    dns {
      action => "replace"
      reverse => [ "hostname" ]
      add_tag => [ "dns_lookup" ]
    }
  }
}