# Windows DNS incompatible character encodings

**URL:** <https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961>\
**Category:** Logstash\
**Created:** [May 2, 2016, 2:09pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961 "2016-05-02T14:09:59Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 2, 2016, 2:09pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/1 "2016-05-02T14:09:59Z")

</div>

I am having a constant issue while sending Windows DNS logs from filebeat to logstash.

This is the config I use during debug:

```
input {
    beats {
        port => XXXX
    }
}

filter {
  if [type] =="Win_DNS" {
    grok {
      patterns_dir => "/etc/logstash/patterns/dns"
      match => { "message" => "%{WINDNS}" }
      break_on_match => false
    }
    mutate {
      add_field => { "log_timestamp" => "%{log_date} %{log_time}" }
      lowercase => "dns_query_name"
    }
    if [dns_client_address] != "xx.yyy.zz.aaa" {
      mutate {
        add_field => { "dest_url" => "%{dns_query_name}" }
      }
    }
    else {
         drop { }
    }
    if [dest_url] == /^10\((?:2|3)\)/ or "dns_query_name" in [dest_url] {
         drop { }
    }
    mutate {
      gsub => ["dest_url","\([1-9][0-9]?\)", "." ]
      gsub => ["dest_url","\(0\)", " "]
      strip => "dest_url"
    }
    if [dest_url] =~ /10\.(?:[0-9]{1,2}|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.(?:[0-9]{1,2}|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.(?:[1-9][0-9]?|1[0-9]{2}|2[0-4][0-9]|25[0-4])\-(?:[0-9]{1,2}|1[0-9]{2}|2[0-4][0-9]|25[0-5])/ {
         drop { }
    }
    mutate {
      add_field => { "dest_ip" => "%{dest_url}" }
    }
    dns {
      resolve => ["dest_ip"]
      action => "replace"
      nameserver => "10.110.24.4"
    }
    if [dest_ip] !~ /[A-Za-z-_\/\\]/ {
      geoip {
        source => "dest_ip"
      }
    } else {
        grok {
          remove_field => ["dest_ip"]
        }
    }
    date {
      match => ["log_timestamp", "yyyyMMdd HH:mm:ss"]
    }
  }
}

```

The config works fine for a while, but the error that follows is what happens once it receives a certain line. I have not been able to track down the line that describes/causes the issue.

> {:timestamp=\>"2016-04-27T10:55:53.670000-0400", :message=\>"Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.", "exception"=\>#\<Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT\>, "backtrace"=\>["org/jruby/RubyString.java:3936:in `[]='", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:792:in `sender'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:508:in `each_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1035:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1034:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1033:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1031:in `resolv'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:503:in `each_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:502:in `each\_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:396:in `each_address'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:120:in `each\_address'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:119:in `each\_address'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:97:in `getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:249:in `getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:238:in `retriable_getaddress'", "org/jruby/RubyProc.java:281:in `call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:216:in `retriable_request'", "org/jruby/ext/timeout/Timeout.java:115:in `timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:215:in `retriable_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:237:in `retriable\_getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:120:in `resolve'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-

Any help would be great.  
Thank you.

```
/opt/logstash/bin/logstash --version
logstash 2.3.1

```

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 2, 2016, 2:11pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/2 "2016-05-02T14:11:07Z")

</div>

original message wouldn't let me put the full error:

> {:timestamp=\>"2016-04-27T10:55:53.670000-0400", :message=\>"Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.", "exception"=\>#\<Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT\>, "backtrace"=\>["org/jruby/RubyString.java:3936:in `='", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:792:in `sender'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:508:in `each\_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1035:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1034:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1033:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1031:in `resolv'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:503:in `each\_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:502:in `each_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:396:in `each\_address'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:120:in `each_address'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:119:in `each_address'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:97:in `getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:249:in `getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:238:in `retriable\_getaddress'", "org/jruby/RubyProc.java:281:in `call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:216:in `retriable\_request'", "org/jruby/ext/timeout/Timeout.java:115:in `timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:215:in `retriable\_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:237:in `retriable_getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:120:in `resolve'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:103:in `resolve'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.1.3/lib/logstash/filters/dns.rb:91:in `filter'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/filters/base.rb:151:in `multi\_filter'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/filters/base.rb:148:in `multi\_filter'", "(eval):13314:in `cond_func_747'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):13305:in `cond_func_747'", "(eval):2286:in `filter\_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:267:in `filter_batch'", "org/jruby/RubyArray.java:1613:in `each'", "org/jruby/RubyEnumerable.java:852:in `inject'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:265:in `filter\_batch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:223:in `worker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:201:in `start\_workers'"], :level=\>:error}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 4, 2016, 2:03am UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/3 "2016-05-04T02:03:11Z")

</div>

What version of all the plugins are you using?  
`/opt/logstash/bin/logstash-plugin list --verbose` will show you.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 4, 2016, 2:03am UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/4 "2016-05-04T02:03:28Z")

</div>

Also, it'd be awesome if you could share those grok patterns for others to use 🙂

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 4, 2016, 6:10am UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/5 "2016-05-04T06:10:08Z")

</div>

This is the output of my plugins list.

> /opt/logstash/bin/logstash-plugin list --verbose  
> logstash-codec-collectd (2.0.4)  
> logstash-codec-dots (2.0.4)  
> logstash-codec-edn (2.0.4)  
> logstash-codec-edn\_lines (2.0.4)  
> logstash-codec-es\_bulk (2.0.4)  
> logstash-codec-fluent (2.0.4)  
> logstash-codec-graphite (2.0.4)  
> logstash-codec-json (2.1.3)  
> logstash-codec-json\_lines (2.1.3)  
> logstash-codec-line (2.1.2)  
> logstash-codec-msgpack (2.0.4)  
> logstash-codec-multiline (2.0.11)  
> logstash-codec-netflow (2.0.5)  
> logstash-codec-oldlogstashjson (2.0.4)  
> logstash-codec-plain (2.0.4)  
> logstash-codec-rubydebug (2.0.7)  
> logstash-filter-anonymize (2.0.4)  
> logstash-filter-checksum (2.0.4)  
> logstash-filter-clone (2.0.6)  
> logstash-filter-csv (2.1.3)  
> logstash-filter-date (2.1.6)  
> logstash-filter-dns (2.1.3)  
> logstash-filter-drop (2.0.4)  
> logstash-filter-fingerprint (2.0.5)  
> logstash-filter-geoip (2.0.7)  
> logstash-filter-grok (2.0.5)  
> logstash-filter-json (2.0.6)  
> logstash-filter-kv (2.0.7)  
> logstash-filter-metrics (3.0.2)  
> logstash-filter-multiline (2.0.5)  
> logstash-filter-mutate (2.0.6)  
> logstash-filter-ruby (2.0.5)  
> logstash-filter-sleep (2.0.4)  
> logstash-filter-split (2.0.5)  
> logstash-filter-syslog\_pri (2.0.4)  
> logstash-filter-throttle (2.0.4)  
> logstash-filter-urldecode (2.0.4)  
> logstash-filter-useragent (2.0.8)  
> logstash-filter-uuid (2.0.5)  
> logstash-filter-xml (2.1.3)  
> logstash-input-beats (2.2.7)  
> logstash-input-couchdb\_changes (2.0.4)  
> logstash-input-elasticsearch (2.0.5)  
> logstash-input-eventlog (3.0.3)  
> logstash-input-exec (2.0.6)  
> logstash-input-file (2.2.5)  
> logstash-input-ganglia (2.0.6)  
> logstash-input-gelf (2.0.5)  
> logstash-input-generator (2.0.4)  
> logstash-input-graphite (2.0.7)  
> logstash-input-heartbeat (2.0.4)  
> logstash-input-http (2.2.2)  
> logstash-input-http\_poller (2.0.5)  
> logstash-input-imap (2.0.5)  
> logstash-input-irc (2.0.5)  
> logstash-input-jdbc (3.0.2)  
> logstash-input-kafka (2.0.6)  
> logstash-input-log4j (2.0.7)  
> logstash-input-lumberjack (2.0.7)  
> logstash-input-pipe (2.0.4)  
> logstash-input-rabbitmq (4.0.1)  
> logstash-input-redis (2.0.6)  
> logstash-input-s3 (2.0.6)  
> logstash-input-snmptrap (2.0.4)  
> logstash-input-sqs (2.0.5)  
> logstash-input-stdin (2.0.4)  
> logstash-input-syslog (2.0.5)  
> logstash-input-tcp (3.0.4)  
> logstash-input-twitter (2.2.2)  
> logstash-input-udp (2.0.5)  
> logstash-input-unix (2.0.6)  
> logstash-input-xmpp (2.0.5)  
> logstash-input-zeromq (2.0.4)  
> logstash-output-cloudwatch (2.0.4)  
> logstash-output-csv (2.0.5)  
> logstash-output-elasticsearch (2.5.5)  
> logstash-output-email (3.0.5)  
> logstash-output-exec (2.0.4)  
> logstash-output-file (2.2.5)  
> logstash-output-ganglia (2.0.4)  
> logstash-output-gelf (2.0.5)  
> logstash-output-graphite (2.0.5)  
> logstash-output-hipchat (3.0.4)  
> logstash-output-http (2.1.3)  
> logstash-output-irc (2.0.4)  
> logstash-output-juggernaut (2.0.4)  
> logstash-output-kafka (2.0.3)  
> logstash-output-lumberjack (2.0.6)  
> logstash-output-nagios (2.0.4)  
> logstash-output-nagios\_nsca (2.0.5)  
> logstash-output-null (2.0.4)  
> logstash-output-opentsdb (2.0.4)  
> logstash-output-pagerduty (2.0.4)  
> logstash-output-pipe (2.0.4)  
> logstash-output-rabbitmq (3.0.9)  
> logstash-output-redis (2.0.4)  
> logstash-output-s3 (2.0.7)  
> logstash-output-sns (3.0.4)  
> logstash-output-sqs (2.0.4)  
> logstash-output-statsd (2.0.7)  
> logstash-output-stdout (2.0.6)  
> logstash-output-tcp (2.0.4)  
> logstash-output-udp (2.0.4)  
> logstash-output-xmpp (2.0.4)  
> logstash-output-zeromq (2.0.4)  
> logstash-patterns-core (2.0.5)

This is my windows DNS debug log pattern

> WINDNS %{NUMBER:log\_date} %{TIME:log\_time} %{WORD:dns\_thread\_id} %{WORD:dns\_context}%{SPACE}%{WORD:dns\_packet\_id} %{WORD:dns\_ip\_protocol} %{WORD:dns\_direction} %{IP:dns\_client\_address}%{SPACE}%{WORD:dns\_xid}%{SPACE}(?:Q|R|U) ?(Q|R|U)?%{SPACE}[%{GREEDYDATA:dns\_hex\_flags}%{SPACE}%{WORD:dns\_response}]%{SPACE}%{WORD:dns\_recordtype}%{SPACE}([1-9][0-9]?)%{GREEDYDATA:dns\_query\_name}

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 23, 2016, 1:27pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/6 "2016-05-23T13:27:37Z")

</div>

Any type of insight you can provide here?

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [May 23, 2016, 3:45pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/7 "2016-05-23T15:45:51Z")

</div>

> [@BongoEADGC6](#):
>
> `"exception"=>#, "backtrace"=>["org/jruby/RubyString.java:3936:in []='", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:792:insender'"`

Unfortunately, the actual exception that is thrown in line 3936 of the RubyString.java class is shown as `#`.  
Meaning we don't know what is causing the code in JRuby `resolv.rb:792` to fail in `request[0,2] = [id].pack('n')`

How did you get to the Title "Windows DNS incompatible character encodings"?

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 23, 2016, 9:46pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/8 "2016-05-23T21:46:50Z")

</div>

```
> {:timestamp=>"2016-05-19T15:03:53.103000-0400", :message=>"Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.", "exception"=>#<Encoding::CompatibilityError: incompatible encodings: ASCII-8BIT and UTF-8>, "backtrace"=>["org/jruby/RubyString.java:2630:in `concat'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1381:in `put_string'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1409:in `put_label'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1402:in `put_labels'", "org/jruby/RubyArray.java:1672:in `each_index'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1395:in `put_labels'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1391:in `put_name'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1337:in `encode'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1335:in `encode'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1355:in `initialize'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1321:in `encode'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:791:in `sender'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:508:in `each_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1035:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1034:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1033:in `resolv'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:1031:in `resolv'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:503:in `each_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:502:in `each_resource'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:396:in `each_address'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:120:in `each_address'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:119:in `each_address'", "/opt/logstash/vendor/jruby/lib/ruby/1.9/resolv.rb:97:in `getaddress'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.0.2/lib/logstash/filters/dns.rb:115:in `resolve'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.0.2/lib/logstash/filters/dns.rb:101:in `resolve'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.0.2/lib/logstash/filters/dns.rb:73:in `filter'", "org/jruby/ext/timeout/Timeout.java:115:in `timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-2.0.2/lib/logstash/filters/dns.rb:72:in `filter'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/filters/base.rb:151:in `multi_filter'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/filters/base.rb:148:in `multi_filter'", "(eval):14158:in `cond_func_799'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):14148:in `cond_func_799'", "(eval):2094:in `filter_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:256:in `filter_batch'", "org/jruby/RubyArray.java:1613:in `each'", "org/jruby/RubyEnumerable.java:852:in `inject'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:254:in `filter_batch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:212:in `worker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:190:in `start_workers'"], :level=>:error}

```

This is the error I receive.

It must have commented out the actual error. but it states here:  
Encoding::CompatibilityError: incompatible encodings: ASCII-8BIT and UTF-8

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [May 24, 2016, 9:08am UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/9 "2016-05-24T09:08:02Z")

</div>

Ahhh. This is a totally different error and stack trace from the one before.

This is a problem in the Ruby and JRuby Stdlib Resolv class.

[https://bugs.ruby-lang.org/issues/4270](https://bugs.ruby-lang.org/issues/4270)  
TL;DR

```auto
Resolv does not handle UTF8 domain names.

```

It looks like the Ruby team is not willing to fix this.

The only way we can fix this is to add a punycode converter into the DNS plugin.

I will add an issue to the Github repo.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [May 24, 2016, 10:42am UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/10 "2016-05-24T10:42:04Z")

</div>

> [@BongoEADGC6](#):
>
> dns {  
> resolve =\> ["dest\_ip"]  
> action =\> "replace"  
> nameserver =\> "10.110.24.4"  
> }

@BongoEADGC6 - In your config you are using resolve but your field is `["dest_ip"]`, I presume `dest_ip` actually holds domain name. Is this correct?

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 24, 2016, 12:27pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/11 "2016-05-24T12:27:28Z")

</div>

That's correct. It copies the manipulated dest\_url field to dest\_ip, then resolves and replaces the dest\_ip field with the resolved IP from the url.  
There's probably a prettier way to do that, but it was the way I figured out to do it.

> mutate {  
> **add\_field =\> { "dest\_ip" =\> "%{dest\_url}" }**  
> }  
> dns {  
> resolve =\> ["dest\_ip"]  
> action =\> "replace"  
> nameserver =\> "10.110.24.4"  
> }  
> if [dest\_ip] !~ /[A-Za-z-\_/\]/ {  
> geoip {  
> source =\> "dest\_ip"  
> }  
> } else {  
> grok {  
> remove\_field =\> ["dest\_ip"]  
> }  
> }

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [May 24, 2016, 2:53pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/12 "2016-05-24T14:53:51Z")

</div>

OK thanks. The dns filter chokes when a domain name has upper byte utf8 characters in it - an IDN. e.g. [müller.com](http://xn--mller-kva.com)

I am fixing the dns filter.

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [May 24, 2016, 6:45pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/13 "2016-05-24T18:45:33Z")

</div>

Excellent!

Thank you so much for taking the time to work on this.

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [June 1, 2016, 2:12pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/14 "2016-06-01T14:12:29Z")

</div>

Hey Guy,

Anything else I can provide to assist in this fix?

Thanks.

---

<div class="post-metadata">

**Author:** ![BongoEADGC6](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@BongoEADGC6](https://discuss.elastic.co/u/BongoEADGC6)\
**Post date:** [February 23, 2017, 2:30pm UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/15 "2017-02-23T14:30:16Z")

</div>

Good morning,

I've been receiving this error further.

> [2017-02-23T06:25:32,189][ERROR][logstash.pipeline] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=\>java.lang.IllegalArgumentException: java.text.ParseException: A prohibited code point was found in the input0��^B, "backtrace"=\>["java.net.IDN.toASCIIInternal(java/net/IDN.java:274)", "java.net.IDN.toASCII(java/net/IDN.java:122)", "java.net.IDN.toASCII(java/net/IDN.java:151)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "LogStash::Filters::DNS.getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:273)", "LogStash::Filters::DNS.getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:273)", "LogStash::Filters::DNS.retriable\_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:261)", "LogStash::Filters::DNS.retriable\_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:261)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "LogStash::Filters::DNS.retriable\_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:239)", "LogStash::Filters::DNS.retriable\_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:239)", "org.jruby.ext.timeout.Timeout.timeout(org/jruby/ext/timeout/Timeout.java:115)", "LogStash::Filters::DNS.retriable\_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:238)", "LogStash::Filters::DNS.retriable\_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:238)", "LogStash::Filters::DNS.retriable\_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:260)", "LogStash::Filters::DNS.retriable\_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:260)", "LogStash::Filters::DNS.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:139)", "LogStash::Filters::DNS.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:139)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "LogStash::Filters::DNS.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:122)", "LogStash::Filters::DNS.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:122)", "LogStash::Filters::DNS.filter(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:95)", "LogStash::Filters::DNS.filter(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:95)", "LogStash::Filters::Base.do\_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145)", "LogStash::Filters::Base.do\_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145)", "LogStash::Filters::Base.multi\_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164)", "LogStash::Filters::Base.multi\_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "LogStash::Filters::Base.multi\_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161)", "LogStash::Filters::Base.multi\_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161)", "LogStash::FilterDelegator.multi\_filter(/usr/share/logstash/logstash-core/lib/logstash/filter\_delegator.rb:41)", "LogStash::FilterDelegator.multi\_filter(/usr/share/logstash/logstash-core/lib/logstash/filter\_delegator.rb:41)", "RUBY.initialize((eval):22547)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.initialize((eval):22544)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.initialize((eval):22584)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.initialize((eval):22575)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.filter\_func((eval):3266)", "LogStash::Pipeline.filter\_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:300)", "LogStash::Pipeline.filter\_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:300)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped\_synchronous\_queue.rb:201)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped\_synchronous\_queue.rb:201)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped\_synchronous\_queue.rb:200)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped\_synchronous\_queue.rb:200)", "LogStash::Pipeline.filter\_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:299)", "LogStash::Pipeline.filter\_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:299)", "RUBY.worker\_loop(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:287)", "RUBY.start\_workers(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263)", "java.lang.Thread.run(java/lang/Thread.java:745)"]}

Seems to be related to the same issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:28am UTC](https://discuss.elastic.co/t/windows-dns-incompatible-character-encodings/48961/16 "2017-07-06T04:28:19Z")

</div>