UTF-8 issue when patterns are loading


(Wayne Hunter) #1

The pattern files which are read have not been changed in some time. Here is the end of the debug output:

{:timestamp=>"2015-07-09T18:52:30.171000+0100", :message=>"Adding pattern", "}"=>"", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
{:timestamp=>"2015-07-09T18:52:30.171000+0100", :message=>"Adding pattern", "}"=>"", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
{:timestamp=>"2015-07-09T18:52:30.172000+0100", :message=>"Adding pattern", "HAPROXYCAPTUREDREQUESTHEADERS"=>"%{DATA:request_header_host}\|%{DATA:request_header_x_forwarded_for}\|%{DATA:request_header_accept_language}\|%{DATA:request_header_referer}\|%{DATA:request_header_user_agent}", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
{:timestamp=>"2015-07-09T18:52:30.173000+0100", :message=>"Adding pattern", "HAPROXYCAPTUREDRESPONSEHEADERS"=>"%{DATA:response_header_content_type}\|%{DATA:response_header_content_encoding}\|%{DATA:response_header_cache_control}\|%{DATA:response_header_last_modified}", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
{:timestamp=>"2015-07-09T18:52:30.173000+0100", :message=>"Adding pattern", "HAPROXYHTTP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \[%{HAPROXYDATE:accept_date}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} (\{%{HAPROXYCAPTUREDREQUESTHEADERS}\})?( )?(\{%{HAPROXYCAPTUREDRESPONSEHEADERS}\})?( )?"(|(%{WORD:http_verb} (%{URIPROTO:http_proto}://)?(?:%{USER:http_user}(?::[^@])?@)?(?:%{URIHOST:http_host})?(?:%{URIPATHPARAM:http_request})?( HTTP/%{NUMBER:http_version})?))?"", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
{:timestamp=>"2015-07-09T18:52:30.174000+0100", :message=>"Adding pattern", "HAPROXYTCP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \[%{HAPROXYDATE:accept_date}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_queue}/%{INT:time_backend_connect}/%{NOTSPACE:time_duration} %{NOTSPACE:bytes_read} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue}", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
{:timestamp=>"2015-07-09T18:52:30.175000+0100", :message=>"Adding pattern", "JAVAEXCEPTION"=>"(^.+Exception: .+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s
Caused by:.+)", :level=>:info, :file=>"/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=>"62", :method=>"add_pattern"}
The error reported is:
invalid byte sequence in UTF-8
org/jruby/RubyRegexp.java:1657:in =~' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb:72:inadd_patterns_from_file'
org/jruby/RubyIO.java:3542:in each' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb:70:inadd_patterns_from_file'
C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb:361:in add_patterns_from_files' org/jruby/RubyArray.java:1613:ineach'
C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb:357:in add_patterns_from_files' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb:269:inregister'
org/jruby/RubyArray.java:1613:in each' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb:265:inregister'
org/jruby/RubyHash.java:1341:in each' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb:261:inregister'
org/jruby/RubyArray.java:1613:in each' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/pipeline.rb:154:instart_filters'
C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/pipeline.rb:81:in run' C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/agent.rb:150:inexecute'
C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/runner.rb:91:in run' org/jruby/RubyProc.java:271:incall'
C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/runner.rb:96:in run' org/jruby/RubyProc.java:271:incall'
C:/Programs/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/task.rb:12:in `initialize'


(Wayne Hunter) #2

No action required, another file was in the patterns directory causing this issue.


(system) #3