Illegal character in authority at index 8:

elaair · August 4, 2020, 11:29pm

I am using logstash 7.8 and am seeing this error while using the elasticsearch filter:

:error=>"Illegal character in authority at index 8: https://{:host=>\"fqdn.here.com\", :scheme=>\"https\", :protocol=>\"https\", :port=>9200}:9200/data-activedirectory-2020
snip

what does this mean?
The config part

elasticsearch {
    hosts => ['FQDN.com']
    ssl => true
    ca_file => "/opt/logstash/certs/FILE.crt"
    user => "username"
    password => "pASS"
    index => "data-activedirectory-2020"
    query =>  "sAMAccountName:%{[user]}"
    fields => { "title" => "title" "department" => "department" "division" => "division" }
    }

Badger · August 4, 2020, 11:43pm

Whilst building the URI to use in the query it has used

{:host=>\"fqdn.here.com\", :scheme=>\"https\", :protocol=>\"https\", :port=>9200}

as the hostname, resulting in an invalid character (the open brace "{") after the first 8 characters of the URI (https://).

That suggests that the value of the hosts option is not what you think it is.

elaair · August 5, 2020, 12:17am

I am not seeing any issues or missing braces anywhere. I will restart in debug after tonights file processing completes.

elaair · August 5, 2020, 1:21am

odd with the repeated hosts

[2020-08-05T01:05:48,375][INFO ][logstash.filters.elasticsearch] New ElasticSearch filter client {:hosts=>[{:host=>{:host=>"usbbcssa0001.corp.inbaxalta.com", :scheme=>"https", :protocol=>"https", :port=>9200}, :scheme=>"https"}]}

Jenni · August 5, 2020, 1:31am

I think you've run into an issue that seems to have been open for a while…

github.com/logstash-plugins/logstash-filter-elasticsearch

Illegal character in authority at index 8

opened 09:14AM - 16 Apr 20 UTC

closed 01:54PM - 12 Aug 21 UTC

tom-x1

I' trying to use this filter plugin. When i run the it with logstash 7.6.2 and n…ewest filter version 3.7.1, I receive the following error: `[WARN ] 2020-04-16 09:05:18.073 [[main]>worker1] elasticsearch - Failed to query elasticsearch for previous event {:index=>"proxy-blacklist", :error=>"Illegal character in authority at index 8: https://{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>{:host=>\"elastic01.example.org\", :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\"}, :scheme=>\"https\"}, :scheme=>\"https\"}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\"}, :scheme=>\"https\"}, :scheme=>\"https\"}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\"}, :scheme=>\"https\"}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\", :protocol=>\"https\", :port=>9200}, :scheme=>\"https\"}, :scheme=>\"https\"}:9200/proxy-blacklist/_search?q=domain.keyword%3A%2F%28www.%29%3Fchat.example.org%2F&size=1&sort=%40timestamp%3Adesc"}` So it comes down to: `Illegal character in authority at index 8` I've found many users which seem to have this problem but no solution at all. Some solve this by not using "ssl => true" and using the prefix "https://" in the hosts section instead. This does not solve the error for me. My filter config looks like this: > filter { > grok { > patterns_dir => [ "/etc/logstash/patterns/patterns" ] > match => [ "message", "%{SQUIDACCESS}" ] > } > elasticsearch { > hosts => [ "elastic.example.org" ] > user => "elastic" > password => "changeme" > ssl => true > ca_file => "/etc/logstash/ca.crt" > index => "blacklist" > query => "domain.keyword:/(www.)?%{zone}/" > fields => { "domid" => "domid" } > fields => { "source" => "source" } > } > }

elaair · August 5, 2020, 2:48am

I did run across that and indeed one of my config from something I was testing in Febuary has the ssl => true commented out and the https in the hosts section. I started with that type of config but when I use it logstash will not start.

Jenni · August 5, 2020, 2:52am

What error message do you get?

elaair · August 5, 2020, 3:00am

[2020-08-05T02:59:35,888][ERROR][logstash.javapipeline    ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Manticore::SocketException: Connection refused (Connection refused)>, :backtrace=>["/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in `call'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:274:in `call_once'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:158:in `code'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:84:in `block in perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/ping.rb:20:in `ping'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.7.1/lib/logstash/filters/elasticsearch.rb:270:in `test_connection!'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.7.1/lib/logstash/filters/elasticsearch.rb:92:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:216:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:215:in `register_plugins'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:520:in `maybe_setup_out_plugins'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:228:in `start_workers'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:170:in `run'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:125:in `block in start'"], "pipeline.sources"=>["/opt/logstash/MetricsResilientandVaronis/conf.d/resilient.conf"], :thread=>"#<Thread:0x5f30b9c run>"}
[2020-08-05T02:59:35,908][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2020-08-05T02:59:36,099][INFO ][logstash.runner          ] Logstash shut down.

elaair · August 5, 2020, 3:00am

I missed one line.

[2020-08-05T02:59:35,786][INFO ][logstash.filters.elasticsearch] New ElasticSearch filter client {:hosts=>["https://usbbcssa0001.corp.inbaxalta.com"]}
[2020-08-05T02:59:35,888][ERROR][logstash.javapipeline    ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Manticore::SocketException: Connection refused (Connection refused)>, :backtrace=>["/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in `call'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:274:in `call_once'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:158:in `code'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:84:in `block in perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/ping.rb:20:in `ping'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.7.1/lib/logstash/filters/elasticsearch.rb:270:in `test_connection!'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-elasticsearch-3.7.1/lib/logstash/filters/elasticsearch.rb:92:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:216:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:215:in `register_plugins'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:520:in `maybe_setup_out_plugins'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:228:in `start_workers'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:170:in `run'", "/opt/logstash/MetricsResilientandVaronis/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:125:in `block in start'"], "pipeline.sources"=>["/opt/logstash/MetricsResilientandVaronis/conf.d/resilient.conf"], :thread=>"#<Thread:0x5f30b9c run>"}
[2020-08-05T02:59:35,908][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2020-08-05T02:59:36,099][INFO ][logstash.runner          ] Logstash shut down.

Jenni · August 5, 2020, 7:24am

But you have made sure that you can connect to ES from this server (e.g. with Curl), that it's definitely only Logstash that has the connection issues?

(You can just edit your response if you have to correct something ;))

elaair · August 5, 2020, 12:58pm

Yes, The same config that I am doing the lookup from is also writing the event to ES. I have also tested with CURL.

Jenni · August 5, 2020, 1:38pm

Hm. Unfortunately no other workaround comes to my mind. Hopefully someone else knows more.

elaair · August 5, 2020, 6:37pm

at some point I had removed the :9200 from the hosts line. When I put that back logstash starts and actually runs the lookup.

system · September 2, 2020, 6:37pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.