It doesn't work(logstash 7.15.1). I can't escape single quotes in query field.
- Elasticsearch input value :
input {
elasticsearch {
hosts => ["..."]
index => "..."
query => '{
"query": {
"bool": {
"must": [
{
"script": {
"script": "doc[\'uri.keyword\'].size() > 0"
}
}
]
}
}
}'
}
}
- error logs :
[2022-01-13T11:11:03,421][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::Json::ParserError: Unrecognized character escape ''' (code 39)
at [Source: (byte[])"{
"query": {
"bool": {
"must": [
{
"script": {
"script": "doc[\'uri.keyword\'].size() > 0"
}
}
]
}
}
}"; line: 7, column: 28]>, :backtrace=>["/app/logstash-7.15.1/logstash-core/lib/logstash/json.rb:32:in `jruby_load'", "/app/logstash-7.15.1/vendor/bundle/ jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:195:in `register'", "/app/logstash-7.15.1/logstash-core/lib/logstash/ java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1820:in `each'", "/app/logstash-7.15.1/logstash-core/lib/logstash/ java_pipeline.rb:227:in `register_plugins'", "/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:386:in `start_inputs'", "/app/logstash-7.15.1/ logstash-core/lib/logstash/java_pipeline.rb:311:in `start_workers'", "/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/app/ logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/get_uri.conf"], : thread=>"#<Thread:0x6767b852 run>"}
[2022-01-13T11:11:03,423][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2022-01-13T11:11:03,435][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, : message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2022-01-13T11:11:03,490][INFO ][logstash.runner ] Logstash shut down.
[2022-01-13T11:11:03,497][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.19.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.19.0.jar:?]
at home.svcapp.app.logstash.lib.bootstrap.environment.<main>(/app/logstash/lib/bootstrap/environment.rb:94) ~[?:?]
If I use double escapes,
"script": "doc[\\'uri.keyword\\'].size() > 0"
it still makes error.
[2022-01-13T11:34:38,086][ERROR][logstash.javapipeline ][main][ea54dbd9353a77451b885058528a641895b4c4e89d97c8bc3a34d2a03b7d7517] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Elasticsearch index=>"...", id=>"ea54dbd9353a77451b885058528a641895b4c4e89d97c8bc3a34d2a03b7d7517", hosts=>["..."], query=>"{\n\"query\": {\n \"bool\": {\n \"must\": [\n {\n \"script\": {\n \"script\": \"doc[\\\\'uri.keyword\\\\'].size() > 0\"\n }\n }\n ]\n }\n}\n }", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_bacb8d4b-47f2-4887-95c6-60666133b840", enable_metric=>true, charset=>"UTF-8">, size=>1000, scroll=>"1m", docinfo=>false, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"], connect_timeout_seconds=>10, request_timeout_seconds=>60, socket_timeout_seconds=>60, ssl=>false>
Error: [400] {"error":{"root_cause":[{"type":"script_exception","reason":"compile error","script_stack":["doc[\\'uri.keyword\\'].size() > ..."," ^---- HERE"],"script":"doc[\\'uri.keyword\\'].size() > 0","lang":"painless","position":{"offset":4,"start":0,"end":29}}],"type":"search_phase_execution_exception", "reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"...","node":"_EpveBLuSb-9O0oQxDSdTg","reason":{"type": "query_shard_exception","reason":"failed to create query: compile error","index_uuid":"b5ITYe96SB2mj35dvN3r-Q","index":"...","caused_by":{"type": "script_exception","reason":"compile error","script_stack":["doc[\\'uri.keyword\\'].size() > ..."," ^---- HERE"],"script":"doc[\\'uri.keyword\\'].size() > 0","lang":"painless","position":{"offset":4,"start":0,"end":29},"caused_by":{"type":"illegal_argument_exception","reason":"unexpected character [\\].", "caused_by":{"type":"lexer_no_viable_alt_exception","reason":"lexer_no_viable_alt_exception: null"}}}}}],"caused_by":{"type":"script_exception","reason": "compile error","script_stack":["doc[\\'uri.keyword\\'].size() > ..."," ^---- HERE"],"script":"doc[\\'uri.keyword\\'].size() > 0","lang":"painless", "position":{"offset":4,"start":0,"end":29},"caused_by":{"type":"illegal_argument_exception","reason":"unexpected character [\\].","caused_by":{"type": "lexer_no_viable_alt_exception","reason":"lexer_no_viable_alt_exception: null"}}}},"status":400}
Exception: Elasticsearch::Transport::Transport::Errors::BadRequest
Stack: g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/transport/base.rb:219:in `__raise_transport_error'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/transport/base.rb:342:in `perform_request'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/transport/http/manticore.rb:85:in `perform_request'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/client.rb:197:in `perform_request'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-7.16.0/lib/elasticsearch.rb:41:in `method_missing'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-7.16.0/lib/elasticsearch/api/actions/search.rb:104:in `search'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:348:in `search_request'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:271:in `do_run_slice'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:250:in `do_run'
g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:238:in `run'
g/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'
g/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'
How to escape single quotes in script in input of logstash config?