How to run script query on logstash

It doesn't work(logstash 7.15.1). I can't escape single quotes in query field.

  • Elasticsearch input value :
input {
  elasticsearch {
    hosts => ["..."]
    index => "..."
    query => '{
"query": {
  "bool": {
    "must": [
      {
        "script": {
          "script": "doc[\'uri.keyword\'].size() > 0"
        }
      }
    ]
  }
}
    }'
  }
}
  • error logs :
[2022-01-13T11:11:03,421][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::Json::ParserError:              Unrecognized character escape ''' (code 39)
  at [Source: (byte[])"{
 "query": {
   "bool": {
     "must": [
       {
         "script": {
           "script": "doc[\'uri.keyword\'].size() > 0"
         }
       }
     ]
   }                                                                                                                                                            
 }
     }"; line: 7, column: 28]>, :backtrace=>["/app/logstash-7.15.1/logstash-core/lib/logstash/json.rb:32:in `jruby_load'", "/app/logstash-7.15.1/vendor/bundle/  jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:195:in `register'", "/app/logstash-7.15.1/logstash-core/lib/logstash/  java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1820:in `each'", "/app/logstash-7.15.1/logstash-core/lib/logstash/              java_pipeline.rb:227:in `register_plugins'", "/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:386:in `start_inputs'", "/app/logstash-7.15.1/   logstash-core/lib/logstash/java_pipeline.rb:311:in `start_workers'", "/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/app/    logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/get_uri.conf"], :             thread=>"#<Thread:0x6767b852 run>"}
 [2022-01-13T11:11:03,423][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}
 [2022-01-13T11:11:03,435][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :       message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
 [2022-01-13T11:11:03,490][INFO ][logstash.runner          ] Logstash shut down.
 [2022-01-13T11:11:03,497][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
 org.jruby.exceptions.SystemExit: (SystemExit) exit
     at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.19.0.jar:?]
     at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.19.0.jar:?]
     at home.svcapp.app.logstash.lib.bootstrap.environment.<main>(/app/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

If I use double escapes,

"script": "doc[\\'uri.keyword\\'].size() > 0"

it still makes error.

[2022-01-13T11:34:38,086][ERROR][logstash.javapipeline    ][main][ea54dbd9353a77451b885058528a641895b4c4e89d97c8bc3a34d2a03b7d7517] A plugin had an             unrecoverable error. Will restart this plugin.
   Pipeline_id:main
   Plugin: <LogStash::Inputs::Elasticsearch index=>"...", id=>"ea54dbd9353a77451b885058528a641895b4c4e89d97c8bc3a34d2a03b7d7517", hosts=>["..."],                query=>"{\n\"query\": {\n  \"bool\": {\n    \"must\": [\n      {\n        \"script\": {\n          \"script\": \"doc[\\\\'uri.keyword\\\\'].size() >            0\"\n        }\n      }\n    ]\n  }\n}\n    }", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_bacb8d4b-47f2-4887-95c6-60666133b840",            enable_metric=>true, charset=>"UTF-8">, size=>1000, scroll=>"1m", docinfo=>false, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"],      connect_timeout_seconds=>10, request_timeout_seconds=>60, socket_timeout_seconds=>60, ssl=>false>
   Error: [400] {"error":{"root_cause":[{"type":"script_exception","reason":"compile error","script_stack":["doc[\\'uri.keyword\\'].size() > ...","    ^----     HERE"],"script":"doc[\\'uri.keyword\\'].size() > 0","lang":"painless","position":{"offset":4,"start":0,"end":29}}],"type":"search_phase_execution_exception",   "reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"...","node":"_EpveBLuSb-9O0oQxDSdTg","reason":{"type":         "query_shard_exception","reason":"failed to create query: compile error","index_uuid":"b5ITYe96SB2mj35dvN3r-Q","index":"...","caused_by":{"type":               "script_exception","reason":"compile error","script_stack":["doc[\\'uri.keyword\\'].size() > ...","    ^---- HERE"],"script":"doc[\\'uri.keyword\\'].size() >   0","lang":"painless","position":{"offset":4,"start":0,"end":29},"caused_by":{"type":"illegal_argument_exception","reason":"unexpected character [\\].",         "caused_by":{"type":"lexer_no_viable_alt_exception","reason":"lexer_no_viable_alt_exception: null"}}}}}],"caused_by":{"type":"script_exception","reason":       "compile error","script_stack":["doc[\\'uri.keyword\\'].size() > ...","    ^---- HERE"],"script":"doc[\\'uri.keyword\\'].size() > 0","lang":"painless",         "position":{"offset":4,"start":0,"end":29},"caused_by":{"type":"illegal_argument_exception","reason":"unexpected character [\\].","caused_by":{"type":          "lexer_no_viable_alt_exception","reason":"lexer_no_viable_alt_exception: null"}}}},"status":400}                                                               
   Exception: Elasticsearch::Transport::Transport::Errors::BadRequest
   Stack: g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/transport/base.rb:219:in               `__raise_transport_error'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/transport/base.rb:342:in `perform_request'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/transport/http/manticore.rb:85:in               `perform_request'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-7.16.0/lib/elasticsearch/transport/client.rb:197:in `perform_request'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-7.16.0/lib/elasticsearch.rb:41:in `method_missing'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-7.16.0/lib/elasticsearch/api/actions/search.rb:104:in `search'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:348:in `search_request'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:271:in `do_run_slice'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:250:in `do_run'
 g/app/logstash-7.15.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.3/lib/logstash/inputs/elasticsearch.rb:238:in `run'
 g/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'
 g/app/logstash-7.15.1/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'

How to escape single quotes in script in input of logstash config?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.