Ruby syntax error only in logstash

Hi, I am trying to use the ruby plugin to parse a field of array type and then generate new fields out of some of elements. However, I encountered the following error. Similar ruby codes(i.e. the statement to set event) works in a regular ruby execution environment; therefore, I wonder if there is some problem escaping some key word or not in the following statement: -

event.set("inboundKafkaFailedMessageCount", matched['inboundFailMessageCount'])

Thanks for any hints in advance.

ruby {
    code => '
      event.get("[Kafka Consumer Web Check][errors]").each { |error|
        if matched = /Inbound Failed Messages (?<inboundFailMessageCount>.*) of (?<threshold>.*) allowed. WARNING/.match(error)
           event.set("inboundKafkaFailedMessageCount", matched['inboundFailMessageCount'])
           event.set("inboundKafkaFailedMessageCountThreshold", matched['threshold'])

[2021-03-04T13:41:42,498][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:logstash_expoacct_webcheck_qa, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"{\", \"}\" at line 54, column 65 (byte 2076) after filter {\n# json {\n#\tsource => \"results\"\n#\n# }\n\n\n mutate {\n add_field => {\n \"[fields][input_name]\" => \"expoacct_webcheck\"\n \"name\" => \"%{[metadata][name]}\"\n }\n add_field => {\n \"[fields][acctDbInfo]\" => \"%{[Database Checks][infos][0]}\" \n \"[fields][transactionLogInfo]\" => \"%{[Database Checks][infos][1]}\" \n \"[fields][spapDbInfo]\" => \"%{[Database Checks][infos][2]}\" \n }\n\n }\n\n\n dissect {\n mapping => { \"[fields][acctDbInfo]\" => \"%{} : %{acctDbUrl} (DB2/%{}) : %{acctDbStatus}\" }\n mapping => { \"[fields][transactionLogInfo]\" => \"Transaction Log Percent @ %{transactionLogPercent}%\"}\n mapping => { \"[fields][spapDbInfo]\" => \"%{} : %{spapDbUrl} (DB2/%{}) : %{spapDbStatus}\"}\n mapping => { \"[Kafka Consumer Web Check][errors][0]\" => \"Inbound Failed Messages %{inboundKafkaConsumerFailedMessageCount} of %{}\" }\n }\n\n ruby {\n code => '\n event.get(\"[Kafka Consumer Web Check][errors]\").each { |error|\n if matched = /Inbound Failed Messages (?<inboundFailMessageCount>.*) of (?<threshold>.*) allowed. WARNING/.match(error)\n event.set(\"inboundKafkaFailedMessageCount\", matched['", :backtrace=>["/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/compiler.rb:58:in compile_imperative'", "/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/compiler.rb:66:in compile_graph'", "/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/compiler.rb:28:in block in compile_sources'", "org/jruby/ map'", "/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/compiler.rb:27:in compile_sources'", "org/logstash/execution/ initialize'", "org/logstash/execution/ initialize'", "/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/java_pipeline.rb:44:in initialize'", "/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/pipeline_action/create.rb:52:in execute'", "/prod/elastic/logstash-7.8.1/logstash-core/lib/logstash/agent.rb:356:in block in converge_state'"]}

The following codes works at this website -

errors = Array.(

  "Inbound Failed Messages 129569 of 0 allowed. WARNING",
  "Outbound Failed Messages 2323 of 0 allowed. WARNING"

errors.each {|error|

if matched = /Inbound Failed Messages (?<inboundFailMessageCount>.*) of (?<threshold>.*) allowed. WARNING/
   p matched['inboundFailMessageCount']
   p matched['threshold']


You are using single quotes for the code option of the ruby filter. I would use double quotes for the two references to matched, or you can probably escape the single quotes with backslashes.

Thanks. It turned out that the main culprit was the double quotes around it, plus the single quote.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.