How to use ruby code to realize “i = i + 1" operation

Elastic Stack Logstash
1

Hi Experts,

I' m using ruby code to parse my target log.
Here is the source code:

if "_xmlparsefailure" not in [tags]{
        ruby {
                code => '
                        res = []
                        pmMeasInfo = event.get("[parsed][measData][0][measInfo]")
                        pmMeasInfo.each do |pr|

                                hash = Hash.new
                                i = 0

                                pmmeasValue = pr["measValue"][0]
                                pmmeasType = pr["measType"][0]
                                hash["measObJ"] = pmmeasValue["measObjLdn"]

                                pmmeasType.each do |pt|
                                        key = pt["content"]
                                        value = pmmeasValue["r"][#i]["content"].to_i
                                        hash[key] = value
                                        i = #i + 1
                                end

                                res.push(hash)
                        end

                        event.set("pm",res)
                '
        }
}

but when I start the logstash, here is the error report is fired:

[2019-08-30T16:11:23,024][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SyntaxError) (ruby filter code):18: syntax error, unexpected tIDENTIFIER
                                                    i = #i + 1

How can I realize “i = i + 1" operation in ruby code section?
Thank you.,

2

If I use global variable $i to replace local variable i, it still fired an issue in the below:

[2019-09-02T16:05:32,636][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SyntaxError) (ruby filter code):18: syntax error, unexpected tGVAR
                                                $i = #$i + 1

Who can help me check it, this issue is blocking a long duration for me...

3

As it is Ruby, there are a few ways, however I don't think #i works as you expect. the # character is a single line comment until EOL character.

What the compiler is seeing is

  value = pmmeasValue["r"][
  hash[key] = value
  i =
end

Just use i e.g. value = pmmeasValue["r"][i]["content"].to_i

To increment a variable you can use any of:

i = i + 1
i += 1
i = i.succ
5

Hi expert,

You are right, i have removed the "#" in my code and showed as below:

if "_xmlparsefailure" not in [tags]{
        ruby {
                code => '
                        res = []
                        pmMeasInfo = event.get("[parsed][measData][0][measInfo]")
                        pmMeasInfo.each do |pr|

                                hash = Hash.new
                                i = 0

                                pmmeasValue = pr["measValue"][0]
                                pmmeasType = pr["measType"][0]
                                hash["measObJ"] = pmmeasValue["measObjLdn"]

                                pmmeasType.each do |pt|
                                        key = pt["content"]
                                        value = pmmeasValue["r"][i]["content"].to_i
                                        hash[key] = value
                                        i = i + 1
                                end

                                res.push(hash)
                        end

                        event.set("pm",res)
                '
        }
}

Now my pattern can be executed, but here is another issued logs when I parsed the target logs, would you please help me check it? Thank you.

2019-09-03T09:36:42,960][ERROR][logstash.filters.ruby ] Ruby exception occurred: no implicit conversion of String into Integer

6

It is hard to help, I need more info.

Please you post an example of the value that the field [parsed][measData].

8

Hi Guyboertje,

I have adjust my ruby code, then this issue is resolved, the cause is i have referred the wrong data format that can't adapt the source log. Thank you.

But when I load my logstash script to parse my source log, i have met another performance issue.

`

[2019-09-03T14:21:49,106][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2019.09.03", :_type=>"_doc", :_routing=>nil}, #LogStash::Event:0x1b6c342a], :response=>{"index"=>{"_index"=>"logstash-2019.09.03", "_type"=>"_doc", "_id"=>"4_PJ9WwB6iwMcN-ZT20T", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [logstash-2019.09.03] has been exceeded"}}}}

`
Would you please help me check it? It seems a parameter that limit the max index count. Of course I have found a solution, but I don't know how to use it.

The detailed information you can access this ticket:

Regarding to logstash performance limit Logstash

Hi experts, I have met a logstash performance limit issue, when I parse my log, logstash report this error log: ` [2019-09-03T14:21:49,106][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2019.09.03", :_type=>"_doc", :_routing=>nil}, #LogStash::Event:0x1b6c342a], :response=>{"index"=>{"_index"=>"logstash-2019.09.03", "_type"=>"_doc", "_id"=>"4_PJ9WwB6iwMcN-ZT20T", "status"=>400, "error"=>{"type"=>"…

Thank you again.
[/quote]

9

Please put the latest error in a new post. More people will see it and help or it will help them if they search for similar.