Output writing wrong event data

Elastic Stack Logstash
1

Heyy! I'm newbie with Logstash so I need some help here with a filter done in .rb file. Situation:

  1. I'm reading from csv file a code, for example 1500.

  2. Then, in filter I use a ruby file to look for that 1500 as a key inside a json file. With read data, I set two events that should be written in the output.

  3. Most of output lines are right, but sometimes key and read data got mixed between some lines. Looking the log, I saw the problem used to be that one output row has the .json data from the next one. I logged the .rb file searches and all key-value are correctly read from .json file, so the problem should be with the speed it writes the output.

I tried to return read data with return from .rb file, but I couldn't. So I thought maybe I can make the output wait for the event overwritten or something similar, but I prefer to ask for the better practice as I'm learning.

Thank you and regards!

2

Are you familiar with the translate filter?

What does the code look like? Could you be getting multi-threading issues? Do they go away of you set '--pipeline.workers 1'?

3

Thanks for answering!

I just solved it. Problem was saving event.get("key") in a @variable. I directly use now event.get and it works perfectly.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.