Logstash ODX input File - unconventional input file

Hi - I was hoping someone could help me regarding input in logstash. I am very new to logstash and I have a requirement where my logstash input file is an unconventional file.

How do I input an ODX input file? I think delimiter for the input file is "#" as we'd like to treat those with "#" as Columns/Fields.

Sample data:

#Customer Market
#Date of Export
Mon Jan 20 08:41:12 2020
#Current Device
#----- Start Task -----

#Customer ID
#Cashier Counter
#Counter Name

Thanks in advance

The approach I would take would be to use a multiline codec on the input to read the file as a single event (as shown at the end of this post). Then use a ruby scan function

    ruby {
        code => '
            s = event.get("message")
            m = s.scan(/#([^\n]*)\n([^\n]*)?(\n|$)/)
            m.each { |x|
                event.set(x[0], x[1])

That does not quite work because

#----- Start Task -----


results in '#----- Start Task -----\n#Location\n' rather than '#----- Start Task -----\n\n#Location\n' and thus you lose the Location field. However, it does process most of the fields correctly.

       "Cashier Counter" => "1",
        "Current Device" => "DigiMarkXP",
           "Taskcounter" => "1",
           "Customer ID" => "0118",
       "Customer Market" => "250",

etc. It is probably possible to gsub in the missing \n and recover the Location field.

Obviously this does not retain any of the task definition or other orchestration information.

this works. Thanks for the help Badger.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.