Logstash ODX input File - unconventional input file

[newbie001]

Hi - I was hoping someone could help me regarding input in logstash. I am very new to logstash and I have a requirement where my logstash input file is an unconventional file.

How do I input an ODX input file? I think delimiter for the input file is "#" as we'd like to treat those with "#" as Columns/Fields.

Sample data:

#Customer Market
250
#Date of Export
Mon Jan 20 08:41:12 2020
#Current Device
DigiMarkXP
#Taskcounter
1
#----- Start Task -----

#Location
FoodMarket_001\Grocery_XXX\118_Cust
#Customer ID
0118
#Cashier Counter
1
#Counter Name
DigiMarkXP
#Items
20

Thanks in advance

[Badger]

The approach I would take would be to use a multiline codec on the input to read the file as a single event (as shown at the end of this post). Then use a ruby scan function

    ruby {
        code => '
            s = event.get("message")
            m = s.scan(/#([^\n]*)\n([^\n]*)?(\n|$)/)
            m.each { |x|
                event.set(x[0], x[1])
            }
        '
    }

That does not quite work because

#----- Start Task -----

#Location

results in '#----- Start Task -----\n#Location\n' rather than '#----- Start Task -----\n\n#Location\n' and thus you lose the Location field. However, it does process most of the fields correctly.

       "Cashier Counter" => "1",
        "Current Device" => "DigiMarkXP",
           "Taskcounter" => "1",
           "Customer ID" => "0118",
       "Customer Market" => "250",

etc. It is probably possible to gsub in the missing \n and recover the Location field.

Obviously this does not retain any of the task definition or other orchestration information.

[newbie001]

this works. Thanks for the help Badger.