# How to add field to every element from an array with logstash ruby **URL:** \ **Category:** Logstash\ **Created:** [March 4, 2021, 3:26pm UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257 "2021-03-04T15:26:37Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![syrine\_chelly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syrine_chelly/32/84013_2.png) [@syrine\_chelly](https://discuss.elastic.co/u/syrine_chelly)\ **Post date:** [March 4, 2021, 3:26pm UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257/1 "2021-03-04T15:26:38Z")
Hi I have an array such like this: ```auto "step": [ "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIC A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE CHGFI1 A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE RCOU A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:42:35 : POUR LA JOURNEE DU 20200731 L'ETAPE RCAP A PRIS 00H12", "> MESSAGE AUTOMATE 03/08/2020 18:42:35 : POUR LA JOURNEE DU 20200731 L'ETAPE SDEC A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:50:51 : POUR LA JOURNEE DU 20200731 L'ETAPE TRI A PRIS 00H08", "> MESSAGE AUTOMATE 03/08/2020 18:50:51 : POUR LA JOURNEE DU 20200731 L'ETAPE RADC A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:50:51 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFI A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:50:51 : POUR LA JOURNEE DU 20200731 L'ETAPE RSFI A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:50:51 : POUR LA JOURNEE DU 20200731 L'ETAPE CHGFI3 A PRIS 00H00", "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE RCIM A PRIS 00H01", "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" ] ``` my requirement is add field to every element from this array because i want to apply a grok filter to every line this my logstash.cong : ```auto filter { kv { source => "message" field_split => "\n" } ruby { code => "etape =[] taches = event.get('/soft/rilprdaglm/scripts/wfc_automate_genere_G_WFL10.sos ') taches.each { | item | if item.include? ('ETAPE') etape.push(item) end event.set('step',etape) } " } ``` how can i add a field to every item ? please help me because i have no solution, i'm actually blocked ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 4, 2021, 5:55pm UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257/2 "2021-03-04T17:55:35Z")
It is really unclear what you want to do, so let me make a couple of general points. Firstly, you can tell grok to match against an array. If we start off with a file containing ``` { "step": [ "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00", "random text", "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" ] } ``` Then we can consume the file and parse it ``` input { file { path => "/home/user/test.txt" sincedb_path => "/dev/null" start_position => beginning codec => multiline { pattern => "^Spalanzani" negate => true what => previous auto_flush_interval => 1 multiline_tag => "" } } } filter { json { source => "message" remove_field => ["message"] } } ``` This gets us an event that contains this field ``` "step" => [ [0] "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00", [1] "random text", [2] "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" ], ``` a grok against that ``` grok { pattern_definitions => { "TIMESTAMP" => "%{MONTHNUM}/%{MONTHDAY}/%{YEAR} %{TIME}" } match => { "step" => "%{TIMESTAMP:timestamp}" } } ``` produces this ``` "timestamp" => [ [0] "03/08/2020 18:39:01", [1] "03/08/2020 18:51:02" ] ``` Secondly, a more compact to remove elements from an array based on a regexp is ``` code => ' step = event.get("step") if step.is_a? Array event.set("step", step.delete_if { |x| x !~ /ETAPE/ }) end ' ``` which will change the [step] field to ``` "step" => [ [0] "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00", [1] "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" ], ``` It is possible you want to use a [split filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html) to create a new event for each entry in the array. I am unclear where '/soft/rilprdaglm/scripts/wfc\_automate\_genere\_G\_WFL10.sos ' comes from. Do you really have a field called that? ---
**Author:** ![syrine\_chelly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syrine_chelly/32/84013_2.png) [@syrine\_chelly](https://discuss.elastic.co/u/syrine_chelly)\ **Post date:** [March 4, 2021, 6:37pm UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257/3 "2021-03-04T18:37:22Z")
Thank you for your answer ! i see that you didn't understand my requirement i already have step field which contains a table of messages.I want to parse the table and add a field to every line of the table for example : "step 1 " : "MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00" you mentionned that i can use a [split filter] to create a new event for each entry in the array. how can i do it please i wish that i make it clear to you hope that you can help me ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 4, 2021, 7:13pm UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257/4 "2021-03-04T19:13:17Z")
OK, still not clear on the requirement, but it sounds like you want to convert each member of the array into a hash. I have not tested this code, but if you start with ``` "step" => [ [0] "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00", [1] "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" ], ``` then the following ruby filter will do that ``` ruby { code => ' step = event.get("step") if step.is_a? Array newStep = [] step.each_index { |x| newStep << { "step#{x}" => step[x] } } event.set("step", newStep) end ' } ``` that would result in ``` "step" => [ [0] { "step0" => "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00" }, [1] { "step1" => "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" } ], ``` With a split filter you could use ``` split { field => "step" } ``` which would give you two events, one containing ``` "step" => "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00" ``` and the other containing ``` "step" => "> MESSAGE AUTOMATE 03/08/2020 18:51:02 : POUR LA JOURNEE DU 20200731 L'ETAPE AFFC A PRIS 00H00" ``` If you try to combine both you would get the first event containing ``` "step" => { "step0" => "> MESSAGE AUTOMATE 03/08/2020 18:39:01 : POUR LA JOURNEE DU 20200731 L'ETAPE UNIP A PRIS 00H00" } } ``` But in that case in some document you would have to reference [step][step0], and in another [step][step1]. A different field name for every event from the same initial event. I cannot imagine a use-case where that would be desireable. ---
**Author:** ![syrine\_chelly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syrine_chelly/32/84013_2.png) [@syrine\_chelly](https://discuss.elastic.co/u/syrine_chelly)\ **Post date:** [March 5, 2021, 9:33am UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257/5 "2021-03-05T09:33:35Z")
Thank you a lot it helps me to solve my problem ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 2, 2021, 9:33am UTC](https://discuss.elastic.co/t/how-to-add-field-to-every-element-from-an-array-with-logstash-ruby/266257/6 "2021-04-02T09:33:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.