Dynamic Nested fields

Elastic Stack Logstash
1

Hi guys,
I’m looking to make some Elasticsearch documents in a very similar format to what is shown here: Nested Objects | Elasticsearch: The Definitive Guide [2.x] | Elastic

I started off inputting a log into Logstash which was 1 large string and have broken it down using regex within a ruby code filter. I now have the static fields and the comment/note fields. Then I split down all the note fields into the various nested fields I will need and store them in an array.

After a lot of trial and error I sort of get what I want using the following at the end of the code:

                     	for field in fieldsArray
                                     if fieldsArray[1].start_with?('NOTE', 'PHONE', 'RESEARCH')
                                             event['[Notes[' + i.to_s + ']][' + FieldsTitle[j] + ']'] = field
                                    elsif  fieldsArray[1].start_with?('Name', 'NAME', 'STATUS')
                                            event['[Notes[' + i.to_s + ']][' + FieldsTitle2[j] + ']'] = field 
                                    else
                                            event['[Notes['  + i.to_s + ']][' + FieldsTitle3[j] + ']'] = field
                                    end
                                    j += 1
  end

FieldArray holds all the fields for a single Note. FieldsTitle is an array of names of the fields nested within each note. ‘J’ is the number of fields within a Note and ‘I’ is the number of notes. Different types of notes have slightly different fields hence the various conditions here. The number of notes will differ in each event.

The (shortened) output of this is:

…
"Notes" => {
         "1" => {
                 "Name" => "foo",
                 "Type" => "NOTES ",
                  …
	},
         "2" => {
                 "Name" => "bar",
                 "Type" => "STATUS ",
	         …
	}

While it has divided the notes and nested the fields as intended this obviously causes a spam of fields in Elasticsearch (Notes.1.Name,Notes.2.Name etc). As the above page showed what I actually want is (Notes.Name):

…
"Notes" => {
         {
                 "Name" => "foo",
                 "Type" => "NOTES ",
	},
         {
                 "Name" => "bar",
                 "Type" => "STATUS ",
	}

If I remove the incrementing number then only the last Note is outputted (I assume it just overwrites each time on the loop?). Any suggestions how I fix this?
Thanks!