# Import CSV file into nested fields

**URL:** <https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278>\
**Category:** Logstash\
**Created:** [January 16, 2023, 8:36pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278 "2023-01-16T20:36:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![markedperf](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@markedperf](https://discuss.elastic.co/u/markedperf)\
**Post date:** [January 16, 2023, 8:36pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278/1 "2023-01-16T20:36:48Z")

</div>

I'm trying to import a CSV file into nested fields. Trying to follow the aggregate example #4, is coming up short for me. This partially works, but not really what I'm looking for. I'd just like the values grouped in some way, either by array index or ?. I didn't see a nested forum post, so creating a new one. Here is where I'm at so far. Just simply trying to group these by Id and then a nested structure / array for the rest.

CSV file:  
1,123Name,2.03  
1,456Name,2.03  
2,123Name,2.03  
2,789Name,2.03

```auto
filter {
  csv {
    autogenerate_column_names => false
    skip_header => true
    columns => ["Id","IdName","IdVersion"]
  }

  aggregate {
    task_id => "%{Id}"
    code => "
      map['Id'] ||= event.get('Id')
      map['ListofVals'] ||= []
      if !( map['ListofVals'].include? event.get('IdName') )
        map['ListofVals'] << {'IdName' => event.get('IdName')}
        map['ListofVals'] << {'IdVersion' => event.get('IdVersion')}
      end
      event.cancel()
    "
    push_previous_map_as_event => true
    timeout => 5
  }

```

```auto
          "Id" : "1",
          "ListofVals" : [
            {
              "IdName" : "123Name"
            },
            {
              "IdVersion" : "2.03"
            },
            {
              "IdName" : "456Name"
            },
            {
              "IdVersion" : "2.03"
            },
          ]
		  ....
          "Id" : "2",
          "ListofVals" : [
            {
              "IdName" : "123Name"
            },
            {
              "IdVersion" : "2.03"
            },
            {
              "IdName" : "789Name"
            },
            {
              "IdVersion" : "2.03"
            },
          ]

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 16, 2023, 8:42pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278/2 "2023-01-16T20:42:44Z")

</div>

What don't you like about the result that you got?

---

<div class="post-metadata">

**Author:** ![markedperf](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@markedperf](https://discuss.elastic.co/u/markedperf)\
**Post date:** [January 16, 2023, 8:52pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278/3 "2023-01-16T20:52:36Z")

</div>

I'm trying to get them to something like:

ListofVals [  
{ IdName:123Name, IdVersion: 2.03 },  
{ IdName: 456Name, IdVersion: 2.03}  
]  
Or  
ListofVals [  
[0]  
IdName:123Name, IdVersion: 2.03  
[1]  
IdName: 456Name, IdVersion: 2.03  
]  
Something like that...a separation of grouped values that corresponds to what is in the CSV.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 16, 2023, 9:41pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278/4 "2023-01-16T21:41:03Z")

</div>

You can get the first using

```
    aggregate {
        task_id => "%{Id}"
        push_map_as_event_on_timeout => true
        code => '
            map["Id"] ||= event.get("Id")
            map["ListofVals"] ||= []
            if !( map["ListofVals"].include? event.get("IdName") )
                map["ListofVals"] << {"IdName" => event.get("IdName"), "IdVersion" => event.get("IdVersion")}
            end
            event.cancel()
        '
        timeout => 5
    }

```

I would use push\_map\_as\_event\_on\_timeout instead of push\_previous\_map\_as\_event in case there are out-of-order lines in the file.

I do not understand what you mean by the second.

---

<div class="post-metadata">

**Author:** ![markedperf](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@markedperf](https://discuss.elastic.co/u/markedperf)\
**Post date:** [January 16, 2023, 10:00pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278/5 "2023-01-16T22:00:41Z")

</div>

Thank you! The second was just bad pseudo code...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2023, 10:01pm UTC](https://discuss.elastic.co/t/import-csv-file-into-nested-fields/323278/6 "2023-02-13T22:01:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.