# How do I take multiple CSV files into a single Elasticsearch index using logstash?

**URL:** https://discuss.elastic.co/t/how-do-i-take-multiple-csv-files-into-a-single-elasticsearch-index-using-logstash/228661
**Category:** Logstash
**Created:** [April 18, 2020, 4:53pm UTC](https://discuss.elastic.co/t/how-do-i-take-multiple-csv-files-into-a-single-elasticsearch-index-using-logstash/228661 "2020-04-18T16:53:16Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Drashti\_Shah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drashti_shah/32/66581_2.png) [@Drashti\_Shah](https://discuss.elastic.co/u/Drashti_Shah)
#### Post date: [April 18, 2020, 4:53pm UTC](https://discuss.elastic.co/t/how-do-i-take-multiple-csv-files-into-a-single-elasticsearch-index-using-logstash/228661/1 "2020-04-18T16:53:16Z")

</div>

I have two CSV files in which,  
CSV 1 (student\_master) has the following fields: Student ID, First Name, Gender, State and in  
CSV 2,(student\_marks\_data) I have Student ID, Date, Math, Physics, Chemistry, Total, Percentage and Grade.

I need to create a logstash data adapter to load the `csv_1` into an Elasticsearch index called “`student_master`”.

While loading the student marks data, I have to link with “`student_master`” index (created in previous step) based on the `student_id` column and fetch the student’s firstname, lastname, gender and load into a new index namely “student\_marks\_[current\_date]”.

I need the output as an index in which it is one index I have Student ID, Name, Gender, Date, Math, Physics, Chemistry, Total, Percentage and Grade.

I tried the below code:

```
         input {
          file {
            type => "csv1"
            path => "/home/vunet/Downloads/student_master.csv"
            start_position => "beginning"
            sincedb_path => "/dev/null"
          }
          file {
            type => "csv2"
            path => "/home/vunet/Downloads/student_marks_new.csv"
            start_position => "beginning"
            sincedb_path => "/dev/null"
          }
        
        }
        
        filter {
                if [type] == "csv1"{
                         csv {
                                columns => ["ID", "First name", "Last name", "Gender", "City", "State"]
                                remove_field => ["City", "State"]
                        }
                }
        
                if [type] == "csv2"{
                         csv {
                                columns => ["ID", "Date", "Chemistry", "Physics", "Biology", "Total", "Percentage", "Grade"]
                                remove_field => ["ID"]
                        }
                        date {
                                match => ["Date", "dd/MM/yyyy"]
                                target => "@timestamp"
                        }
                }}
    
    output {
           # elasticsearch {
           # doc_as_upsert => true
           # document_type => "doc"
           # index => "students-new-%{+YYYY.MM.dd}"
           # }
            stdout {
                    codec => rubydebug
            }
    }

```

How do I make a join (in SQL words) or how do I map with the ID of one CSV with the ID of the second CSV to create a single index?

Please do help. Thank you in advance.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 16, 2020, 4:53pm UTC](https://discuss.elastic.co/t/how-do-i-take-multiple-csv-files-into-a-single-elasticsearch-index-using-logstash/228661/2 "2020-05-16T16:53:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
