# Load Several CSV Files to several indices in Elastic cloud based on date field
**URL:** https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382
**Category:** Logstash
**Created:** [March 27, 2020, 10:03am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382 "2020-03-27T10:03:34Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@rsuper\_6616](https://discuss.elastic.co/u/rsuper_6616)
#### Post date: [March 27, 2020, 10:03am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/1 "2020-03-27T10:03:34Z")
I have several CSV I need to load them to elastic cloud based on date field such that each index will contain only one day. is it possible?
---
### Author:  [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [March 28, 2020, 3:06am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/2 "2020-03-28T03:06:56Z")
Put into the output as below:
```auto
output {
elasticsearch {
hosts => ["blah:9200"]
index => "csv-%{+YYYY.MM.dd}"
}
}
```
Regards,
Fadjar Tandabawana
---
### Author:  [@rsuper\_6616](https://discuss.elastic.co/u/rsuper_6616)
#### Post date: [March 28, 2020, 7:31am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/3 "2020-03-28T07:31:43Z")
Thanks Fadjar,
What I mean that I have history data split in the following based on a date field inside the data
csv-20200112
Csv-20200114
Csv-20200115
.
.
Csv-20200325
And I want to load each csv to a separate index on elastic search corresponding to the input csv
---
### Author:  [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [March 28, 2020, 8:56am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/4 "2020-03-28T08:56:26Z")
Do your process per file
ex:
CSV-20200112
```auto
input {
file => "csv-20200112"
tags => "20200112"
.
.
}
filter {
if [tags] == "20200112"
process something here
}
output {
if [tags] == "20200112"
elasticsearch {
host => "localhost:9200"
index => "blah-%tags"
}
}
```
save as 20200112.conf
do to another csv
Don't forget the tags of each process.
You can build the conf file using bash script and using template and sed
Regards,
Fadjar Tandabawana
---
### Author:  [@rsuper\_6616](https://discuss.elastic.co/u/rsuper_6616)
#### Post date: [March 28, 2020, 9:11am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/5 "2020-03-28T09:11:56Z")
Thanks Fadjar340
What is the usage of tags if I am going to create conf for each file file (date)
---
### Author:  [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [March 28, 2020, 11:21am UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/6 "2020-03-28T11:21:01Z")
I think you can use script to do it, like below:
```auto
#!/bin/bash
shopt -s extglob nullglob
basedir=/home/csv
first=10
increment=1
omitdir=cmmdm
if [[-z $omitdir]]; then
cdarray=( "$basedir"/*/ )
else
cdarray=( "$basedir"/!($omitdir)/ )
fi
cdarray=( "${cdarray[@]#"$basedir/"}" )
cdarray=( "${cdarray[@]%/}" )
length=${#cdarray[@]}
vals=($(seq -w -s ' '10 $increment $length))
for ((i=0; i<${length}; i++)); do
# echo "First parameter : ${cdarray[$i]} -- second parameter : ${vals[$i]}"
sed s/%file%/${cdarray[$i]}/g ./template.txt > ./${vals[$i]}-${cdarray[$i]}.conf
done
```
The template should follow the standard conf then just add %file% as tags that you want to use
Regards,
Fadjar Tandabawana
---
### Author:  [@rsuper\_6616](https://discuss.elastic.co/u/rsuper_6616)
#### Post date: [March 29, 2020, 9:53pm UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/7 "2020-03-29T21:53:11Z")
Thanks for you help
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 26, 2020, 9:53pm UTC](https://discuss.elastic.co/t/load-several-csv-files-to-several-indices-in-elastic-cloud-based-on-date-field/225382/8 "2020-04-26T21:53:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.