# Ingest several csv and create geo\_point

**URL:** <https://discuss.elastic.co/t/ingest-several-csv-and-create-geo-point/272816>\
**Category:** Logstash\
**Created:** [May 12, 2021, 1:47pm UTC](https://discuss.elastic.co/t/ingest-several-csv-and-create-geo-point/272816 "2021-05-12T13:47:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![RichaMax](https://avatars.discourse-cdn.com/v4/letter/r/ebca7d/32.png) [@RichaMax](https://discuss.elastic.co/u/RichaMax)\
**Post date:** [May 12, 2021, 1:47pm UTC](https://discuss.elastic.co/t/ingest-several-csv-and-create-geo-point/272816/1 "2021-05-12T13:47:59Z")

</div>

Hello everyone,  
I'm new to ELK and I have some questions regarding the ingestion of csv files and I couldn't find what I was looking for on the forum.

I want to ingest several csv into elasticsearch in different index and display their content on the map from Kibana. The csvs have almost the same schema (the header of the csv are the same but the type of each column the might change from one file to an other).

I manage to ingest one of the csv using `logstash` with the following configuration:

(I simplified the header of the csv for readability)

```auto
input {
    file {
        path => "path_to_my_first_csv"
        start_position => beginning
    }
}
filter {
    csv {
        columns => ["col1","col2","longitude","latitude"]
        separator => ","
        }

    mutate {
        convert => { "latitude" => "float" }
        convert => { "longitude" => "float" }
    }

    mutate {
        add_field => { "geo_location" => "%{latitude},%{longitude}" }
    }
}

output {
    stdout
    {
        codec => rubydebug
    }
     elasticsearch {
        action => "index"
        hosts => ["my_adress:9200"]
        index => "my_first_data"
    }
}

```

I added the "geo\_location" to create a geo\_point during mapping

for the mapping I created an "Index Template" in `Stack Management\Index Management` with the following mapping:

```json
"mappings": {
	"dynamic": "true",
	"dynamic_date_formats": [
		"strict_date_optional_time",
		"yyyy/MM/dd HH:mm:ss Z||yyyy/MM/dd Z"
	],
	"dynamic_templates": [],
	"date_detection": true,
	"numeric_detection": false,
	"properties": {
		"col1": {
		"type": "keyword"
		},
		"col2": {
		"type": "keyword"
		},
		"geo_location": {
		"type": "geo_point"
		},
		"latitude": {
		"type": "float"
		},
		"longitude": {
		"type": "float"
		}
	}
}

```

After this I was able to retrieve my data and display it on the Map in Kibana.

My first question is, is this the correct way to ingest a single CSV and create a geo\_point given latitude and longitude coordinates ? Are there more efficient way to ingest CSV file?  
As of now it took me ~25minutes to ingest a csv with 2376299 rows and 40 columns (~400MB on disk). I was thinking of using [GitHub - moshe/elasticsearch\_loader: A tool for batch loading data files (json, parquet, csv, tsv) into ElasticSearch](https://github.com/moshe/elasticsearch_loader) but I'm not sure this is the right approach.

Secondly, I created a python script to generate the mappings given a csv and I would like to use the generated mapping instead of having to create a Template manualy for each csv, who can I do this ?

Finally, as the logstash configuration is enough to ingest each of the csv how can I reuse it to ingest all of my csv but use my different mappings ? I would like to have a different index for each of the csv

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 9, 2021, 1:48pm UTC](https://discuss.elastic.co/t/ingest-several-csv-and-create-geo-point/272816/2 "2021-06-09T13:48:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.