# Error ingestion two csv in elastic search

**URL:** <https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435>\
**Category:** Logstash\
**Created:** [April 23, 2020, 10:07am UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435 "2020-04-23T10:07:52Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 10:07am UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/1 "2020-04-23T10:07:52Z")

</div>

Hello it is possible to ingest of two csv files in elastic using file beat and logstsh with the same index

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 23, 2020, 10:33am UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/2 "2020-04-23T10:33:04Z")

</div>

Yes.

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 10:53am UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/3 "2020-04-23T10:53:36Z")

</div>

I use this config logstsh

```
input {
beats {
host => "localhost"
port => "5044"
 }
}                  

filter {
csv {
    separator => ";"
	columns => ["chaine", "job", "date_plan", "statut", "date_debut", "date_fin", "serveur", "numero_passage", "application", "sous_application", "date fraicheur"]
}

    if [sous_application] == "DWH PRODUCTION" {
     mutate {
     add_field => { "UNIVERS" => "Entrepôt de données Production" }
     }
}
     if [statut] == "OK" {
     mutate {
     add_field => { "statut_globale" => "1" }
     }
     }

   if [statut] == "EN-ERREUR" {
     mutate {
     add_field => { "statut_globale" => "0" }
     }	  	  	  
    }

     if [statut] == "EN-ATTENTE" {
     mutate {
     add_field => { "statut_globale" => "3" }
     }	  	  	  
  }

   date {
   match => ["date_plan" , "YYYY-MM-dd"]
   timezone => "Europe/Paris"
   }

 date {
 match => ["date_debut" , "YYYY-MM-dd HH:mm:ss"]
 timezone => "Europe/Paris"
}

date {
match => ["date_fin" , "YYYY-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
 }

date {
match => ["date fraicheur" , "YYYY-MM-dd"]
timezone => "Europe/Paris"
}

}

output {
stdout {
codec => rubydebug
}
   elasticsearch { 
   hosts => "http://localhost:9200" 
   index => "v00"
   }
 }
```

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 11:00am UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/4 "2020-04-23T11:00:02Z")

</div>

I have a erroe: the the column date fraicheur not creat in elastic search

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 23, 2020, 12:05pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/5 "2020-04-23T12:05:25Z")

</div>

Bonjour 😉

May I recommend you change `date fraicheur` to `date_fraicheur`.

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 2:57pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/6 "2020-04-23T14:57:47Z")

</div>

I tested this but the date\_fraicheur column is not created in elastic

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 23, 2020, 3:15pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/7 "2020-04-23T15:15:43Z")

</div>

Can you share the full logstash configuration file and the first 10 lines of your CSV file?

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 5:12pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/8 "2020-04-23T17:12:03Z")

</div>

this is the first CSV file

 ![csv1](https://us1.discourse-cdn.com/elastic/original/3X/c/1/c1224395b8637d7643c3849d3b75a129c6142865.png)

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 5:12pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/9 "2020-04-23T17:12:38Z")

</div>

this is the second CSV file

 ![Capturecsv3](https://us1.discourse-cdn.com/elastic/original/3X/4/a/4a74c33481616b1d580636fe5dc9a0322a9f060a.png)

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 23, 2020, 5:25pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/10 "2020-04-23T17:25:25Z")

</div>

and the full logstash configuration file is :

```
input {
beats {
host => "localhost"
port => "5044"
 }
}                  

filter {
csv {
separator => ";"
columns => ["chaine", "job", "date_plan", "statut", "date_debut", "date_fin", "serveur", "numero_passage", "application", "sous_application", "date fraicheur"]
}

if [sous_application] == "DWH PRODUCTION" {
 mutate {
 add_field => { "UNIVERS" => "Entrepôt de données Production" }
 }
} 
 if [statut] == "OK" {
  mutate {
 add_field => { "statut_globale" => "1" }
 }
 }

if [statut] == "EN-ERREUR" {
 mutate {
 add_field => { "statut_globale" => "0" }
 }	  	  	  
}

 if [statut] == "EN-ATTENTE" {
 mutate {
 add_field => { "statut_globale" => "3" }
 }	  	  	  
 }

 date {
 match => ["date_plan" , "YYYY-MM-dd"]
 timezone => "Europe/Paris"
 }

date {
match => ["date_debut" , "YYYY-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
}

 date {
 match => ["date_fin" , "YYYY-MM-dd HH:mm:ss"]
 timezone => "Europe/Paris"
}

date {
match => ["date fraicheur" , "YYYY-MM-dd"]
timezone => "Europe/Paris"
}

}

 output {
 stdout {
 codec => rubydebug
}
   elasticsearch { 
   hosts => "http://localhost:9200" 
   index => "v00"
 }
}
```

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [April 27, 2020, 11:11am UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/11 "2020-04-27T11:11:00Z")

</div>

Any idea???

---

<div class="post-metadata">

**Author:** ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\
**Post date:** [May 11, 2020, 2:08pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/12 "2020-05-11T14:08:32Z")

</div>

Any Idea???

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 8, 2020, 2:08pm UTC](https://discuss.elastic.co/t/error-ingestion-two-csv-in-elastic-search/229435/13 "2020-06-08T14:08:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.