# Data Lost while import data in elastic search using logstash

**URL:** <https://discuss.elastic.co/t/data-lost-while-import-data-in-elastic-search-using-logstash/200177>\
**Category:** Logstash\
**Created:** [September 19, 2019, 10:20am UTC](https://discuss.elastic.co/t/data-lost-while-import-data-in-elastic-search-using-logstash/200177 "2019-09-19T10:20:46Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![chintanvasani](https://avatars.discourse-cdn.com/v4/letter/c/ecd19e/32.png) [@chintanvasani](https://discuss.elastic.co/u/chintanvasani)\
**Post date:** [September 19, 2019, 10:20am UTC](https://discuss.elastic.co/t/data-lost-while-import-data-in-elastic-search-using-logstash/200177/1 "2019-09-19T10:20:46Z")

</div>

I am using logstash 5.5.0 and elastic search 5.5.0.  
I am trying to import data from txt file with | separator to elastic search.  
The total number of records in txt file are **2139406** approx but while importing the data to elastic search roughly **1795821** records are imported while remaining records are lost.

Then I tried to import the file though the same logstash config file to txt file rather than elastic search and found no records are missed/lost.

eg  
output {  
if "insertupdate" in [tags] {  
# write events that didn't match to a file  
file { "path" =\> "\*\*\*\*\*\*\*\*\*/grok\_failures.txt" }  
}

stdout { codec =\> rubydebug { metadata =\> true } }  
}

instead of ES  
output {  
if "insertupdate" in [tags]  
{  
elasticsearch {  
hosts =\> ["\*\*\*\*\*\*\*\*\*\*\*"]  
index =\> "test143"  
action =\> "update"  
doc\_as\_upsert =\> true  
document\_id =\> "%{entityId}"  
document\_type=\> "client"  
}  
}

Hence please help me to resolve the same and what configuration i need to do for logstash and elastic search.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 17, 2019, 10:20am UTC](https://discuss.elastic.co/t/data-lost-while-import-data-in-elastic-search-using-logstash/200177/2 "2019-10-17T10:20:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.