# Logstash exports data multiple times to csv

**URL:** <https://discuss.elastic.co/t/logstash-exports-data-multiple-times-to-csv/162779>\
**Category:** Logstash\
**Created:** [January 3, 2019, 10:09am UTC](https://discuss.elastic.co/t/logstash-exports-data-multiple-times-to-csv/162779 "2019-01-03T10:09:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![eblindeman](https://avatars.discourse-cdn.com/v4/letter/e/b782af/32.png) [@eblindeman](https://discuss.elastic.co/u/eblindeman)\
**Post date:** [January 3, 2019, 10:09am UTC](https://discuss.elastic.co/t/logstash-exports-data-multiple-times-to-csv/162779/1 "2019-01-03T10:09:18Z")

</div>

Hello

I'm trying to create an export using logstash (5.6.12) with the pipeline as written below. But unfortunately it exports everything continously in stead of everything just once.

I start it with the following command in a pod in a kubernetes cluster

logstash --path.data /tmp/ --http.port 9601 -f ./test-erik-acc.conf

What do i do wrong?

test-erik-acc.conf:  
input {  
elasticsearch {  
hosts =\> "es-client.cts-prd:9200"  
index =\> "cts-portaal\*"  
scroll =\> "30s"  
query =\> '  
{  
"query": {  
"bool": {  
"must": [  
{  
"query\_string": {  
"query": "batchRequestInfo.receiverID: (8716871000002)",  
"analyze\_wildcard": true,  
"all\_fields": true  
}  
},  
{  
"match\_phrase": {  
"topic": {  
"query": "pushed-meteringrequests"  
}  
}  
},  
{  
"range": {  
"batchRequestInfo.intakeDateTime": {  
"gte": "now-9d/d",  
"lt": "now-8d/d",  
"format": "epoch\_millis"  
}  
}  
}  
],  
"must\_not":   
}  
}  
}'  
}}  
filter {  
mutate {  
rename =\> {  
"pushDateTime" =\> "[headEndPushInfo][pushDateTime]"  
}  
}  
ruby {  
init =\> "  
begin  
@@csv\_file = '/tmp/pushed.csv'  
@@csv\_headers = ['headEndPushInfo', 'correlationId', 'facilitatingRNB' ,]  
if File.zero?(@@csv\_file) || !File.exist?(@@csv\_file)  
CSV.open(@@csv\_file, 'w') do |csv|  
csv \<\< @@csv\_headers  
end  
end  
end  
"  
code =\> "  
begin  
@@csv\_file = event.get('[@metadata][csv\_file]')  
@@csv\_headers = event.get('[@metadata][csv\_headers]')  
end  
"  
}  
csv {  
columns =\> ["pushDateTime", "correlationId", "facilitatingRNB"]  
}

}  
output {  
csv {  
fields =\> ["[headEndPushInfo][pushDateTime]", "correlationId", "facilitatingRNB"]  
path =\> "/tmp/pushed.csv"  
}  
stdout {  
codec =\> rubydebug {  
metadata =\> true  
}  
}  
}

after a few hours (export should be 900000 lines) the export looks like(and is 4660000 lines):

headEndPushInfo,correlationId,facilitatingRNB  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696003509,b8497f80-6696-49a7-a1f0-52b24f562c29,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002  
1545696022448,786cc485-4efd-4387-8a43-a12e56c1605a,8716871000002

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 31, 2019, 10:09am UTC](https://discuss.elastic.co/t/logstash-exports-data-multiple-times-to-csv/162779/2 "2019-01-31T10:09:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.