I'm working with logstash, for uploading data into the elaststicsearch from a CSV file.
I've succeeded to upload it to the elasticsearch, but the only issue I am having right now
is when the CSV file is uploaded, logstash still waits on the command prompt until
this file is modified and as soon as file contents are changed/saved it starts updating
data again to the elasticsearch and does not simply teminates.
IT NEEDS TO BE CLOSED EXPLICITLY
Is is possible to terminate logstast process after data is uploaded once?
Can we configure this some where?
No, this is not possible with the file input. You could use the stdin input but then you won't be able to resume if Logstash is interrupted. See issue #48.
@magnusbaeck Thanks for the update.
Is these any way [input] that I can write a program to read CSV and keep putting records one by one until EOF.
In other words,
Can I upload record by record into elasticsearch using logstash?
We have a case where file sizes are limited(<1GB or <500MB) and are never updated/appended.
We have created a local web app which provides role based access where users can upload their files, and at the server side we have written functionality to automatically upload the file using elasticsearch.
But since logstash keeps waiting for file/content change even after file upload completed it keeps no of command prompts alive/listening to files. This is cumbersome and is a overhead.
Suggest a solution.
If you don't care about being able to resume submissions of files, use the stdin input. Once that plugin reaches EOF it'll shut down Logstash.
If you do need to care about resuming submissions, use the file input but monitor the sincedb file. Once the offset of the input file is equal to the file size Logstash is done.