Execute logstash script as batch


(Amjad Hussain) #1

I am using Logstash 5.4.1.
I need to execute one of my logstash script only once.
I don't want logstash thread to run in background. I want to use logstash script like a batch.
Once thread reach end of the file in input file. It should terminate automatically.
Can someone help me with that?

Note: my input is some data file and output is elasticsearch.

I tried with this syntax. But it didn't work.
/logstash.bat -f myconf.conf < testLog.log


(Mark Walkom) #2

type testLog.log | logstash.bat -f myconf.conf will work.


(Amjad Hussain) #3

It didn't work. Logstash script is not terminated automatically.

This is my script.

input {
file {
path => "C:/PADS_ELK/Logstash/logstash-5.4.1/data/input.txt"
start_position => “beginning”
}
}
filter{
csv{
columns => [“col1”,“col2”,“col3”]
separator => “,”
}
ruby {
add_field => {
“col3” => “%{[@timestamp]}”
“updated_by” => “batch”
}
code => "if event.get('col1').to_i < 1000
event.set('col1','1000')
end
if event.get('col2').to_i < 10
event.set('col2','10')
end"

remove_field => ["message", "path","@version","@timestamp","type","host"]
}
}

output {

stdout { codec => rubydebug }
elasticsearch{
hosts => [“localhost:9200”]
index => "my_index"
action => "update"
document_type => "my_type"
document_id => "%{col1}%{col2}%{col3}"
doc_as_upsert => “true”
}
}


(Magnus Bäck) #4

If you want to redirect the file to Logstash you need to use a stdin input, not a file input.


(Amjad Hussain) #5

I need to read one file which has data. Then post it to Elasticsearch.
Once Logstash reach end of the file, logstash thread should terminate.
It should not keep running.


(Leandro Maciel) #6

Use the stdin input instead of the file input.

Your input should be:

input { 
    stdin { } 
}

Then you will need to use the command that @warkolm posted before.

type testLog.log | logstash.bat -f myconf.conf 

Also, read this topic, it seems to be what you want, maybe it will help: stop logstash after processing the file.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.