Logstash stuck midway while processing while running from java

Elastic Stack Logstash
1

Logstash stuck at midway while processing a log file only while running from java(This issue doesn't occur when i run the same thing from logstash console)

This is my config file

input {
file {
path => "D:\logstash\logstash-6.1.0\bin\repoprolog.log"
start_position => "beginning"
sincedb_path => "/dev/null"
codec => multiline
{
pattern => "[%{DATA}"
negate => true
what => "previous"
auto_flush_interval => 1
}
}
}
filter {
grok {
match => {"message" => "%{DATE}_%{TIME} - %{POSINT:value} %{DATA:status} %{WORD:result} %{GREEDYDATA:final}"}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
}
log file contains only 1000 lines.Do i have to make any changes with jvm.options file?

2

What do you mean "running from Java" and "Logstash console"? Are these specific commands? Specific configurations? Modern releases of Logstash always runs in Java, regardless of how it is invoked.

Do the logs reveal anything interesting?
Your grok pattern should probably be anchored (prefixed with ^, with pattern capturing from beginning of each message), as unanchored patterns tend to do a lot more work than we intend them to do.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

4

Answering this old thread as I met @sunilkumar at a conference.

My advice is to use instead Ingest Node and the Grok processor instead of trying to start Logstash "embedded" from a Java application.

Also starting Logstash as a service with a pipeline defined in elasticsearch itself might be another solution. So you can just update the pipeline in Elasticsearch using either the API or using Kibana pipeline management feature (available in X-Pack basic).