Logstash pipeline is starting but not executing

Elastic Stack Logstash
1

Hi

My logstash config file is

input {
file {
path => "/home/Esearch/inp-lgstsh"
start_position => "beginning"
ignore_older => 0
}
}
filter {
if [path] =~ "access" {
mutate { replace => { "type" => "apache_access" } }
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}
geoip {
source => "clientip"
}
date {
match => [ "timestamp" , "YYYY-mm-dd HH:mm:ss Z" ]
}
}

output {
elasticsearch {

protocol => http

hosts => ["localhost:9200"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"

}
stdout { codec => rubydebug }
}

Input file is
2016-06-05 02:42:55,664 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(87.987.65.34, storageID=DS-1921357720-127.0.0.1-50010-1384944321031, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-5058f35e-cc62-4ee5-be05-a5ec592d9238;nsid=2087330736;c=0), blocks: 647, processing time: 6 msecs

when executing bin/logstash -f logstash.config -

Settings: Default pipeline workers: 4
Pipeline main started

and it is not executing furthur.

Can i get reason for this abnormal behaviour.

Thanks in advance

2

Logstash is probably tailing the file. Shutting down Logstash, clearing the sincedb file used for the input file, and restarting Logstash should fix the problem.

3

Hi
I restarted logstash and cleared sincedb file. But still problem persists.
Please help me on this.
And what is this sincedb file created?

Thanks in advance

4

Hi

I am new to logstash.
Actually when I change input file it is working but not with the same input file.
Can you explain me the reason and also about sincedb file.

Thanks in advance

5

The file input's documentation describes what sincedb is and how it works.

6

Hi

I m new to ELK too and having same issue here. after clearing sincedb. it is still same not running

7

Hi

Try changing the input file to another file.

sincedb is the registry which consists of inode number,major device number,minimum device number,byte offset.
Due to Byte offset, logstash may be tailing in the file given as input.
Once logged by logstash,it cant be taken by logstash until either new logs should be added to input file or input file need to be changed to another file.

Hope this helps.