Not able to create Index in ES with Logstash

[rkhapre (RK)]

Continuing the discussion from Not able to create Index with Logstash:

@rkhapre

input {
file {
path => "C:/TEMP/*.txt"
type => "core2"
start_position => "beginning"
}
}

filter {

}

output {
elasticsearch {
action => "index"
host => "localhost"
cluster => "clustertest"
node_name=> "test"
index => "market"

}
 stdout {
    codec => rubydebug
 }

}

[warkolm (Mark Walkom)]

You haven't given much information.

What happens when you run this?
Have you checked your ES logs? Have you made sure your cluster is up and running ok?

[rkhapre (RK)]

Hi

Yes the 'clustertest' cluster is up and running and also when it starts it is showing the node has started.

When i just create a index with a document in ES it works, but when i run the above config file through logstash it doesn't create a index.

The above code is in my config file- simple.conf for logstash, i run it through logstash command

logstash -f simple.conf

[warkolm (Mark Walkom)]

If you are always using the same log input then you are probably running into a sincedb issue. See if you have a .sincedb file and if you do delete and try running LS again.

[rkhapre (RK)]

Hi,

You seems to be correct as when i change the index name its working but not the next time
Where i can find .sincedb file to delete it

I do not see any, below is screenshot of my logstash home folder. In the documentation its written it will be available @$HOME/.sincedb*

[magnusbaeck (Magnus Bäck)]

That's the Logstash installation directory and I doubt you have the HOME variable pointing there. If you start Logstash with --debug it'll tell you what sincedb files it reads. See below for the (Logstash 1.5) logic used when deciding the directory for sincedb files (if the sincedb_path parameter isn't set).

logstash-plugins/logstash-input-file/blob/v0.1.10/lib/logstash/inputs/file.rb#L94-L104