Logstash-5.0.0 not creating ES index on Windows 10


(Steve Brown) #1

Hey all,

I've been playing around with Logstash-5.0.0 on Windows 10 for awhile, trying to get it to index some data into Elasticsearch. I currently have Elasticsearch-5.0.0 and Kibana-5.0.0 running and working, but whenever I try to use Logstash, it doesn't create an index in ES. I've been looking around for an answer in the forums but I can't find anything to help me.

Here is my logstash.conf file:

input {
file {
path => "D:\ProjectData-2015\highways.csv"
start_position => "beginning"
sincedb_path => "D:\dbfile\.sincedb_729a52b6638a2073b1e7c998dc6ad48b"
}
}

filter {
csv {
columns => ["highwayid", "shortdirection", "direction", "highwayname"]
separator => ","
}
}

output {
elasticsearch {
action => "index"
hosts => ["localhost:9200"]
index => "cs453_project"
workers => 1
}
stdout {
codec => rubydebug
}
}

And here is my .csv file contents:

highwayid,shortdirection,direction,highwayname
3,N,NORTH,I-205
4,S,SOUTH,I-205

When I run bin\logstash -f logstash.conf in the command prompt, I get this:

C:\logstash-5.0.0\bin>logstash.bat -f logstash.conf
Sending Logstash logs to C:/logstash-5.0.0/logs which is now configured via log4j2.properties.
[2016-11-07T10:50:52,325][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://localhost:9200"]}}
[2016-11-07T10:50:52,325][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2016-11-07T10:50:52,456][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"default"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2016-11-07T10:50:52,456][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["localhost:9200"]}
[2016-11-07T10:50:52,472][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2016-11-07T10:50:52,472][INFO ][logstash.pipeline ] Pipeline main started
[2016-11-07T10:50:52,525][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

I've tried several different things, but I still can't get the index to show up in ES. I'm using Sense in Chrome to check whether or not it's been indexed. Any ideas on how to fix this? I'm not sure what I'm missing. I appreciate any help!


(Mark Walkom) #2

It's probably the sincedb, delete the file and try again.


(Steve Brown) #3

I've tried doing that, but it doesn't seem to fix the issue. I know you can specify a path such as /dev/null in Linux that will basically force the sincedb file to not be created so you don't have to delete it. Is there a way to do that in Windows?


(Mark Walkom) #4

I think it's just nul?


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.