Hey all,
I've been playing around with Logstash-5.0.0 on Windows 10 for awhile, trying to get it to index some data into Elasticsearch. I currently have Elasticsearch-5.0.0 and Kibana-5.0.0 running and working, but whenever I try to use Logstash, it doesn't create an index in ES. I've been looking around for an answer in the forums but I can't find anything to help me.
Here is my logstash.conf file:
input {
file {
path => "D:\ProjectData-2015\highways.csv"
start_position => "beginning"
sincedb_path => "D:\dbfile\.sincedb_729a52b6638a2073b1e7c998dc6ad48b"
}
}
filter {
csv {
columns => ["highwayid", "shortdirection", "direction", "highwayname"]
separator => ","
}
}
output {
elasticsearch {
action => "index"
hosts => ["localhost:9200"]
index => "cs453_project"
workers => 1
}
stdout {
codec => rubydebug
}
}
And here is my .csv file contents:
highwayid,shortdirection,direction,highwayname
3,N,NORTH,I-205
4,S,SOUTH,I-205
When I run bin\logstash -f logstash.conf in the command prompt, I get this:
C:\logstash-5.0.0\bin>logstash.bat -f logstash.conf
Sending Logstash logs to C:/logstash-5.0.0/logs which is now configured via log4j2.properties.
[2016-11-07T10:50:52,325][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://localhost:9200"]}}
[2016-11-07T10:50:52,325][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2016-11-07T10:50:52,456][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"default"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2016-11-07T10:50:52,456][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["localhost:9200"]}
[2016-11-07T10:50:52,472][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2016-11-07T10:50:52,472][INFO ][logstash.pipeline ] Pipeline main started
[2016-11-07T10:50:52,525][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
I've tried several different things, but I still can't get the index to show up in ES. I'm using Sense in Chrome to check whether or not it's been indexed. Any ideas on how to fix this? I'm not sure what I'm missing. I appreciate any help!