Good morning everyone.
Today I have downloaded Logstash, ElasticSearch and Kibana.
I have everything configured and its all displaying the way it should except for when I run my config file in Powershell. It just sits and hangs and doesnt do much. The CPU Spikes to 100% and stays there but no new "index" is created.
Here is my config file for those that wanna see what I am working with:
input {
file {
path => "X:/XXXXXXX/*.*"
start_position => "beginning"
sincedb_path => "NUL"
}
}
filter {
csv {
separator => ","
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "XXXXX"
}
stdout { codec => rubydebug }
}
If anyone has any advice that would be great. I have tried changing the trailing on "X:/XXXXXXX/." to "X:\XXXXXXX*.*" and it just sits there also.
Here is the output I have from Powershell.
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Try the new cross-platform PowerShell https://aka.ms/pscore6
PS C:\Windows\system32> D:\ELKSTACK\logstash\bin\logstash.bat -f D:\ELKSTACK\logstash\config\email.conf
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService (file:/D:/ELKSTACK/logstash/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to D:/ELKSTACK/logstash/logs which is now configured via log4j2.properties
[2019-06-24T10:00:53,477][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-06-24T10:00:53,489][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.1.1"}
[2019-06-24T10:00:57,160][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-06-24T10:00:57,334][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2019-06-24T10:00:57,374][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>7}
[2019-06-24T10:00:57,376][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2019-06-24T10:00:57,397][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}
[2019-06-24T10:00:57,406][INFO ][logstash.outputs.elasticsearch] Using default mapping template
[2019-06-24T10:00:57,416][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, :thread=>"#<Thread:0x64d91542 run>"}
[2019-06-24T10:00:57,506][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2019-06-24T10:00:57,705][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-06-24T10:00:57,732][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2019-06-24T10:00:57,735][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-06-24T10:00:57,979][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}