Hi
I have installed Elastic Search, Logstash and Kibana on my windows laptop.
Windows directory path to Kibana: C:\Users\rajapraveenk\Downloads\kibana-6.4.2-windows-x86_64
Windows directory path to Logstash: C:\Users\rajapraveenk\Downloads\logstash-6.4.2\logstash-6.4.2
Windows directory path to Elastic search: C:\Users\rajapraveenk\Downloads\elasticsearch-6.4.2\elasticsearch-6.4.2
I have my logstash.conf file in C:\Users\rajapraveenk\Downloads\elasticsearch-6.4.2\elasticsearch-6.4.2\config\logstash.conf
My logstash.conf file contains below data
input {
file {
path => "/var/lib/tomcat8/logback/*.log"
codec => "json"
type => "logback"
}
}
output {
if [type]=="logback" {
elasticsearch {
hosts => [ "localhost:9200" ]
index => "logback-%{+YYYY.MM.dd}"
}
}
}
To see the logs in Kibana, I have initially started Elastic Search by providing C:\Users\rajapraveenk\Downloads\elasticsearch-6.4.2\elasticsearch-6.4.2\bin\elasticsearch.bat in the windows command prompt.
Then I started logstash by providing C:\Users\rajapraveenk\Downloads\logstash-6.4.2\logstash-6.4.2\bin> logstash -f C:\Users\rajapraveenk\Downloads\logstash-6.4.2\logstash-6.4.2\config\logstash.conf
The console logs of windows command prompt after running the logsatsh.conf file is as below
Sending Logstash logs to C:/Users/rajapraveenk/Downloads/logstash-6.4.2/logstash-6.4.2/logs which is now configured via log4j2.properties
[2018-10-18T13:10:47,159][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-10-18T13:10:47,812][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.2"}
[2018-10-18T13:10:52,120][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-10-18T13:10:52,903][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://localhost:9200/]}}
[2018-10-18T13:10:52,927][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-10-18T13:10:53,166][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-10-18T13:10:53,267][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-10-18T13:10:53,273][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>6}
[2018-10-18T13:10:53,305][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-10-18T13:10:53,338][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-10-18T13:10:53,393][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"default"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-10-18T13:10:53,999][INFO ][logstash.inputs.file ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"C:/Users/rajapraveenk/Downloads/logstash-6.4.2/logstash-6.4.2/data/plugins/inputs/file/.sincedb_bec0d662147837b9e3af8a448a7d314b", :path=>["/var/lib/tomcat8/logback/*.log"]}
[2018-10-18T13:10:54,054][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x64ed3f87 run>"}
[2018-10-18T13:10:54,135][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2018-10-18T13:10:54,154][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2018-10-18T13:10:54,613][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
I don't know whether logstash is reading the logstash.conf file or not
When I ran Kibana by providing below like
C:\Users\rajapraveenk\Downloads\kibana-6.4.2-windows-x86_64\kibana-6.4.2-windows-x86_64\bin\kibana.bat
I could not find my index pattern when search for logback- in the index pattern field in Kibana server.
Please find below screenshot for your reference.
Can anyone help me....
