I am using Redhat version 7.0 (Maipo). I am trying to install XPack on ELK. I got success in ELK and Kibana. But when I am trying to run config file below error is coming.
[2019-06-07T13:48:05,392][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}
[2019-06-07T13:48:05,394][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>2, :thread=>"#<Thread:0x2751c7e8@/home/ELK/logstash-6.0.1/logstash-core/lib/logstash/pipeline.rb:290 run>"}
[2019-06-07T13:48:05,462][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-06-07T13:48:05,463][INFO ][logstash.licensechecker.licensereader] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2019-06-07T13:48:05,467][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2019-06-07T13:48:05,475][WARN ][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused) {:url=>http://localhost:9200/, :error_message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)
[2019-06-07T13:48:05,476][ERROR][logstash.licensechecker.licensemanager] Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2019-06-07T13:48:05,477][WARN ][logstash.licensechecker.xpackinfo] Nil response from License Server
Further below is my config file. I am using 6.0.1 version and at the beginner stage. Request you to help on getting the step by step XPack installation process.
input {
file {
path => "/var/application/logs/osbserver1.log"
}
}
filter
{
grok {
patterns_dir => "/home/ELK/logstash-6.0.1/pattern/patterm"
break_on_match => false
match => [ "message" ,"%{osb}"]
}
if [EventType] == "BEGIN" and [eventType] != "OUTBOUND" and [eventType] != "INBOUND"
{
mutate{
add_tag => "taskbegin"
}
}
mutate{
add_field => {
"StartTime" => "%{logtime}.%{ms}"
}
remove_field => ["logtime","ms" ]
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["//RequestHeader/GeneralConsumerInformation/consumerID/text()",consumerID]
force_array => "false"
store_xml => "false"
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["//RequestHeader/GeneralConsumerInformation/correlationID/text()",correlationID]
force_array => "false"
store_xml => "false"
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["//RequestHeader/GeneralConsumerInformation/RequestBody/payingCustomerID/text()",payingCustomerID]
force_array => "false"
store_xml => "false"
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["//RequestBody/amount/text()",amount]
force_array => "false"
store_xml => "false"
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["//RequestBody/channelId/text()",channelId]
force_array => "false"
store_xml => "false"
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["/RequestBody/productId/text()",productId]
force_array => "false"
store_xml => "false"
}
xml{
target => "data"
source => "payload"
remove_namespaces => "true"
xpath => ["//RequestBody/externalTransactionID/text()",externalTransactionID]
force_array => "false"
store_xml => "false"
}
mutate {
add_field => { "token" => "kNBDTnesOnWvUhhhwiVYbBZiqWZaQvYd" }
}
}
output
{
elasticsearch {
hosts => [ "192.168.41.4:9200" ]
user => elastic
password => elastic
index => "logstash-*"
}
}