X PACK CONFIGURATION

I am using Redhat version 7.0 (Maipo). I am trying to install XPack on ELK. I got success in ELK and Kibana. But when I am trying to run config file below error is coming.

[2019-06-07T13:48:05,392][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}
    [2019-06-07T13:48:05,394][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>2, :thread=>"#<Thread:0x2751c7e8@/home/ELK/logstash-6.0.1/logstash-core/lib/logstash/pipeline.rb:290 run>"}
    [2019-06-07T13:48:05,462][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
    [2019-06-07T13:48:05,463][INFO ][logstash.licensechecker.licensereader] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
    [2019-06-07T13:48:05,467][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
    [2019-06-07T13:48:05,475][WARN ][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused) {:url=>http://localhost:9200/, :error_message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
    Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)
    [2019-06-07T13:48:05,476][ERROR][logstash.licensechecker.licensemanager] Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
    [2019-06-07T13:48:05,477][WARN ][logstash.licensechecker.xpackinfo] Nil response from License Server

Further below is my config file. I am using 6.0.1 version and at the beginner stage. Request you to help on getting the step by step XPack installation process. :frowning:

    input {
  file {
    path => "/var/application/logs/osbserver1.log"
  }
}
filter
{
 grok {
  patterns_dir => "/home/ELK/logstash-6.0.1/pattern/patterm"
break_on_match => false
match => [ "message" ,"%{osb}"]
}
if [EventType] == "BEGIN" and [eventType] != "OUTBOUND" and [eventType] != "INBOUND"
{
   mutate{
    add_tag => "taskbegin"
      }
        }
mutate{
                        add_field => {
                        "StartTime" => "%{logtime}.%{ms}"
                        }
                        remove_field => ["logtime","ms" ]
                        }
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["//RequestHeader/GeneralConsumerInformation/consumerID/text()",consumerID]
                force_array => "false"
                store_xml => "false"
}
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["//RequestHeader/GeneralConsumerInformation/correlationID/text()",correlationID]
                force_array => "false"
                store_xml => "false"
}
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["//RequestHeader/GeneralConsumerInformation/RequestBody/payingCustomerID/text()",payingCustomerID]
                force_array => "false"
                store_xml => "false"
}
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["//RequestBody/amount/text()",amount]
                force_array => "false"
                store_xml => "false"
}
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["//RequestBody/channelId/text()",channelId]
                force_array => "false"
                store_xml => "false"
}
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["/RequestBody/productId/text()",productId]
                force_array => "false"
                store_xml => "false"
}
xml{
        target => "data"
        source => "payload"
        remove_namespaces => "true"
        xpath => ["//RequestBody/externalTransactionID/text()",externalTransactionID]
                force_array => "false"
                store_xml => "false"
}
mutate {
    add_field => { "token" => "kNBDTnesOnWvUhhhwiVYbBZiqWZaQvYd" }
}
}
output
{
elasticsearch {
    hosts => [ "192.168.41.4:9200" ]
user => elastic
password => elastic
index => "logstash-*"
}
}

Hi @Debasish_Rath
Make sure that elasticsearch is reachable

Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)

Thanks for the replay. elastic search is configured in 192.168.41.4 ip, but it's calling local host. Can you please let me know where I need to change it.

How you are running logstash, could you provide the commande line used (logstash -f configfile ....) ? Your config output is using hosts => [ "192.168.41.4:9200" ] but logstash log saying that you are using New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}

I am running logstash and config file usinf below command:

bin/logstash -f a.conf

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.