Filebeat to logstash seems OK. Logstash to Elasticsearch seems to be having authentication issues

Elastic Stack Logstash
1

Hi All,
Once again, please be kind. im 2 days into this and still getting round the learning curve.

ES 5.4.2 installed from tar.gz
filebeat-5.4.2-1.x86_64 Installed from RPM
logstash-5.4.3-1.noarch Installed from RPM

I have been trying to configure Filebeat -> Logstash -> Elasticsearch using this URL

https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html

Here is my logstash first-pipline,conf file

input {
    beats {
        port => "5043"
    }
}
 filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
    stdout { codec => rubydebug }
}

When i execute this against the tutorial data logstash fires up and I get a nice bit of json output in the rubydebug

{
        "request" => "/style2.css",
          "agent" => "\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36\"",
          "geoip" => {
              "timezone" => "America/Denver",
                    "ip" => "71.212.224.97",
              "latitude" => 39.8663,
        "continent_code" => "NA",
             "city_name" => "Denver",
          "country_name" => "United States",
         "country_code2" => "US",
              "dma_code" => 751,
         "country_code3" => "US",
           "region_name" => "Colorado",
              "location" => {
            "lon" => -105.0061,
            "lat" => 39.8663
        },
           "postal_code" => "80260",
           "region_code" => "CO",
             "longitude" => -105.0061
    },
         "offset" => 20664,
           "auth" => "-",
          "ident" => "-",
     "input_type" => "log",
           "verb" => "GET",
         "source" => "/tmp/tutorial/logstash-tutorial.log",
        "message" => "71.212.224.97 - - [04/Jan/2015:05:27:35 +0000] \"GET /style2.css HTTP/1.1\" 200 4877 \"http://www.semicomplete.com/projects/xdotool/\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.36 (KHTML, li  ke Gecko) Chrome/32.0.1700.107 Safari/537.36\"",
           "type" => "log",
           "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
       "referrer" => "\"http://www.semicomplete.com/projects/xdotool/\"",
     "@timestamp" => 2017-06-27T19:59:20.797Z,
       "response" => "200",
          "bytes" => "4877",
       "clientip" => "71.212.224.97",
       "@version" => "1",
           "beat" => {
        "hostname" => "bravo",
            "name" => "bravo",
         "version" => "5.4.2"
    },
           "host" => "bravo",
    "httpversion" => "1.1",
      "timestamp" => "04/Jan/2015:05:27:35 +0000"
}

My problem is when I move it across to output to elastic search and then try and query the data I get the following error message.

[log@bravo config]# curl --user USERNAME:PASSWORD -XGET 'localhost:9200/logstash-2017-06-27/_search?pretty&q=response=200'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index",
        "resource.type" : "index_or_alias",
        "resource.id" : "logstash-2017-06-27",
        "index_uuid" : "_na_",
        "index" : "logstash-2017-06-27"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index",
    "resource.type" : "index_or_alias",
    "resource.id" : "logstash-2017-06-27",
    "index_uuid" : "_na_",
    "index" : "logstash-2017-06-27"
  },
  "status" : 404
}

I read the following page but this did not seem to work for me either
https://www.elastic.co/guide/en/x-pack/current/logstash.html

I think its an authentication problem but I just cant nail it down.

2

Have you looked in the Logstash log? If it's having problems sending data to ES it'll tell you about it.

3

Hi Magnus,

Looking in /var/log/logstash-stderr.log i get the following

21:16:00.310 [Ruby-0-Thread-6: /opt/apps/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:224] INFO  logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
21:16:00.315 [Ruby-0-Thread-6: /opt/apps/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:224] WARN  logstash.outputs.elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x159cae14 URL:http://localhost:9200/>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://localhost:9200/'"}

When I look in Kibana the index is also not created

4

Yeah, that looks like an authentication problem.