LogStash.conf with multiple outputs: Data displayed on console, but not sent to ElasticSearch server

Elastic Stack Logstash
1

Here is my logstash.conf:

input {
	http {
		host => "127.0.0.1"
		port => 31311 
	}
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
  }
  stdout {
    codec => "rubydebug"
  }
}

As a test, I ran this command in PowerShell:

C:\Users\Me\Downloads\curl-7.64.1-win64-mingw\bin> .\curl.exe -XPUT "http://127.0.0.1:31311/customer?pretty"

On my Logstash console, as expected, I saw the following output:

{
    "@timestamp" => 2019-04-08T13:45:40.267Z,
       "headers" => {
              "http_host" => "127.0.0.1:31311",
        "http_user_agent" => "curl/7.64.1",
           "request_path" => "/customer?pretty",
         "content_length" => "0",
         "request_method" => "PUT",
           "http_version" => "HTTP/1.1",
            "http_accept" => "*/*"
    },
          "host" => "127.0.0.1",
      "@version" => "1",
       "message" => ""
}

I then ran the following command:

C:\Users\Me\Downloads\curl-7.64.1-win64-mingw\bin> .\curl.exe -XGET "http://127.0.0.1:9200/_cat/indices?v"

I expected to see that the customer index had also been created on my ES server; however, all I saw was

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

with an empty set of results.

What am I missing?

2

Hi @Miao,

if one output works, all outputs should be healthy, or more percisely, if one output is not healthy, Logstash will shutdown to prevent data loss... So I would expect the Elasticsearch output to be ok.

The index should be named logstash-%{+YYYY.MM.dd}.

I have not used the http input in Logstash so can't help you with that.

Everything looks fine to me. (not very helpful but still)

3

Thank you for your reply! I am confused: Why should the index be named logstash-%{+YYYY.MM.dd}, instead of customer, given that I ran the command C:\Users\Me\Downloads\curl-7.64.1-win64-mingw\bin> .\curl.exe -XPUT "http://127.0.0.1:31311/customer?pretty"?

4

I do not use http input with Logstash so I might very well be wrong... The documentation states

Using this input you can receive single or multiline events over http(s)

I would expect the Elasticsearch index to be logstash-%{+YYYY.MM.dd} as that is the Logstash default and the elasticsearchoutput config has no indexname configured.

The call you make to Logstash looks like something you would make to the Elasticsearch API.