Logstash output to file but not right hostname

Hi everyone, I'm a newer of using logstash, here is my case:

  • In right case:
    log nginx (hostname: staging) --> filebeat ---> logstash ---> output to file: /opt/logs/staging/2016-09-27.log ( right folder "staging" as I want).
  • But using redis as broker:
    log nginx (hostname: staging) --> filebeat --->redis---> logstash ---> output to file: /opt/logs/%{host}/2016-09-27.log ( it create folder %{host} instead of "staging").
    So what's problem?
    Here's my logstash config
    /etc/logstash/conf.d/02-beats-input.conf:
    input { redis { host => "10.84.87.148" port => "6379" type => "redis-input" data_type => "list" key => "filebeat" } }

/etc/logstash/conf.d/14-file-filter.conf:

filter { if [type] == "nginx-access" { grok { match => { "message" => "%{NGINXACCESS}" } } } }

/etc/logstash/conf.d/30-elasticsearch-output.conf:
output { elasticsearch { hosts => ["localhost:9200"] } stdout { codec => rubydebug } file { codec => line { format => "%{message}" } path => "/opt/logs/%{host}/%{+YYYY-MM-dd}.log" } }

And the output of logstash.stdout:
{ "@timestamp" => "2016-10-12T03:15:27.323Z", "beat" => { "hostname" => "staging", "name" => "staging" }, "count" => 1, "fields" => nil, "input_type" => "log", "message" => "10.84.10.186 - - [12/Oct/2016:10:15:26 +0700] \"GET /bigbuckbunny_1100.mp4/seg-9-v1-a1.ts HTTP/1.1\" 200 788472 \"http://osmfhls.kutu.ru/\" \"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\"", "offset" => 65958, "source" => "/opt/nginx/logs/hls_access.log", "type" => "nginx-access", "@version" => "1", "clientip" => [ [0] "10.84.10.186", [1] "10.84.10.186", [2] "10.84.10.186" ], "ident" => [ [0] "-", [1] "-", [2] "-" ], "auth" => [ [0] "-", [1] "-", [2] "-" ], "timestamp" => [ [0] "12/Oct/2016:10:15:26 +0700", [1] "12/Oct/2016:10:15:26 +0700", [2] "12/Oct/2016:10:15:26 +0700" ], "verb" => [ [0] "GET", [1] "GET", [2] "GET" ], "request" => [ [0] "/bigbuckbunny_1100.mp4/seg-9-v1-a1.ts", [1] "/bigbuckbunny_1100.mp4/seg-9-v1-a1.ts", [2] "/bigbuckbunny_1100.mp4/seg-9-v1-a1.ts" ], "httpversion" => [ [0] "1.1", [1] "1.1", [2] "1.1" ], "response" => [ [0] "200",

Your events clearly don't have a host field so %{host} won't work. Use %{[beat][hostname]} instead.

1 Like

oh, It's work. I understand the problem now. Really appreciate for your quick responce :smiley: