Don't binding name index on Elasticsearch when using RabbitMQ

Elastic Stack Logstash
1

Hi,
I using Message queue RabbitMQ :
Client (Using Beat) ->Logstash (forwarder) -> RabbitMQ -> Logstash (Indexer) -> ES

I got errror when i config on Logstash Indexer, Out is ES. But when i see all my indices, new name indices : "%{[@metadata][beat]}-2015.12.25".
Before, using Client -> logstash(indexer) -> ES, so name of new indices are ""filebeat-2015-12-25" ...

Here is my config when using rabbitMQ (file config on Logstash Indexer)

input {
    rabbitmq {
        host => "10.1.11.177"
        queue => "logstash-queue"
        #durable => true
        key => "logstash-key"
        exchange => "logstash-rabbitmq"
        threads => 3
        prefetch_count => 50
        port => 5677
        user => "username"
        password => "password"
    }
}

output {
  elasticsearch {
    hosts => ["10.1.11.173:9200", "10.1.11.174:9200", "10.1.1.171:9200"]
    user => "username"
    password => "password"
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

And here is config on Logstash Forwarder

input {
  beats {
    port => 5044
  }
}
output {
    rabbitmq {
        exchange => "logstash-rabbitmq"
        exchange_type => "direct"
        key => "logstash-key"
        host => "10.1.11.177"
        vhost => "/"
        workers => 4
        durable => true
        persistent => true
        port => 5677
        user => "username"
        password => "password"

    }
}

What is wrong ?
Can you help me fix it.
Thank you!

2

Can you help me? :grin:

3

As far as I understand, your problem happens when you use RabbitMQ. Do you have the same problem if you feed the data from LS directly into ES? If not, I recommend to post this question in the LS forum as it seems to be Logstash related. If that is ok for you, I can also move the topic.

4

When i feed data from LS to ES., i dont see anything problem. Please help move topic to LS Forum.
And help me remove new my topic in LS forum.
Thanks you very much :slight_smile:

5

Can you help me ?
I try to reconfig logstash but got same error :expressionless:

6

Hi,

see this response for workaround. Problem is the '@metadata' field is not exported in output plugins.

1 Like