Logstash filebeat json payload problem

Hello everyone, I´m having trouble trying to parse simple json data from logstash to elasticsearch . My config is : 
filebeat suscribed to some mosquitto topic > logstash mqqt input > elasticsearch > kibana.
 @timestamp , mqtt topic and the message json payload structured in fields are my goals  .
While I do get information  I don´t see it well structured , i played with filebeat json processor , logstash input (json_lines) , logstash json filter , but got the best result with json codec . 
I'll post my config ,hoping someone can tell me what I'm missing . If you haven't noticed I´m really a newbie at ELK .

logstash config :


input {
	 beats {
        port => "5044"
		codec => "json"
    }
}

output {

		stdout { codec => rubydebug { metadata => true } }
		elasticsearch {
		hosts => ["http://localhost:9200"]
		index => "block"
		}
}


This is the output I´m getting , I see it organized , however ,back in kibana I don´t see the resulting fields . Do you guys  validate this output and maybe  I´m missing something in Elasticsearch???

{
           "input" => {
        "type" => "mqtt"
    },
           "agent" => {
            "hostname" => "DESKTOP-564O36T",
                "name" => "DESKTOP-564O36T",
                  "id" => "72b87c44-b538-4a3d-b195-f3f35824acf2",
                "type" => "filebeat",
        "ephemeral_id" => "5a1658fa-4bd4-4657-ba9d-4e2fba7357dd",
             "version" => "7.9.2"
    },
      "@timestamp" => 2021-02-23T17:22:16.663Z,
             "ecs" => {
        "version" => "1.5.0"
    },
       "@metadata" => {
        "ip_address" => "0:0:0:0:0:0:0:1",
              "beat" => "filebeat",
              "type" => "_doc",
           "version" => "7.9.2"
    },
            "mqtt" => {
          "retained" => false,
               "qos" => 0,
             "topic" => "Block/SalaN/Equipamiento",
        "message_id" => 0,
         "duplicate" => false
    },
        "@version" => "1",
            "host" => {
        "name" => "DESKTOP-564O36T"
    },
            "tags" => [
        [0] "beats_input_codec_json_applied"
    ],
    "Equipamiento" => {
        "Corriente" => 0.015769,
               "Id" => "12010-13402",
             "Unit" => "Amp"
    }
}

Thanks in any way you can help me .

How are you verifying the data in Kibana?

In Dev Tools run GET block/_search to see if you have any data.

Also did you create an index pattern? You can only see the data in Visualizations and Discover after you have done that.

Hi aaron , I gave up using filebeat because rabbitmq suits me better , configured the logstash plugin some filter and everything now works fine!!!

Thanks so much for your feedcabk.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.