When outputting from logstash to elasticsearch, some data is missing

[sbyunnn (수빈 윤)]

Hi, I am building ELK Stack
The output of my logstash is Elasticsearch
but, some data is missing in ES

However, some data is missing from ES.
After removing all indexes, I can see missed data. but, the other data missing

I checked logstash's stdout, and nothing is missing

Why is the data missing?
help me T.T

There are a total of 6 nodes.
3 are master and 3 are data nodes.

I set logstash to 3 data nodes

my pipeline is

input {
  kafka {
    bootstrap_servers => "192.168.xxx.xxx:9092,192.168.xxx.xxx:9092,192.168.xxx.xxx:9092"
    group_id => "dev-all"
    topics => ["dev-all"]
    consumer_threads => 1
    decorate_events => true
    }
}

filter {
 json {
   source => "message"
 }
}

output{
  stdout {
    codec => rubydebug
  }
  elasticsearch {
    hosts => ["192.168.xxx.xxx:9200","192.168.xxx.xxx:9200","192.168.xxx.xxx:9200"]
    index => "dev-all-%{+YYYY.MM.dd}"
    user => "elastic"
    password => "${es_password}"
  }
}

below is logstash's stdout ( Omit some unnecessary data)

{
       "message" => "....",
          "time" => "2022-06-22T01:51:20.547705299Z",
      "@version" => "1",
    "kubernetes" => {
              ...
               "pod_name" => "ingress-nginx-controller-lwnsr",
         "namespace_name" => "ingress-nginx",
         "container_name" => "controller"
    },
        "stream" => "stdout",
    "@timestamp" => 2022-06-22T01:51:20.547Z,
           "log" => "..."
}		   
{
       "message" => "...",
          "time" => "2022-06-22T01:50:26.597591464Z",
      "@version" => "1",
    "kubernetes" => {
              ...
               "pod_name" => "logtest-58b9747db5-xcqbz",
         "namespace_name" => "logging-test",
         "container_name" => "count"
    },
        "stream" => "stdout",
    "@timestamp" => 2022-06-22T01:50:26.597Z,
           "log" => "..."
}

There are two data, "namespace_name" key is 'ingress-nginx' and 'logging-test' data.

I can see 'ingress-nginx' data

but I can not see 'logging-test' data

please help me..T.T

[sbyunnn (수빈 윤)]

When the UTC based date changes, as a new index is created, I checked that there is no 'ingress-nginx' data and the 'logging-test' data is visible.
Why? Let me know what I did wrong...

[sbyunnn (수빈 윤)]

I found the cause.
Because the index template was not specified, it tried to save in a different format.
So when the newly index create, data was missing.
I set different format of index template as 'flattened'
Thanks you