# Kibana - Stack Monitoring - No structured logs found

**URL:** <https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213>\
**Category:** Kibana\
**Tags:** elastic-stack-monitoring\
**Created:** [July 15, 2020, 12:41am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213 "2020-07-15T00:41:47Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 15, 2020, 12:41am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/1 "2020-07-15T00:41:47Z")

</div>

Hi All,

We have an issue where Kibana throws a "No structured logs found" message in the Log section when viewing the Stack Monitoring page, where it should show the last 10 log entries.

I have Filebeat running on each of the ES nodes using the standard configuration (with updated var.paths to our log file locations) as documented here [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-elasticsearch.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-elasticsearch.html)

The logs are being successfuly written to an index with name filebeat-7.8.0-elasticsearch. Can anyone help to identify why Kibana does not display them when viewing the Stack Monitoring page?

Elasticsearch, Beats, Logstash, and Kibana all version 7.8.0.

![no_structured_logs](https://us1.discourse-cdn.com/elastic/original/3X/6/f/6f53fdef7536094c348e3293f074d4aece070e74.png)

Thanks

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 15, 2020, 1:16am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/2 "2020-07-15T01:16:11Z")

</div>

Can you share your Elasticsearch filebeat module yml file?

You’ll need to make sure you are reading the json logs

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 15, 2020, 2:07am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/3 "2020-07-15T02:07:22Z")

</div>

Sure thing.

```auto
# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.6/filebeat-module-elasticsearch.html

- module: elasticsearch
  # Server log
  server:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      #- /elk/elasticsearch/xyz/logs/xyz-cluster.log
      - /elk/elasticsearch/xyz/logs/xyz-cluster_server.json

  gc:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /elk/elasticsearch/xyz/logs/gc.log.[0-9]*
      - /elk/elasticsearch/xyz/logs/gc.log

  audit:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /elk/elasticsearch/xyz/logs/*_audit.json

  slowlog:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /elk/elasticsearch/xyz/logs/*_index_search_slowlog.json
      - /elk/elasticsearch/xyz/logs/*_index_indexing_slowlog.json

  deprecation:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /elk/elasticsearch/xyz/logs/*_deprecation.json

```

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 15, 2020, 5:25pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/4 "2020-07-15T17:25:28Z")

</div>

> [@jason\_0](#):
>
> ```auto
> gc:
> enabled: true
> # Set custom paths for the log files. If left empty,
> # Filebeat will choose the paths depending on your OS.
> var.paths:
> - /elk/elasticsearch/xyz/logs/gc.log.[0-9]*
> - /elk/elasticsearch/xyz/logs/gc.log
> 
> ```

Can you temporarily disable this and see if it fixes the problem?

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 16, 2020, 6:16am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/5 "2020-07-16T06:16:41Z")

</div>

I tried it with the below for 4+ hours. No difference.

```auto
- module: elasticsearch
  # Server log
  server:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /elk/elasticsearch/xyz/logs/*_server.json

  gc:
    enabled: false
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /elk/elasticsearch/xyz/logs/gc.log.[0-9]*
      - /elk/elasticsearch/xyz/logs/gc.log
...

```

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 16, 2020, 2:44pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/6 "2020-07-16T14:44:47Z")

</div>

Can you try running this query against the cluster that stores the filebeat data?

```auto
POST filebeat-*/_search?filter_path=hits.hits._source.event.dataset,hits.hits._source.@timestamp,hits.hits._source.elasticsearch
{
  "size": 10,
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ],
  "collapse": {
    "field": "event.dataset"
  }
}

```

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 16, 2020, 10:44pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/7 "2020-07-16T22:44:00Z")

</div>

Hey Chris,

Yep, this is the output

```auto
{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "event" : {
            "dataset" : "elasticsearch.server"
          },
          "@timestamp" : "2020-07-16T22:43:01.510Z"
        }
      },
      {
        "_source" : {
          "event" : {
            "dataset" : "system.syslog"
          },
          "@timestamp" : "2020-07-16T22:42:53.423Z"
        }
      },
      {
        "_source" : {
          "@timestamp" : "2020-07-16T22:42:53.107Z"
        }
      },
      {
        "_source" : {
          "@timestamp" : "2020-07-16T11:26:47.433Z",
          "event" : {
            "dataset" : "system.auth"
          }
        }
      },
      {
        "_source" : {
          "@timestamp" : "2020-07-16T03:57:37.601Z",
          "event" : {
            "dataset" : "elasticsearch.gc"
          }
        }
      },
      {
        "_source" : {
          "@timestamp" : "2020-07-09T13:01:41.489Z",
          "event" : {
            "dataset" : "elasticsearch.deprecation"
          }
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 20, 2020, 2:45pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/8 "2020-07-20T14:45:13Z")

</div>

Thanks.

It seems likely the gc collection is the problem here, as it does not have structured logs.

There are two things we can do about this:

1. File a ticket in Kibana to ensure that the logs continue to show in the Stack Monitoring UI, even if there are some indexed logs that are collected from non structured logs

2. File a ticket in Elasticsearch and ask if we can get the gc logs as `.json` documents too. I'm not sure why that doesn't exist now

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 21, 2020, 6:04am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/9 "2020-07-21T06:04:19Z")

</div>

Thanks Chris. How does the gc collection affect it though, as it still didn't work even after i disabled the gc logs for several hours?

I'm happy to file tickets, assuming you just mean issues on the git repo's?

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 24, 2020, 1:44pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/10 "2020-07-24T13:44:41Z")

</div>

That's a good point.

This is the exact query we make for that page:

```auto
POST filebeat-*/_search
{
  "size": 0,
  "sort": {
    "@timestamp": {
      "order": "desc"
    }
  },
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "service.type": "elasticsearch"
          }
        },
        {
          "range": {
            "@timestamp": {
              "format": "epoch_millis",
              "gte": 1595594404844,
              "lte": 1595598004844
            }
          }
        },
        {
          "term": {
            "elasticsearch.cluster.uuid": "{cluster_uuid}"
          }
        }
      ]
    }
  },
  "aggs": {
    "types": {
      "terms": {
        "field": "event.dataset"
      },
      "aggs": {
        "levels": {
          "terms": {
            "field": "log.level"
          }
        }
      }
    }
  }
}

```

You can adjust the time period to be relative time to test too:

```auto
"range": {
  "@timestamp": {
    "gte": "now-1h",
    "lte": "now"
  }
}

```

Run that for the specified period in your time picker and lemme know what it returns

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 27, 2020, 6:27am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/11 "2020-07-27T06:27:44Z")

</div>

Hey Chris, so it gives back:

```auto
{
  "took" : 126,
  "timed_out" : false,
  "_shards" : {
    "total" : 45,
    "successful" : 45,
    "skipped" : 26,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : []
  },
  "aggregations" : {
    "types" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : []
    }
  }
}

```

Looking over the index in kibana none of the logs are populating the elasticsearch.cluster.uuid field.

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 27, 2020, 6:39am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/12 "2020-07-27T06:39:24Z")

</div>

It looks like the server json logs all use **cluster.uuid** and looking over the filebeat elasticsearch module in github it seems to expect the field to be **elasticsearch.server.cluster.uuid** which gets renamed to **elasticsearch.cluster.uuid**

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 27, 2020, 6:47pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/13 "2020-07-27T18:47:21Z")

</div>

Can you share your mappings and at least one filebeat pipeline?

`GET filebeat-*/_mapping` - You can just return the `properties.elasticsearch` json blob here  
`GET _ingest/pipeline/filebeat-8.0.0-elasticsearch-server-pipeline`

Then let's see some full documents:

```auto
POST filebeat-*/_search
{
  "size": 20,
  "sort": {
    "@timestamp": {
      "order": "desc"
    }
  },
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "service.type": "elasticsearch"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "now-1h",
              "lte": "now"
            }
          }
        }
      ]
    }
  },
  "collapse": {
    "field": "event.dataset"
  }
}

```

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 28, 2020, 12:56am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/14 "2020-07-28T00:56:54Z")

</div>

GET filebeat-\*/\_mapping

```auto
"elasticsearch" : {
          "properties" : {
            "audit" : {
              "properties" : {
                "action" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "event_type" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "indices" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "layer" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "message" : {
                  "type" : "text",
                  "norms" : false
                },
                "origin" : {
                  "properties" : {
                    "type" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    }
                  }
                },
                "realm" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "request" : {
                  "properties" : {
                    "id" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    },
                    "name" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    }
                  }
                },
                "url" : {
                  "properties" : {
                    "params" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    }
                  }
                },
                "user" : {
                  "properties" : {
                    "realm" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    },
                    "roles" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    }
                  }
                }
              }
            },
            "cluster" : {
              "properties" : {
                "name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "uuid" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                }
              }
            },
            "component" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            "deprecation" : {
              "type" : "object"
            },
            "gc" : {
              "properties" : {
                "heap" : {
                  "properties" : {
                    "size_kb" : {
                      "type" : "long"
                    },
                    "used_kb" : {
                      "type" : "long"
                    }
                  }
                },
                "jvm_runtime_sec" : {
                  "type" : "float"
                },
                "old_gen" : {
                  "properties" : {
                    "size_kb" : {
                      "type" : "long"
                    },
                    "used_kb" : {
                      "type" : "long"
                    }
                  }
                },
                "phase" : {
                  "properties" : {
                    "class_unload_time_sec" : {
                      "type" : "float"
                    },
                    "cpu_time" : {
                      "properties" : {
                        "real_sec" : {
                          "type" : "float"
                        },
                        "sys_sec" : {
                          "type" : "float"
                        },
                        "user_sec" : {
                          "type" : "float"
                        }
                      }
                    },
                    "duration_sec" : {
                      "type" : "float"
                    },
                    "name" : {
                      "type" : "keyword",
                      "ignore_above" : 1024
                    },
                    "parallel_rescan_time_sec" : {
                      "type" : "float"
                    },
                    "scrub_string_table_time_sec" : {
                      "type" : "float"
                    },
                    "scrub_symbol_table_time_sec" : {
                      "type" : "float"
                    },
                    "weak_refs_processing_time_sec" : {
                      "type" : "float"
                    }
                  }
                },
                "stopping_threads_time_sec" : {
                  "type" : "float"
                },
                "tags" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "threads_total_stop_time_sec" : {
                  "type" : "float"
                },
                "young_gen" : {
                  "properties" : {
                    "size_kb" : {
                      "type" : "long"
                    },
                    "used_kb" : {
                      "type" : "long"
                    }
                  }
                }
              }
            },
            "index" : {
              "properties" : {
                "id" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                }
              }
            },
            "node" : {
              "properties" : {
                "id" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                }
              }
            },
            "server" : {
              "properties" : {
                "gc" : {
                  "properties" : {
                    "collection_duration" : {
                      "properties" : {
                        "ms" : {
                          "type" : "float"
                        }
                      }
                    },
                    "observation_duration" : {
                      "properties" : {
                        "ms" : {
                          "type" : "float"
                        }
                      }
                    },
                    "overhead_seq" : {
                      "type" : "long"
                    },
                    "young" : {
                      "properties" : {
                        "one" : {
                          "type" : "long"
                        },
                        "two" : {
                          "type" : "long"
                        }
                      }
                    }
                  }
                },
                "stacktrace" : {
                  "type" : "keyword",
                  "index" : false,
                  "ignore_above" : 1024
                }
              }
            },
            "shard" : {
              "properties" : {
                "id" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                }
              }
            },
            "slowlog" : {
              "properties" : {
                "extra_source" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "id" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "logger" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "routing" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "search_type" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "source" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "source_query" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "stats" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "took" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "total_hits" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "total_shards" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "type" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "types" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                }
              }
            }
          }
        },

```

GET \_ingest/pipeline/filebeat-8.0.0-elasticsearch-server-pipeline returns a 404

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 28, 2020, 12:57am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/15 "2020-07-28T00:57:04Z")

</div>

Search results (note i cleaned up the log content by deleting the majority of the stack trace

```auto
{
  "took" : 4544,
  "timed_out" : false,
  "_shards" : {
    "total" : 48,
    "successful" : 45,
    "skipped" : 24,
    "failed" : 3,
    "failures" : [
      {
        "shard" : 0,
        "index" : "filebeat-7.8.0",
        "node" : "njKm_18uRcugxs6ZVODAtA",
        "reason" : {
          "type" : "illegal_argument_exception",
          "reason" : "unknown type for collapse field `event.dataset`, only keywords and numbers are accepted"
        }
      }
    ]
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "filebeat-7.8.0-elasticsearch-000008",
        "_type" : "_doc",
        "_id" : "0OXlknMBPeMNtYkwpVEh",
        "_score" : null,
        "_source" : {
          "service" : {
            "type" : "elasticsearch"
          },
          "ls_node" : "xyz204",
          "log" : {
            "file" : {
              "path" : "/elk/elasticsearch/xyz/logs/xyz-cluster_server.json"
            },
            "flags" : [
              "multiline"
            ],
            "offset" : 61583874,
            "original" : """{"type": "server", "timestamp": "2020-07-28T10:41:40,398+10:00", "level": "INFO", "component": "o.e.a.b.TransportShardBulkAction", "cluster.name": "xyz-cluster", "node.name": "xyz202", "message": "[testxyz-000002][1] mapping update rejected by primary", "cluster.uuid": "ZZKjFPMkQIqdSHpz1H-oeQ", "node.id": "X2mCgm_WTP2iyZbS2bquVg" , 
"stacktrace": ["java.lang.IllegalArgumentException: Limit of total fields [4000] in index }"""
          },
          "@version" : "1",
          "fileset" : {
            "name" : "server"
          },
          "@timestamp" : "2020-07-28T00:41:40.572Z",
          "agent" : {
            "version" : "7.8.0",
            "hostname" : "xyz202.xyz.xyz.xyz",
            "id" : "f4f68361-80e4-4257-b08b-68490facc16a",
            "name" : "xyz202.xyz.xyz.xyz",
            "ephemeral_id" : "b8f62b1a-3d86-40e1-90b5-fa57814efc2d",
            "type" : "filebeat"
          },
          "input" : {
            "type" : "log"
          },
          "host" : {
            "containerized" : false,
            "ip" : [
              "10.",
              "10.",
              "fe80:"
            ],
            "hostname" : "xyz202.xyz.xyz.xyz",
            "id" : "d83f39e122e947898a4e895eb130f93b",
            "name" : "xyz202.xyz.xyz.xyz",
            "mac" : [
              "24:6e:9
            ],
            "os" : {
              "family" : "redhat",
              "version" : "7.8 (Maipo)",
              "kernel" : "3.10.0-1127.el7.x86_64",
              "codename" : "Maipo",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel"
            },
            "architecture" : "x86_64"
          },
          "event" : {
            "module" : "elasticsearch",
            "dataset" : "elasticsearch.server"
          },
          "tags" : [
            "elasticsearch-host",
            "beats_input_codec_plain_applied",
            "logfilename_grok_failure"
          ],
          "message" : """{"type": "server", "timestamp": "2020-07-28T10:41:40,398+10:00", "level": "INFO", "component": "o.e.a.b.TransportShardBulkAction", "cluster.name": "xyz-cluster", "node.name": "xyz202", "message": "[testxyz-000002][1] mapping update rejected by primary", "cluster.uuid": "ZZKjFPMkQIqdSHpz1H-oeQ", "node.id": "X2mCgm_WTP2iyZbS2bquVg" , 
"stacktrace": ["java.lang.IllegalArgumentException: Limit of total fields [4000] in in] }""",
          "ecs" : {
            "version" : "1.5.0"
          }
        },
        "fields" : {
          "event.dataset" : [
            "elasticsearch.server"
          ]
        },
        "sort" : [
          1595896900572
        ]
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 28, 2020, 7:23pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/16 "2020-07-28T19:23:57Z")

</div>

> [@jason\_0](#):
>
> GET \_ingest/pipeline/filebeat-8.0.0-elasticsearch-server-pipeline returns a 404

My bad. Replace `8.0.0` with the `7.8.0`. You can double check the names through `GET _ingest/pipeline`

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [July 28, 2020, 10:58pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/17 "2020-07-28T22:58:14Z")

</div>

Still a 404. This is GET \_ingest/pipeline

```auto
{
  "xpack_monitoring_6" : {
    "description" : "This pipeline upgrades documents from the older version of the Monitoring API to the newer version (7) by fixing breaking changes in those older documents before they are indexed from the older version (6).",
    "version" : 7000199,
    "processors" : [
      {
        "script" : {
          "source" : "ctx._type = null"
        }
      },
      {
        "gsub" : {
          "field" : "_index",
          "pattern" : """(.monitoring-\w+-)6(-.+)""",
          "replacement" : "$17$2"
        }
      }
    ]
  },
  "xpack_monitoring_7" : {
    "description" : "This is a placeholder pipeline for Monitoring API version 7 so that future versions may fix breaking changes.",
    "version" : 7000199,
    "processors" : []
  }
}

```

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [July 29, 2020, 2:48pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/18 "2020-07-29T14:48:54Z")

</div>

It looks like those pipelines don't exist for some reason.

See [https://www.elastic.co/guide/en/beats/filebeat/master//load-ingest-pipelines.html](https://www.elastic.co/guide/en/beats/filebeat/master//load-ingest-pipelines.html) for helping getting those installed.

---

<div class="post-metadata">

**Author:** ![jason\_0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_0/32/66801_2.png) [@jason\_0](https://discuss.elastic.co/u/jason_0)\
**Post date:** [August 7, 2020, 3:56am UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/19 "2020-08-07T03:56:09Z")

</div>

Hey Chris, I've loaded the ingest pipeline for elasticsearch now, but that hasn't fixed it still it seems.

```auto
{
  "filebeat-7.8.0-elasticsearch-server-pipeline" : {
    "description" : "Pipeline for parsing elasticsearch server logs",
    "processors" : [
      {
        "rename" : {
          "field" : "@timestamp",
          "target_field" : "event.created"
        }
      },
      {
        "grok" : {
          "field" : "message",
          "patterns" : [
            "^%{CHAR:first_char}"
          ],
          "pattern_definitions" : {
            "CHAR" : "."
          }
        }
      },
      {
        "pipeline" : {
          "name" : "filebeat-7.8.0-elasticsearch-server-pipeline-plaintext",
          "if" : "ctx.first_char != '{'"
        }
      },
      {
        "pipeline" : {
          "if" : "ctx.first_char == '{'",
          "name" : "filebeat-7.8.0-elasticsearch-server-pipeline-json"
        }
      },
      {
        "script" : {
          "lang" : "painless",
          "source" : """if (ctx.elasticsearch.server.gc != null && ctx.elasticsearch.server.gc.observation_duration != null) {
  if (ctx.elasticsearch.server.gc.observation_duration.unit == params.seconds_unit) {
    ctx.elasticsearch.server.gc.observation_duration.ms = ctx.elasticsearch.server.gc.observation_duration.time * params.ms_in_one_s;
  }
  if (ctx.elasticsearch.server.gc.observation_duration.unit == params.milliseconds_unit) {
    ctx.elasticsearch.server.gc.observation_duration.ms = ctx.elasticsearch.server.gc.observation_duration.time;
  }
  if (ctx.elasticsearch.server.gc.observation_duration.unit == params.minutes_unit) {
    ctx.elasticsearch.server.gc.observation_duration.ms = ctx.elasticsearch.server.gc.observation_duration.time * params.ms_in_one_m;
  }
} if (ctx.elasticsearch.server.gc != null && ctx.elasticsearch.server.gc.collection_duration != null) {
  if (ctx.elasticsearch.server.gc.collection_duration.unit == params.seconds_unit) {
    ctx.elasticsearch.server.gc.collection_duration.ms = ctx.elasticsearch.server.gc.collection_duration.time * params.ms_in_one_s;
  }
  if (ctx.elasticsearch.server.gc.collection_duration.unit == params.milliseconds_unit) {
    ctx.elasticsearch.server.gc.collection_duration.ms = ctx.elasticsearch.server.gc.collection_duration.time;
  }
  if (ctx.elasticsearch.server.gc.collection_duration.unit == params.minutes_unit) {
    ctx.elasticsearch.server.gc.collection_duration.ms = ctx.elasticsearch.server.gc.collection_duration.time * params.ms_in_one_m;
  }
}""",
          "params" : {
            "milliseconds_unit" : "ms",
            "ms_in_one_s" : 1000,
            "ms_in_one_m" : 60000,
            "minutes_unit" : "m",
            "seconds_unit" : "s"
          }
        }
      },
      {
        "set" : {
          "field" : "event.kind",
          "value" : "event"
        }
      },
      {
        "set" : {
          "value" : "database",
          "field" : "event.category"
        }
      },
      {
        "script" : {
          "lang" : "painless",
          "source" : """def errorLevels = ['FATAL', 'ERROR']; if (ctx?.log?.level != null) {
  if (errorLevels.contains(ctx.log.level)) {
    ctx.event.type = 'error';
  } else {
    ctx.event.type = 'info';
  }
}"""
        }
      },
      {
        "set" : {
          "field" : "host.name",
          "value" : "{{elasticsearch.node.name}}",
          "if" : "ctx?.elasticsearch?.node?.name != null"
        }
      },
      {
        "set" : {
          "field" : "host.id",
          "value" : "{{elasticsearch.node.id}}",
          "if" : "ctx?.elasticsearch?.node?.id != null"
        }
      },
      {
        "remove" : {
          "field" : [
            "elasticsearch.server.gc.collection_duration.time",
            "elasticsearch.server.gc.collection_duration.unit",
            "elasticsearch.server.gc.observation_duration.time",
            "elasticsearch.server.gc.observation_duration.unit"
          ],
          "ignore_missing" : true
        }
      },
      {
        "remove" : {
          "field" : [
            "elasticsearch.server.timestamp",
            "elasticsearch.server.@timestamp"
          ],
          "ignore_missing" : true
        }
      },
      {
        "remove" : {
          "field" : [
            "first_char"
          ]
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "field" : "error.message",
          "value" : "{{ _ingest.on_failure_message }}"
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [August 10, 2020, 1:43pm UTC](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213/20 "2020-08-10T13:43:37Z")

</div>

Does this exist?

`GET _ingest/pipeline/filebeat-7.8.0-elasticsearch-server-pipeline-json`

[Next page](https://discuss.elastic.co/t/kibana-stack-monitoring-no-structured-logs-found/241213.md?page=2)