# Internal metric collection ends up in "Standalone Cluster" regardless cluster\_uuid setting **URL:** \ **Category:** Beats\ **Tags:** elastic-stack-monitoring\ **Created:** [July 8, 2020, 10:22am UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320 "2020-07-08T10:22:08Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 8, 2020, 10:22am UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/1 "2020-07-08T10:22:09Z")
Hi, ES, APM version: 7.2.1 I've seen this before with Filebeat but it wasn't important at the time but now that I added an APM server to the cluster it's frustrating. ES, Kibana, and Logstash are correctly grouped so I figured this will be a beats issue. I've set up monitoring in `apm-server.yml`: ``` apm-server: ... output.kafka: ... logging: ... monitoring: enabled: true cluster_uuid: "PR0d_CluSteR_Uu1d" elasticsearch: hosts: ["host1:9200", "host2:9200"] username: "custom_apm_user" password: "secure_password" ``` However, in Kibana --\> Stack Monitoring, I get a new "Standalone Cluster" entry: ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/a/dab7b99ed8c74633f12801709e9a92ff8abbbc73.png) When I checked the `.monitoring-beats-7-2020.07.08` index I saw this: ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/8/a84e2ef8bc1bafaae02d59c6eaa78956b89181fc.png) It seems that it doesn't pass the information at all ☹ Is there any workaround without adding any intermediate app (e.g., Logstash)? Thank you! ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [July 9, 2020, 2:25pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/2 "2020-07-09T14:25:33Z")
Could you please share your Filebeat configuration formatted using `` and maybe debug logs? ---
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 9, 2020, 5:53pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/3 "2020-07-09T17:53:19Z")
Hello Noémi, Filebeat isn't a part of this equation. APM server sends internal monitoring data directly. If you're referring to this: > I've seen this before with Filebeat but it wasn't important at the time That was almost a year ago for a development cluster (same version though 7.2.X) and I don't have it anymore. I can't lower the logging level from info to avoid flooding the logs (ES+APM). If there's something specific that may help out I'll look into it. This is the full configuration in `apm-server.yml`: ``` apm-server: host: ":8200" output.kafka: hosts: [...] topic: "..." partition.round_robin: reachable_only: true ssl.certificate_authorities: ["..."] username: "..." password: "..." required_acks: -1 compression: snappy max_message_bytes: 1000000 logging.level: info logging.metrics.enabled: false logging.to_files: true logging.to_syslog: false logging.files: path: /var/log/apm-server name: apm-server keepfiles: 7 monitoring: enabled: true cluster_uuid: "..." elasticsearch: hosts: ["..."] username: "..." password: "..." ssl.certificate_authorities: ["..."] ssl.certificate: "..." ssl.key: "..." ``` ---
**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\ **Post date:** [July 9, 2020, 5:55pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/4 "2020-07-09T17:55:47Z")
Hmm. I don't think `monitoring.cluster_uuid` is available in `7.2.x`. Per [https://github.com/elastic/beats/pull/13182](https://github.com/elastic/beats/pull/13182), it is first available in `7.3.2`. ---
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 9, 2020, 6:01pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/5 "2020-07-09T18:01:18Z")
Aww.... Thank you @chrisronline! Is there a way to avoid the issue above with the current setup or I need to wait until the cluster will be upgraded? ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [July 9, 2020, 7:41pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/6 "2020-07-09T19:41:43Z")
I have not tested this but you could create a custom [Ingest pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html) on your monitoring Elasticsearch cluster. This pipeline would use the [`set` processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/set-processor.html) to set the `cluster_uuid` to `PR0d_CluSteR_Uu1d`. Then you'd reference this pipeline name in the `output.elasticsearch.pipeline` setting in your `apm-server.yml`. There are a couple of options by which you can load your custom Ingest pipeline into Elasticsearch; you can read about them [here](https://www.elastic.co/guide/en/apm/server/master/configuring-ingest-node.html#custom-pipelines). Hope that helps, Shaunak ---
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 9, 2020, 9:19pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/7 "2020-07-09T21:19:55Z")
I'm sorry if I'm mistaken, but according to the [monitor configuration](https://www.elastic.co/guide/en/apm/server/current/configuration-monitor.html), there's no `output.elasticsearch.pipeline` option. I send the transactions to Kafka and I don't intend to use [ES as the ingest node](https://www.elastic.co/guide/en/apm/server/current/elasticsearch-output.html#pipeline-option-es) directly. ---
**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\ **Post date:** [July 10, 2020, 1:18pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/8 "2020-07-10T13:18:44Z")
The ES monitoring cluster needs to have at least one ingest node, as the monitoring ES plugin creates a default ingest pipeline. See [https://www.elastic.co/guide/en/elasticsearch/reference/current/collecting-monitoring-data.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/collecting-monitoring-data.html) You can create an ingest pipeline on this cluster that can format the documents before indexing. It's not something you configure within Beats, but rather something you configure on the ES monitoring cluster directly. ---
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 10, 2020, 1:48pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/9 "2020-07-10T13:48:47Z")
I'm sorry, but how is that relevant to my issue? ---
**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\ **Post date:** [July 10, 2020, 4:52pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/10 "2020-07-10T16:52:45Z")
The problem here is your`.monitoring-beats-*` indices do not contain a valid `cluster_uuid` field because your output is not Elasticsearch (which is a completely valid setup). We fixed this bug in `7.3.2` and beyond but you aren't able to upgrade to this version to get the fix. The fix involves manually setting the `cluster_uuid` within the beat yml file. As a way to fix the issue without needing to upgrade, you can use an ingest pipeline on the monitoring cluster to manually add the proper `cluster_uuid` to the `.monitoring-beats-*` documents as they are indexed. Ingest pipelines allow you to perform various actions before a document in indexed. It does mean you will need to hard-code the proper `cluster_uuid` in the pipeline itself but it should be a short-term fix that you can safely remove once you are able to upgrade to a version of the stack with a fix. Does that help? ---
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 13, 2020, 10:54am UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/11 "2020-07-13T10:54:05Z")
Thanks @chrisronline ! I understand that now. I was confused by `output.elasticsearch.pipeline` from @shaunak. I didn't use ES pipelines myself and couldn't find any information on how to attach the `.monitoring-beats-*` indices to a pipeline I create. The best I [got](https://www.elastic.co/guide/en/apm/server/master/configuring-ingest-node.html#apply-pipelines) (using the links from Shaunak) is still pointing to the output section in the _apm-server.yml_. 🤷‍♂️ ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [July 13, 2020, 3:45pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/12 "2020-07-13T15:45:49Z")
Sorry @YvorL, I said `output.elasticsearch.pipeline` 🤦‍♂️ but I was thinking `monitoring.elasticsearch.pipeline`. There are many settings under `output.elasticsearch.*` that are also available under `monitoring.elasticsearch.*`, however `pipeliene` is unfortunately not one of them. So setting `monitoring.elasticsearch.pipeline` would not work either 🤦‍♂️ 🤦‍♂️. I'm thinking of another workaround. ---
**Author:** ![YvorL](https://avatars.discourse-cdn.com/v4/letter/y/9fc348/32.png) [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Post date:** [July 14, 2020, 1:17pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/13 "2020-07-14T13:17:32Z")
I appreciate it but not sure if it's worth it 😃 It's a nuisance for sure but the data ends up in Kibana and graphs are available. I'll upgrade the cluster in a couple of months and this will go away. Thank you for your inputs! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 11, 2020, 3:17pm UTC](https://discuss.elastic.co/t/internal-metric-collection-ends-up-in-standalone-cluster-regardless-cluster-uuid-setting/240320/14 "2020-08-11T15:17:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.