# Elastic Agent and index names **URL:** \ **Category:** Elastic Agent\ **Created:** [January 12, 2024, 7:49pm UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985 "2024-01-12T19:49:38Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![wrender1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wrender1/32/130685_2.png) [@wrender1](https://discuss.elastic.co/u/wrender1)\ **Post date:** [January 12, 2024, 7:49pm UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/1 "2024-01-12T19:49:38Z")
I'm deploying the Elastic Agent in standalone on Kubernetes. I've got he default yaml file, but I'm having a hard time with the index naming that it creates. It is a little unclear to me from the documentation. Do the index names for Elastic Agent logs and metrics always need to start with logs- or metrics- ? I would like to override this if possible. For example in my test the only way I was able to change the data stream names was at the end of the name like this: logs-kubernetes.container\_logs-my-custom-namespace metrics-kubernetes.pod-my-custom-namespace What I would like is for them to have index names like this: myteam-logs-kubernetes.container\_logs myteam-metrics-kubernetes.pod Here are some small sections of my related elastic agent settings: ```auto - id: container-log-${kubernetes.pod.name}-${kubernetes.container.id} type: filestream use_output: default meta: package: name: kubernetes version: 1.52.0 data_stream: namespace: my-custom-namespace streams: # Stream ID for this data stream allowing Filebeat to track the state of the ingested files. Must be unique. # Each filestream data stream creates a separate instance of the Filebeat filestream input. - id: container-log-${kubernetes.pod.name}-${kubernetes.container.id} data_stream: dataset: kubernetes.container_logs type: logs inputs: - id: kubernetes-cluster-metrics condition: ${kubernetes_leaderelection.leader} == true type: kubernetes/metrics use_output: default meta: package: name: kubernetes version: 1.52.0 data_stream: namespace: my-custom-namespace streams: - data_stream: dataset: kubernetes.apiserver type: metrics ``` If I try changing something like type: metrics to type: myteam-metrics I start getting 400 errors in the elastic agent and nothing is indexed. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [January 12, 2024, 9:27pm UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/2 "2024-01-12T21:27:30Z")
> [@wrender1](#): > > Do the index names for Elastic Agent logs and metrics always need to start with logs- or metrics- ? Yes, the data stream naming scheme for the elastic agent is the [following](https://www.elastic.co/guide/en/fleet/current/data-streams.html#data-streams-naming-scheme): ```auto -- ``` the _type_ can be `logs`, `metrics`, `traces` or `synthetics`, it will depending o what is being collected, the integration. the _dataset_ is based on the integration and the _namespace_ is the only part that can be customized. ---
**Author:** ![yago82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yago82/32/97755_2.png) [@yago82](https://discuss.elastic.co/u/yago82)\ **Post date:** [January 12, 2024, 9:44pm UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/3 "2024-01-12T21:44:19Z")
Hi, in your case, you're trying to change the `` part of the index name, which is not allowed because it's a fundamental part of how data streams work. However, you can customize the `` part of the index name to achieve a similar effect. For example, you could set the namespace to `myteam-logs` or `myteam-metrics`. This would result in index names like `logs-kubernetes.container_logs-myteam-logs` and `metrics-kubernetes.pod-myteam-metrics`. Regards ---
**Author:** ![wrender1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wrender1/32/130685_2.png) [@wrender1](https://discuss.elastic.co/u/wrender1)\ **Post date:** [January 13, 2024, 12:27am UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/4 "2024-01-13T00:27:23Z")
Ok. Thanks for confirming. I’ll make do with just changing the namespace. What about using our own index template for this? Would it be correct to clone the logs and metrics index template and then set a higher priority on it? So we can then use things like our own index lifecycle policies or customize fields for extra stuff? On a side note it would be nice if there was an official helm chart for elastic agent to configure in standalone. I can’t seem to find anything for it. Thanks! ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [January 13, 2024, 1:51am UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/5 "2024-01-13T01:51:14Z")
> [@wrender1](#): > > What about using our own index template for this? Would it be correct to clone the logs and metrics index template and then set a higher priority on it? So we can then use things like our own index lifecycle policies or customize fields for extra stuff? You can create custom templates to add extra fields and use a different log retention, but depending on the number of integrations this adds a lot of work because this needs to be done in the dataset level, and some integrations have multiple datasets. For example, the Google Workspace has something around 13 different datasets, if you wanted to add a field named `custom.field` for example, you would need to clone and manage 13 different custom templates, the same thing applies to custom ingest pipelines for example, but at least for ingest pipelines this is going to be easier on version 8.12 if I'm not wrong. At the moment any customization in mappings and lifecycle policies requires a lot of work. There is [work](https://github.com/elastic/kibana/issues/149484) being done to improve this, but no ETA when this will be available. > [@wrender1](#): > > On a side note it would be nice if there was an official helm chart for elastic agent to configure in standalone. I can’t seem to find anything for it. Yeah, I don't think you will find anything official by Elastic as they do not publish Helm Charts anymore. ---
**Author:** ![wrender1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wrender1/32/130685_2.png) [@wrender1](https://discuss.elastic.co/u/wrender1)\ **Post date:** [January 13, 2024, 1:21pm UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/6 "2024-01-13T13:21:30Z")
I created two index templates, by copying the default logs a metrics ones, creating the pattern for the index to match our namespace and just set a higher priority of 105 on them. It seems to have applied my lifecycle policy fine. Is this a bad way of doing it? It seems to work ok. I’m not sure what integrations we will end up using in the end. That is unfortunate about the helm chart. I also see the ones for filebeat and metricbeat are marked on GitHub as read only project and seem to be unmaintained. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 10, 2024, 1:21pm UTC](https://discuss.elastic.co/t/elastic-agent-and-index-names/350985/7 "2024-02-10T13:21:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.