I have about 2000 Elastic agents (version 8.9.0) connected to a system with 3 Fleet servers (version 8.9.0).
We have about 20 different agent policies, because the various Elastic agents are sending
slightly different logs, and for certain cases we need to specify specific pipelines to process the logs.
In the Fleet UI for configuring an Elastic Agent policy,
the namespace is initially set to
On this page: Data streams | Fleet and Elastic Agent Guide [8.9] | Elastic
It mentions that the default naming scheme is:
If I have 20 different agent policies over 2000 elastic agents, do you recommend that from a performance perspective,
I configure 20 different namespaces, 1 unique namespace for each agent policy?
In the past, we left everything as namespace
default, and this did not seem to be ideal from a performance perspective.
I did some research and consulted with Elastic and learned the following, which I believe
addresses my questions:
Specifying different namespaces in the agent policy
offers performance and organizational benefits.
Data writes will be distributed across multiple namespaces
instead of having all agents write to a single default
This can reduce contention and potential bottlenecks of
using a single namespace, which can lead to improved performance.
Using separate namespaces allows for more fine-grained
acess control and organization. This allows tailoring
the permissions for specific policy requirements, which
can enhance security and data governance.
The trade-off of more namespaces is that each namespace
introduces some overhead, such as resource consumption for
management and administration.
Another approach to improving performance with thousands
of elastic agents is adjusting the number of shards within the index
- Increasing the shards in the index can improve performance by distributing the data
more evenly across the index. Shard management introduces its
own set of complexities such as increased resource usage and more
complex indexing strategies.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.