Elastic Agent and Tagging

Elastic Stack Elasticsearch
1

Greetings,

I have an EC account and I have been really enjoying using the elastic agent. However, I am hitting a few rough spots.

  1. I am trying to tag in a general sense in the Elasticsearch output but I notice that -- that no matter what I do in advanced yaml settings I can not get it to hit the pipeline. Been checking the _node end point and for whatever reason i just can't get it to fire.

  2. I am having a hardtime looking for guidance on what policies should look like we have an appplication that has a lot of different pieces like part a, part b, etc. Does elastic have any best practices for figuring what our policies should look like? I have a lot of agents to install and I am curious how architecture should look like for each environment. I mean are the policies supposed to relate to the logical or the role of the policy (like web server vs part a)

2

Hi,

thanks for the feedback.

for (1) I believe you are trying to enhance the user data and add metadata. This is best achieved using processors. As of now we are not able to globally achieve this (i.e all data ingested by agents in a given policy). Please track this public issue: [Elastic Agent] Support Global processors on a per policy basis · Issue #29459 · elastic/beats · GitHub

We are also working on tagging the agents themselves so they are grouped and categorized in the UI. I don;t think this is what you are after.

One option that exist is for you to add processors on the input itself. Most of the integrations will have a processor block under the advanced config. This a bit more effort but if you don;t have that many processors it would be manageable.

for (2) I would like to understand the issue a bit better. At the moment Agents can only belong to one policy. these agents should be grouped based on the data you intend to ingest (so that the integrations enabled on them are the same for all agents). there's no hard and fast rule, you can use the policy how you wish. We have seen users dedicate a policy to a tenant/customer or even agents in a data center. In some cases also when data governance is a concern, they dedicate a policy to agents in a singe; jurisdiction.

We are considering a model where agents could potentially belong to multiple policies. But that is some ways off.

hope this helps
Nima

3

@Nima_Rezainia thank you for the reply

(2) I have about 22 policies right now to group my agents. But this is only for one environment (QA) so that means by the end of this I will have 44 policies to manage and this is going to be difficult to manage without solid automation in place. Something isn't connecting on how I can map the application to be somewhat manageable. For example assume the application had the following seperations [Billing] --> [Billing Frontend] --> [Recieving] --> [Printing] ...

I am wondering if that I should just do something like this

[Web] -- [App] -- [Workers] and then do tagging on these so that I only have to manage 6 total plus the integrations for the 6 rather then 44 plus integrations.

I noticed the advanced config -- but also noticed that not all integrations have that specific section in their integration. I really do like of what the promise of ElasticAgent will bring but it seems like it's not fit for purpose yet with the fleet management and I may be best looking at Stand Alone mode based on my requirements.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.