Q: is it possible to send from filebeats/metricbeats via elasticagent to logstash or elastic?

Elastic Observability
1

Hello everybody,

Background: We have a setup of a set of computers, where "just one" of that is connected to the Companies Network - with a dedicated unique IP Adress etc.
The other two systems (lets call them sysb, sysc) are ONLY connected the first one (call it sysa) in a local network.
Such that - sysa manages local subnet for sysb and sysc -therefore also sysb and sysc have always the very same Network. Note: This setting is replicated in at least 100+ installations beeing monitored.

My requirements: I want to monitor sysa, sysb and sysc with the cool tools of elastic (elasticagent,beats) w.r.t. behavior, statistics, logs etc...

Therefore, I focussed on the elasticagent beeing installed to sysa.

My idea is, that the agent on sysa now can act as some sort of data-relay for sysb and sysc,
such that - on sysb, sysc some of the beats collect data, and send those data to the agent on sysa.

On sysa, the data is appended with additional fields (feature of the elasticagent, providers) - to place additional markers on the documents for later data analysis.

Such that - is there a chance to connect a beats output to the elasticagent as input (similar to the apm techniques) ?
Would there be other solutions - I am not aware of?
Clarification: For a set of reasons, there is NO way/ no option to attach sysb and sysc directly to the network.

Any ideas?

Thanks in advance,
DrG

2

Hi @DrG

From what I understand in your context, sysa will be your output proxy, that is, it will receive the sysb and sysc metrics.

If my understanding is correct, you can install logstash on sysa with sysb and sysc collection pipelines. Pipelines can be input from beats coming from sysb and sysc, then the pipeline will have an output for its elasticsearch that will centralize the collections.

Below is the link with some models of the logstash pipelines:

Beats input plugin | Logstash Reference [8.9] | Elastic

Regards

3

Elastic Agent can only output to Elasticsearch or Logstash, so in your case you would need to set up a Logstash instance on the sysa server to receive the logs from the other Agents and send them to Elasticsearch.

Another issue is that Elastic Agent needs to talk with a Fleet Server, so you choose to use Elastic Agent you will also need a Fleet that is able to talk with both elasticsearch and your agents.