Embedding a custom beat in my application

Elastic Stack Beats
1

I am exploring publishing metrics from my application to ES, and it looks like libbeat might be a good way to do this.

However, I'm having trouble seeing how to instantiate a github.com/elastic/beats/libbeat/beat.Beat to pass to the constructor of my custom github.com/elastic/beats/libbeat/beat.Beater. All of the examples I can find are for building a standalone monitoring application that reads config files, etc.

Am I wrong that libbeat could help me here? Can anyone point me to a similar use case?

Thanks!

1 Like
2

Libbeat so far is definitively focused to make building a Beat but publisher and all other functionality is inside.

Can you perhaps share a bit more details on what the exact features are you are looking for?

3

hi ruflin,
I want to develop a output-plugins (will publish all events to my own service via that output) in a beat, is there any guide or doc?

thanks!

4

Have a look at this thread here: How to create a new Beats output?

It's a bit outdated as lots of things in the interfaces used changed with 6.0. I would recommend you to have a look at one of the simpler outputs like the file output to see what the minimal interfaces are that are required. https://github.com/elastic/beats/tree/master/libbeat/outputs/fileout

5

Great, that's helpful! Thank you!

6

Right now, we send lots of metrics on everything in our application to graphite (using go-metrics). However, we are experiencing some growing pains with graphite, so I wanted to explore using ES as the backend for this. I thought embedding beats into our application (in place of go-metrics-graphite) would make sense, but maybe it isn't a good fit.

Thanks for helping clarify!

7

It probably can be done but is not really what beats was designed for.

If you are looking for a library that you can to send data to Elasticsearch (through apm-server) have a look at our go apm agent: https://github.com/elastic/apm-agent-go It does not support yet metrics but should happen in the future.

As I'm very interested on how people want to send metrics to ES can you perhaps share some more details on the metrics you generate? To you aggregate them on the client side or send one event for each? What is the structure of the metric events you send out?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.