Lack of Beats resources

Elastic Stack Beats
1

Hi All;
I am new to ELK, and have been tasked with implementing the Beats side of the deployment. Sadly, I find that I cannot find any reference materials online that address working with Beats. Yes, I have seen the documentation on the ES site, but I find it inadequate for my needs. I cannot find any commercial reference material at this time either.

I am trying to do stuff the rest of you may find basic but is puzzling me. For example, I set me up a prospector, and I am trying to filter out all the lines in the Chef-Client log that say "INFO: Processing". Seems like it should be easy, but it aint working. The example is rather sparse, does not cover multiple entries well.

I was hoping to find more examples of people posting a filebeat.yml that shows some more complex configurations, perhaps some that are already set up for files like that. There dont seem to be many out there for me to dissect and learn from.

Does anybody know of any reference books available besides the sparse online docs?

3

What does your config look like? What is and isn't happening?

4

Thanks Mark. Withdrew that question after finding a rogue Filebeat running in background that was using an old config. Still seeking a more comprehensive manual for working with Beats though.. :slight_smile:

5

It sounds like you've probably already seen the Filebeat documentation here:
https://www.elastic.co/guide/en/beats/filebeat/current/index.html

If you have specific feedback about what's missing from docs, we'd love to hear it. You can open an issue in GitHub describing the areas of content that require better coverage.

We do have plans to add more content to the documentation (especially around specific use cases). But for now, as you've discovered, the Beats forum is a great place to go when you have questions.

Thanks!

1 Like
6

Thanks Dede;
Specifically, I like examples. Much of the information I have is simply based on reading the comments in the filebeat.full.yml file. I am working my way through but many of the options have no example of what they might look like. For example, if I am listing out the paths for log files, the format looks like unto this:

  • input_type: log
    paths:
    ##Global
    • /var/log/cron
    • /var/log/messages
    • /var/log/secure

When I list out Excludes it is different. Are they both legal? Can I use the – as above to group my excludes?

******** chef-client.log ******** (All?)

  • input_type: log
    paths:
    • /var/log/chef-client.log
      exclude_lines: ["INFO: Processing ", "INFO: Checking ", "INFO: Capability ", "INFO: Report handlers","INFO: Running report handlers "]
      fields:
      level: FileLevelField
      review: FileLevelField

Tried to uncomment some lines to try inserting File level fields, that does not seem to be working. Is it because my indenting is off? Or some other reason?

For this reason, I like to see config files that are actually using the extra options, and would ideally like to find me a book that walks through the best ways to work with Beats. Next, I am going to need to figger out how to script the creation of the config file based on the type of server it is getting pushed to. I am playing with options now to work out how I can best do that.

7

You'll find some examples in the topics nested under Configuration Options, but I agree that the docs could use more examples.

YAML also has some odd behaviors that sometimes cause problems. You might want to read this topic if you haven't already: https://www.elastic.co/guide/en/beats/filebeat/current/yaml-tips.html

Thanks for the detailed feedback!

8

Hi Dede;
Books are good. Reference books are essential. The ES site has a link to a pre-release version of an O'Reilly book. Sadly, that version is out of date, as the actual release version is available on Amazon, but even THAT version is over two years old and does not cover 5.4. The online docs are not available as a PDF for download, which makes them less useful to me. I have been muddling through, figuring things out the hard way, but it shouldn't be that way. Thanks for talking with me.
BTW, if you want to hire me to learn this stuff and produce useful training material, I would be very happy.

9

We definitely want to make the user experience as easy as possible.

You should also check out the courses offered by the folks in training: https://www.elastic.co/training

We have traditional instructor-led courses, as well as virtual and on-demand training. Sometimes the early versions of the on-demand courses are available for free, so I'd recommend watching that space.

10

This topic was automatically closed after 21 days. New replies are no longer allowed.