Creating New Indices using Functionbeat

[joabrb22 (Jeremiah)]

I have searched across the internet after having read the documentation, but I cannot seem to find the answer to my question.

I want to create multiple new indicesfor Functionbeat, besides the default. I created a new index, let's call it example-index, using the Kibana web interface. I can see the new index using the CLI, alongside the default functionbeat one. In my functionbeat.yml file, I have added an "indices" section under the "output.elasticsearch" section. Under that section I have added the following.

output.elasticsearch:
    hosts: [...]
    indices:
         - index: "example-index"
           when.contains:
               log_group: "/example_logs"

The issue is that nothing is being logged to this new index right now. What am I missing?

[Mario_Castro]

Hi @joabrb22

You are not using the indices feature correctly. when.contains will match against the content of the log line. You don't write there the path to logs... unless that specific string is the expected content of the message in such case I'll maybe remove the / and try, just in case.

Also, remember that log_group must be part of the JSON message as you can see in the overall description of the conditions https://www.elastic.co/guide/en/beats/functionbeat/current/defining-processors.html#conditions

[joabrb22 (Jeremiah)]

Hi @Mario_Castro
Thanks for your response

I am basing the when.contains on the search query I use to find all of the logs that are part of that log group. I have tried searching for a "piece" of the log group and not received any results, so my assumption was that it needed to be exact. A log group would look more like /log_group/example_logs/example_log1 in my case. This information is contained in the JSON message as well.

Do I need to redeploy functionbeat or anything like that to make the changes take affect after editing the yml file?

Thanks for the help with this!

[joabrb22 (Jeremiah)]

I wanted to follow up on this as I have seen this question asked a few times and want to provide a resource for those looking.

Being new to the utility, I was over thinking it. It really is a as simple as setting up the named indices and using the functionbeat "update" command on them once you have setup your yml. The default functionbeat template can be used with your custom indices with minimal adjustment made to the yml. I set the template name and pattern to the functionbeat default and the template fields to the default one included.