Hi-
I just noticed the HTTP metricset added to the metricbeats modules, Is there a way to make a http get call and add the response to the all Windows metricset documents. Would it be possible? Please let me know. Thanks !
No, they are 2 different modules and they each ship their own separate documents/events into the output configured in metricbeat.
To augment events emitted by a module or add something to them you would use the "fields" parameter that can be defined in a module configuration like described here:
https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-metricbeat.html
You can also use the processors which can also be configured per module and they can modify the events emitted by a module, like add certain fields or modify them.
Maybe if you described what you're trying to do, there is something else we can suggest. But you can't use the events from one module to augment the events from another.
@martinr_ubi
We are collecting the Windows Counters data from a server (created graphs out of it) and installing a software with different versions (Rest API enabled on it).
We need to have a dashboard with a FILTER with the installed software Version and compare it with the previous versions.
so we got to have version also to be sent across all the documents in Windows metricset.
Please let me know if its unclear. Thanks !
Which version of metricbeat are you using?
Also from your description I understand the previous version of the software was installed on the same server? I mean we're talking about 1 server that had a previous version of a software, then you installed a new version on that same server?
This server is sending windows perfmon metrics during all that time and you want to be able to easily compare the perfmon for the 2 versions?
You'd like the perfmon metrics to have a field indicating which version of the software was running on that server at the time the metric was emitted. Essentially tagging all the perfmon metrics that are emitted with the version of the software.
Is the server running in some kind of cloud where the server can be tagged? Like AWS?
How is the new version of the software installed on the server? Configuration management? Custom install scripts? Manually? Is the server completely rebooted each time your install a new version of the software?
@martinr_ubi-
Am using Metricbeat version 7.0 and its the same Server (Windows box in local infrastructure not in the Cloud)
I have my responses below:
This server is sending windows perfmon metrics during all that time and you want to be able to easily compare the perfmon for the 2 versions? --- **Yes, exactly**
You'd like the perfmon metrics to have a field indicating which version of the software was running on that server at the time the metric was emitted. Essentially tagging all the perfmon metrics that are emitted with the version of the software. --
> Yes, Exactly
How is the new version of the software installed on the server? Configuration management? Custom install scripts? Manually? Is the server completely rebooted each time your install a new version of the software? - **Its is thru the Configuration Management tools and restart will not happen**
As documented here:
https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-configuration-reloading.html
and here:
https://www.elastic.co/guide/en/beats/metricbeat/current/_live_reloading.html
You'll see that metricbeat modules can be configured in their own file under modules.d directory and that metricbeat can watch those files for changes and reload the module when changes are detected.
How I would try to solve your usecase is that I would use the configuration management that is installing the "software". During this process, the config management is obviously aware of the version of the software it is installing or upgrading. If it's not, it can, I hope, easily be made aware of it.
Now if the configuration management uses this to create/update the .yml file where you have configured the windows module with the perfmon metricset, it can add a "fields" configuration to the module configuration which will add the version in a custom field to all the events emitted by the module. It could also use a processor.
See here for an example of using either "fields" or a processor in a module configuration to add custom fields to all events emitted by a module:
https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-metricbeat.html
In theory because the yml file has now changed, metricbeat should reload the module and take account of the value you now want to add to all the events for that module. If that doesn't work I think that would be a bug... Because metricbeat should reload the module when it detects the configuration file for the module has changed under the modules.d directory. I say it like that because I'm not sure I have ever actually tested it specifically myself, to see if just changing a value under the "fields" parameter of a module lead to the correct reload and reconfiguration of the module. That is what I expect though.
Now you could say that's all dandy but isn't there a delay here... If Metricbeat detect changes every 10s like is suggested in the documentation. The perfmon could be tagged with the wrong version after an upgrade for up to 10 seconds because metricbeat has not yet reloaded the module configuration and has not yet changed the version value with which it augments all the events of the module(all the perfmon metrics).
Yes, if that is not acceptable, you could lower the reload.period setting.
If that is not desirable, not deterministic enough, or if you prefer, you could include a restart of metricbeat in the configuration management after you have deployed a new version of the "software". Going at it this way would actually enable you to not even use metricbeat automatic reloading of configuration through file change detection... Because you would restart metricbeat through the configuration management after it has updated the metricbeat module config file with the new value of the version.
Variations on this theme include using the "fields" parameter or a processor in the main metricbeat config file so that ALL events emitted by metricbeat are tagged with the version. But because the main config file is not subject to auto-reload on change, you would be obligated to restart metricbeat from the config management after it has updated the metricbeat config file.
In essence the idea here is similar to someone using any Metricbeat module which require coordination of settings between Metricbeat and the monitored software.
For example if my config management installs and configures mongoDB on a certain port, with a certain user and password. It MUST pass this same information to the MongoDB module configuration of metricbeat or else metricbeat won't be able to successfully monitor MongoDB.
So the config management would (re)configure MongoDB AND the metricbeat module for MongoDB and then restart metricbeat because they share some values that needs to match across reconfigurations.
You would do the same thing, but here, it's to make a module aware of the version of something that was running at the time the metrics are emitted and tag them with this information.
Not saying there is no other way of course, it' just one I would look into. It's also similar to what I do to inject machine tags into Beats config file and like I said it's similar to anyone configuring metricbeat modules at the same time they are configuring the software being monitored by metricbeat.
Thanks so mich @martinr_ubi. It makes sense to update the field and reload the config file. 
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.