Infrastructure UI with Windows and Unix hosts

[Marc-Antoine_J (Marc Antoine Jutras)]

I'm discovering the new Infrastructure UI and I like it so far. But I had a few issue especially with the year change.

I was configuring my beats to use the metricbeat-{+YYYY.MM} index naming pattern. Since I set it up in december it only created one index (metricbeat-2018.12). Then I have set my elastic host (unix) to log metricbeats directly to elasticsearch. After I have set many Windows host to do the same. Everything worked fine so far.

Then the month (and year) changed... The new index created with no templates. The infrastructure stopped showing the hosts info. I went to look for the template mapping and it was set for metricbeat-6.5.4 index names only. I changed it for metricbeat-*. It worked for my unix host then I noticed that I had set 2 different template mapping for windows hosts and unix host. I had set Windows hosts to metricbeat-local. Don't bother asking for the confusion it was before the holiday leave. My head was already gone.

Then I noticed that the template were different. If it was created by the unix host it started with the docker settings and maybe things are just not in the same order I thought. But then fixing both configs (Windows and Unix) I noticed that both did not get their logs correctly in the templates. So now I have configured metricbeat-unix-{+YYYY.MM.dd} (added day for rollover) for unix and metricbeat-windows-{+YYYY.MM.dd} for Windows hosts.

I thought that infrastructure was taking it's docs from metricbeat-* but now I can see I have both indexes, they both have tons of documents but infrastructure UI shows only Windows hosts.

I must be missing something?

Thanks,

[exekias (Carlos Pérez Aradros)]

Hi @Marc-Antoine_J,

Can you paste your settings after changing the index name? When doing so, you will also need to change template name and pattern to make sure it matches the new name. More documentation on this can be found here:

https://www.elastic.co/guide/en/beats/metricbeat/current/elasticsearch-output.html#index-option-es

[Marc-Antoine_J (Marc Antoine Jutras)]

I did update the template name and patterns. I made 2, one for unix, one for windows since the auto-creation of the templates from one or the other are different.

Here's the Unix config:

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true

setup.template.enabled: true
setup.template.name: "metricbeat-local-unix"
setup.template.pattern: "metricbeat-unix-*"

setup.template.settings:
  index:
    number_of_shards: 1
    codec: best_compression
  _source:
    enabled: false

setup.kibana:
  host: "xxx.xxx.xxx.xxx:xxxx"
  
output.elasticsearch:
  hosts: ["xxx.xxx.xxx.xxx:xxxx"]
  index: "metricbeat-unix-%{+YYYY.MM.dd}"
  
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch:

Here's the Windows config:

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  
setup.template.enabled: true
setup.template.name: "metricbeat-local"
setup.template.pattern: "metricbeat-windows-*"

setup.template.settings:
  index:
    number_of_shards: 1
    codec: best_compression
  _source:
    enabled: false
	
setup.kibana:

output.elasticsearch:
  hosts: ["xxx.xxx.xxx.xxx:xxxx"]
  index: "metricbeat-windows-%{+YYYY.MM.dd}"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

logging.level: error

logging.to_files: true
logging.files:
  path: C:\MetricBeat\Logs
  name: metricbeat
  rotateeverybytes: 10485760 # = 10MB
  keepfiles: 7

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch:

[exekias (Carlos Pérez Aradros)]

Hi @Marc-Antoine_J,

You don't need different mapping for beats per operative system, all of them share the same template.

That said, your settings look correct anyway. Have you confirmed that Metricbeat is reporting data for both cases?

PD: As a note, Beats documentation recommends to keep %{[beat.version]} in the index name & pattern, so version upgrades don't end up in conflicting mappings: https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#index-option-es

[Marc-Antoine_J (Marc Antoine Jutras)]

Hi @exekias,

Thanks for your support. I feel ashamed, I just noticed that working on the new windows metricbeat configs I stopped metricbeat on my elasticsearch unix server. Starting it fixed it!

I used WinMerge to differentiate the windows generated and the unix generated templates and now they seems to have the same parameters it's just that it's all mixed up and not in the same order at all. I don't know if the parameters order is randomly generated. The first time I looked it showed so much differences because of the mixup that I thought they were different.

Things are getting clearer about the infrastructure UI. I like it. It could be nice if it was adding some monitoring of drives space as this is very often an issue on our servers and it causes failures. I still monitor it through the logs of metricbeat. There's just no metric for this in the Infrastructure UI. Maybe customization of the available metrics in the UI could be nice?

I removed the beat version from the name for an easier cleanup of the indexes as I'm managing a certain retention with local scripts but I could re-include it and just use wildcards. I'll consider this.

Thanks and sorry for my mistake.

[exekias (Carlos Pérez Aradros)]

I'm glad you got it working!

Thank you for the feedback, it really help us when shaping our roadmap!

Best regards

[kaduswapnali456 (swapnali)]

Excellent topic. Thank you.