Good afternoon,
I'm currently seeing lots of missing data from the IIS module, specifically the application_pool metricset:
I believe there are a couple of different things going on here, including the same errors mentioned in this post, but I think the dropped app_pool data is primarily due to errors like the following:
2020-07-16T19:05:32.808Z WARN [elasticsearch] elasticsearch/client.go:407 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbfbc471eefb138f0, ext:69921277629501, loc:(*time.Location)(0x6122ec0)}, Meta:null, Fields:{"agent":{"ephemeral_id":"6cffe5b7-a5c8-4427-82d6-fed248a5bf6a","hostname":"v1bwft1st1001","id":"6da871d8-b368-4ada-9eee-919fdd0a1723","name":"v1bwft1st1001","type":"metricbeat","version":"7.8.0"},"cloud":{"account":{"id":"*******"},"availability_zone":"*******","image":{"id":"********"},"instance":{"id":"********"},"machine":{"type":"t3.large"},"provider":"aws","region":"us-east-1"},"ecs":{"version":"1.5.0"},"event":{"dataset":"iis.application_pool","duration":17596900,"module":"iis","timezone":"+00:00"},"host":{"architecture":"x86_64","hostname":"v1bwft1st1001","id":"*********","ip":["**********","*******","*******"],"mac":["*******","*******"],"name":"v1bwft1st1001","os":{"build":"9600.19404","family":"windows","kernel":"6.3.9600.19395 (winblue_ltsb.190606-0600)","name":"Windows Server 2012 R2 Standard","platform":"windows","version":"6.3"}},"iis":{"application_pool":{"":160.000000,"name":"Admin"}},"kibo":{"environment":"*******","su":"*******","tenant":"*******"},"metricset":{"name":"application_pool","period":10000},"service":{"type":"iis"},"tags":["*******","*******","*******"]}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}}
Note the empty string field name under iis.application_pool. See below for relevant config info.
iis.yml
# Module: iis
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.8/metricbeat-module-iis.html
- module: iis
metricsets:
- webserver
- website
- application_pool
enabled: true
period: 10s
# filter on application pool names
# application_pool.name: []
metricbeat.yml
#========================== Modules configuration ============================
metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
#_source.enabled: false
setup.ilm.enabled: true
setup.ilm.rollover_alias: "metricbeat"
setup.ilm.pattern: "{now/d}-000001"
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["${ES_HOST}"]
port: "${ES_PORT}"
protocol: "${ES_PROTOCOL}"
ssl.verification_mode: "none"
username: "${ES_USER}"
password: "${ES_PASSWORD}"
#================================ Processors =====================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
- add_host_metadata:
netinfo.enabled: true
- add_locale: ~
- add_fields:
target: "kibo"
fields:
environment: "*****"
su: "*****"
tenant: "*****"
- add_cloud_metadata: ~
- add_tags:
tags: ["*****","*****","*****"]
#================================ Logging =====================================
# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug
# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]
I've tried a few different settings and processors to try to work around this issue without success. I'm currently working on reproducing this metricset manually using the windows perfmon metricset directly. Any information you might have about this issue would be most appreciated.
Edit: Should add that I've tried downgrading and see the same behavior on 7.7.