Metricbeat 7.8 IIS Module Malformed Events Emitted from application_pool metricset

Good afternoon,

I'm currently seeing lots of missing data from the IIS module, specifically the application_pool metricset:

I believe there are a couple of different things going on here, including the same errors mentioned in this post, but I think the dropped app_pool data is primarily due to errors like the following:

2020-07-16T19:05:32.808Z	WARN	[elasticsearch]	elasticsearch/client.go:407	Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbfbc471eefb138f0, ext:69921277629501, loc:(*time.Location)(0x6122ec0)}, Meta:null, Fields:{"agent":{"ephemeral_id":"6cffe5b7-a5c8-4427-82d6-fed248a5bf6a","hostname":"v1bwft1st1001","id":"6da871d8-b368-4ada-9eee-919fdd0a1723","name":"v1bwft1st1001","type":"metricbeat","version":"7.8.0"},"cloud":{"account":{"id":"*******"},"availability_zone":"*******","image":{"id":"********"},"instance":{"id":"********"},"machine":{"type":"t3.large"},"provider":"aws","region":"us-east-1"},"ecs":{"version":"1.5.0"},"event":{"dataset":"iis.application_pool","duration":17596900,"module":"iis","timezone":"+00:00"},"host":{"architecture":"x86_64","hostname":"v1bwft1st1001","id":"*********","ip":["**********","*******","*******"],"mac":["*******","*******"],"name":"v1bwft1st1001","os":{"build":"9600.19404","family":"windows","kernel":"6.3.9600.19395 (winblue_ltsb.190606-0600)","name":"Windows Server 2012 R2 Standard","platform":"windows","version":"6.3"}},"iis":{"application_pool":{"":160.000000,"name":"Admin"}},"kibo":{"environment":"*******","su":"*******","tenant":"*******"},"metricset":{"name":"application_pool","period":10000},"service":{"type":"iis"},"tags":["*******","*******","*******"]}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"field name cannot be an empty string"}}

Note the empty string field name under iis.application_pool. See below for relevant config info.


# Module: iis
# Docs:

- module: iis
    - webserver
    - website
    - application_pool
  enabled: true
  period: 10s

 # filter on application pool names
 # []


#==========================  Modules configuration ============================

  path: ${path.config}/modules.d/*.yml

#==================== Elasticsearch template setting ==========================
  index.number_of_shards: 1
  index.codec: best_compression
  #_source.enabled: false

setup.ilm.enabled: true
setup.ilm.rollover_alias: "metricbeat"
setup.ilm.pattern: "{now/d}-000001"

#-------------------------- Elasticsearch output ------------------------------
  # Array of hosts to connect to.
  hosts: ["${ES_HOST}"] 
  port: "${ES_PORT}"
  protocol: "${ES_PROTOCOL}"
  ssl.verification_mode: "none"
  username: "${ES_USER}"
  password: "${ES_PASSWORD}"

#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

  - add_host_metadata:
      netinfo.enabled: true
  - add_locale: ~
  - add_fields:
      target: "kibo"
        environment: "*****"
        su: "*****"
        tenant: "*****"
  - add_cloud_metadata: ~
  - add_tags:
      tags: ["*****","*****","*****"]
#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

I've tried a few different settings and processors to try to work around this issue without success. I'm currently working on reproducing this metricset manually using the windows perfmon metricset directly. Any information you might have about this issue would be most appreciated.

Edit: Should add that I've tried downgrading and see the same behavior on 7.7.


It seems that something is broken in the mapping. I don't know if this is because of something related to your environment or it is a generic issue. No matter what is the reason I think this kind of errors should be handled properly. Could you please open a Github issue providing information so as the team to try and reproduce and come with a fix/improvement?


hi @william.shipman, a fix for this has been recently merged

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.