Errors with filebeat when trying to integrate any windows integration logs with the agent

Hello everyone, I need some help regarding what's going on
I setup my Elasticsearch + fleet server + kibana with a ubuntu machine.
I added a windows machine with elastic agent. (Interation ElasticDefend => I can see all logs)
Now when i try to "windows" or "system" integration to the agent. I don't see any logs related to these 2 (I have sysmon installed and running and I confirmed in event viewer that logs exists)
After a while I found out that I am getting a bunch of errors like this for every winlog configured

"message": "Input 'winlog' failed with: input winlog-windows.sysmon_operational-3de9734d-7e7a-418e-8255-c9b99eb35a87 failed: input panic with: name winlog-windows_sysmon_operational-3de9734d-7e7a-418e-8255-c9b99eb35a87 already used\ngoroutine 152 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x5e\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).runSource.func1()\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:172 +0x58\npanic({0x872a180?, 0xc0033074f0?})\n\truntime/panic.go:792 +0x132\ngithub.com/elastic/elastic-agent-libs/monitoring.panicErr(...)\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:287\ngithub.com/elastic/elastic-agent-libs/monitoring.(*Registry).Add(0xc00241f100, {0xc002171270?, 0xc00330e7e0?}, {0xa843860, 0xc00241fbc0}, 0x0?)\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:185 +0xca\ngithub.com/elastic/elastic-agent-libs/monitoring.(*Registry).NewRegistry(0xc00241f100, {0xc002171270, 0x46}, {0x0?, 0xa81af10?, 0x1?})\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:94 +0x155\ngithub.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry({0x996d6aa, 0x6}, {0xc0025a5270, 0x46}, 0x0)\n\tgithub.com/elastic/beats/v7/libbeat/monitoring/inputmon/input.go:73 +0x36f\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.newInputMetrics({0xc0025bed80, 0x24}, {0xc0025a5270?, 0xc001b6c050?})\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/metrics.go:79 +0x56\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.(*winEventLog).Open(0xc001ee8780, {{0x0, 0x0}, 0x0, {0x0, 0x0, 0x0}, {0x0, 0x0}})\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/wineventlog.go:167 +0xda\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.Run({0xa84f240, 0xc001ae0580}, {0xa8e6d88, 0xc001b6c050}, {0xa9056a0, 0xc001ee8780}, {{0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/runner.go:80 +0x331\ngithub.com/elastic/beats/v7/filebeat/input/winlog.winlogInput.Run({}, {0xc003132290, {0xc001de8240, 0x8e}, {0xc0025a5270, 0x46}, {0x996d6aa, 0x6}, {{0x99767c6, 0x8}, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/filebeat/input/winlog/input.go:103 +0x3ff\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).runSource(_, {0xc003132290, {0xc001de8240, 0x8e}, {0xc0025a5270, 0x46}, {0x996d6aa, 0x6}, {{0x99767c6, 0x8}, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:196 +0x483\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).Run.func1()\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:151 +0x2f2\ngithub.com/elastic/go-concert/unison.(*MultiErrGroup).Go.func1()\n\tgithub.com/elastic/go-concert@v0.3.0/unison/multierrgroup.go:42 +0x66\ncreated by github.com/elastic/go-concert/unison.(*MultiErrGroup).Go in goroutine 259\n\tgithub.com/elastic/go-concert@v0.3.0/unison/multierrgroup.go:40 +0x76\n",

Even though the policy is quite simple and nothing dulicate in it. even when trying to delete all integrations inside the policy and creating a new "Windows" one => same issue.

 - id: winlog-windows-3de9734d-7e7a-418e-8255-c9b99eb35a87
    name: windows-1
    revision: 1
    type: winlog
    use_output: default
    meta:
      package:
        name: windows
        version: 3.1.0
    data_stream:
      namespace: default
    package_policy_id: 3de9734d-7e7a-418e-8255-c9b99eb35a87
    streams:
      - id: winlog-windows.powershell-3de9734d-7e7a-418e-8255-c9b99eb35a87
        name: Windows PowerShell
        data_stream:
          dataset: windows.powershell
          type: logs
        condition: ${host.platform} == 'windows'
        event_id: 400, 403, 600, 800
        ignore_older: 72h
        processors:
          - translate_sid:
              field: winlog.event_data.MemberSid
              account_name_target: winlog.event_data._MemberUserName
              domain_target: winlog.event_data._MemberDomain
              account_type_target: winlog.event_data._MemberAccountType
              ignore_missing: true
              ignore_failure: true
      - id: >-
          winlog-windows.powershell_operational-3de9734d-7e7a-418e-8255-c9b99eb35a87
        name: Microsoft-Windows-PowerShell/Operational
        data_stream:
          dataset: windows.powershell_operational
          type: logs
        condition: ${host.platform} == 'windows'
        event_id: 4103, 4104, 4105, 4106
        ignore_older: 72h
        processors:
          - translate_sid:
              field: winlog.event_data.MemberSid
              account_name_target: winlog.event_data._MemberUserName
              domain_target: winlog.event_data._MemberDomain
              account_type_target: winlog.event_data._MemberAccountType
              ignore_missing: true
              ignore_failure: true
      - id: winlog-windows.sysmon_operational-3de9734d-7e7a-418e-8255-c9b99eb35a87
        name: Microsoft-Windows-Sysmon/Operational
        data_stream:
          dataset: windows.sysmon_operational
          type: logs
        condition: ${host.platform} == 'windows'
        ignore_older: 72h
        processors:
          - translate_sid:
              field: winlog.event_data.MemberSid
              account_name_target: winlog.event_data._MemberUserName
              domain_target: winlog.event_data._MemberDomain
              account_type_target: winlog.event_data._MemberAccountType
              ignore_missing: true
              ignore_failure: true

what could be causing this?

i believe i have the exact same issue.
Noticed this too late, i created a thread here Elastic agent Windows integration issue - Elastic Stack / Elastic Agent - Discuss the Elastic Stack

Hello @v_i_x_x

Welcome to the community!!

As per the logs/error shared :

"message": "Input 'winlog' failed with: input winlog-windows.sysmon_operational-3de9734d-7e7a-418e-8255-c9b99eb35a87 failed: input panic with: name winlog-windows_sysmon_operational-3de9734d-7e7a-418e-8255-c9b99eb35a87 already used\ngoroutine 152 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26

Could you please check the Policy if it has multiple similar integration because of we see this error or the elasticsearch.yml file to see if there is similar inputs section already available for the winlog input ?

Thanks!!

Hello,

given i have a similar problem and the exact same error, i'm posting my output.
please let me know if this is not relevant.

elasticsearch.yml: no winlog input.
elastic-agent.yml:

EDIT: ran the agent with the Windows integration (and nothing else, not even the default System integration), only enabling the sysmon events, it produced the same error.

Hello @tortoise thank you for the help.
I did today a fresh install on a new server and a new windows machine. same issue.


this is the full windows-policy i am using

id: f602927f-dd23-4a97-b92f-b39ef8c2b189
revision: 1
outputs:
  fleetserver-output-6adb209b-6310-45c9-b1c2-ea06d6baf1bb:
    type: elasticsearch
    ssl:
      certificate_authorities:
        - |-
          -----BEGIN CERTIFICATE-----
          MIIFajCCA1KgAwIBAgIVAO2dzIdCsmxt5njR01VI+pqLFoceMA0GCSqGSIb3DQEB
          CwUAMDwxOjA4BgNVBAMTMUVsYXN0aWNzZWFyY2ggc2VjdXJpdHkgYXV0by1jb25m
          aWd1cmF0aW9uIEhUVFAgQ0EwHhcNMjUwODA2MTI1NzU0WhcNMjgwODA1MTI1NzU0
          WjA8MTowOAYDVQQDEzFFbGFzdGljc2VhcmNoIHNlY3VyaXR5IGF1dG8tY29uZmln
          dXJhdGlvbiBIVFRQIENBMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA
          qlxv7Vf4bmIc7GpZoaM8qNvXD0P0vZ9IPsUGIaUXjbg9PdLkhgocQcNh04KWDB+0
          fml7Kfr5zusmbe3r+f9Qni0zMzc8yxQTgXZdYPDx9eWj1RjRIdNWmPRx9Bf8llQz
          476kf9FZi3a6O8BkVrQ99OWiq3mmGkEpPWuY5H5pcGqIZdi5TBQGk0mwipr9vFXy
          VOwBRnRnYv11BeWyWKbPpr4IWeiE/WpIxTEnyBJuoSYFJlGYjiBlkFyDQUrljStU
          epn2tlokD/XL7KHfWmd6N6xSXaqX//f2MB7aPYZnI7up1v8xfP2XbCqANz2drsuD
          LYNoOC3u4xYWJEO2qYeP7oDD7G21UO7UJjXNXbe2VcWSYgbD1IKK8llYIB7VVDmA
          BmVOVCyQZrOsJaA1VnGP1cpQCANYGHjbjAJnauWQ6B9WQ1jTYf2BctenwMXMnJ+i
          BeTC5cqU7MJVXmejbix28TVw/q/92wzBrNpu9VoL07fTAApW9Cvr3C4K7LToUZfg
          ow0IqBNF8VwP7UybLgJGgCOrZ8wwTRh78TxFV3dNZf1x9fwWVymEg3Yqm23U42k4
          +ct7VdWqBRbAawWRZ+om6eVpOt+Slbs5coyPiXYhtMk0JPBX+fcAbL6W1snPhzTx
          1lXGeLSS2BU51WYxTfNciA1k6faocytKjNnt18Ao3oECAwEAAaNjMGEwHQYDVR0O
          BBYEFAJI5MLEBXw8fai9MeupG3vtwWSaMB8GA1UdIwQYMBaAFAJI5MLEBXw8fai9
          MeupG3vtwWSaMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMA0GCSqG
          SIb3DQEBCwUAA4ICAQAPw0dK7yZKw/UrxUS55xxaJr0b2H/pKYwQp6tkzDNznA1k
          xw+65Xtn/5qx9XHIQpi1YgvXfMrVVkXzOeal5vi3UKL7UlZ2HUkX1yr/Jikq5Z3J
          7xYqqQAZsX3HqbZiP6xyyac15DOCHQJmVt1GjfHvYQIrNnpTBXfHm017oMA/K3Dv
          EbMfrlS8kmlB/FscXgXbtKjiWw+3izy7lnWQL9w2otmBTgJ8SWf9N1YJuquCLw+r
          Mxh1vM6vhaesFQL3TCwrQJgpH5TYu2d0lWWjuRy92LcN1ceWCytqnpdBSJNybCLc
          41iQxext1jUPNs8CdtA/osIM7z7XbFBmomkdKTPZ1snlWgZuaAKk5IYpq5jrCz+L
          zPrNYGxSFQw9vt17SQlgt73m9a29xsqrtC6AvtA0QwMXDFjS58ID9pc7UDKWbl4H
          KRhU98F493qe/U369NVb4j+77KKNkiL5EZ+r2TJTb+4UUBkhTPBomwwI343NahFO
          A1jMzySu0A0pQeknNFPYqZVE2mNk+aM+n4aJpyD51xqIvreVk+dpHKdbqaJCSJud
          S/l9/NnVPhFne22cYIkx+qC46zCfmSuncnq3FesBBevGwR7NAyyu+Px+ANKkJxmC
          Z9XY1TAh1NSpCfaTmu9PhFksgI0Y5tpBtBx5ihDpjMfB7jbT47B8rHhoZaQyzw==
          -----END CERTIFICATE-----
  default:
    type: elasticsearch
    hosts:
      - https://192.168.100.2:9200
    ssl.ca_trusted_fingerprint: 10697B6F2A9AD002E9282D865103C980554602D1E8E189B8AB81D0FF7D045517
    preset: balanced
fleet:
  hosts:
    - https://192.168.100.2:8220
  ssl:
    certificate_authorities: []
output_permissions:
  fleetserver-output-6adb209b-6310-45c9-b1c2-ea06d6baf1bb: {}
  default:
    _elastic_agent_monitoring:
      indices:
        - names:
            - logs-elastic_agent.apm_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.apm_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.auditbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.auditbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.cloud_defend-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.cloudbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.cloudbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.elastic_agent-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.endpoint_security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.endpoint_security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.filebeat_input-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.filebeat_input-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.filebeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.filebeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.fleet_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.fleet_server-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.heartbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.heartbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.metricbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.metricbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.osquerybeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.osquerybeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.packetbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-elastic_agent.packetbeat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.pf_elastic_collector-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.pf_elastic_symbolizer-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.pf_host_agent-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-elastic_agent.status_change-default
          privileges:
            - auto_configure
            - create_doc
    _elastic_agent_checks:
      cluster:
        - monitor
    aa4e6012-e859-46a2-950c-029ece86e67b:
      indices:
        - names:
            - logs-system.auth-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.syslog-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.auth-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.syslog-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.application-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.system-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.cpu-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.diskio-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.filesystem-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.fsstat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.load-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.memory-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.network-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.process-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.process.summary-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.socket_summary-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.uptime-default
          privileges:
            - auto_configure
            - create_doc
agent:
  download:
    sourceURI: https://artifacts.elastic.co/downloads/
  monitoring:
    enabled: true
    use_output: default
    logs: true
    metrics: true
    traces: true
    namespace: default
  features: {}
  protection:
    enabled: false
    uninstall_token_hash: xBEbm2qZqI8fJ0Vu5J44H2pEofJTIOUXDLP14phNiiI=
    signing_key: >-
      MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEOecBHTnUmEuT8hw5SeZU/JyR1qn0+h7KcnKibEmo6D68MKhYC7iepwQHu4uD7NdruEZAo7DHBd3Lo4L6Et0IOA==
inputs:
  - id: logfile-system-aa4e6012-e859-46a2-950c-029ece86e67b
    name: system-2
    revision: 1
    type: logfile
    use_output: default
    meta:
      package:
        name: system
        version: 2.5.2
    data_stream:
      namespace: default
    package_policy_id: aa4e6012-e859-46a2-950c-029ece86e67b
    streams:
      - id: logfile-system.auth-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.auth
          type: logs
        condition: >-
          ${host.os_version} != "12 (bookworm)" and (${host.os_platform} !=
          "amzn" or ${host.os_version} != "2023") and (${host.os_platform} !=
          "sles" and startsWith(${host.os_version}, "15") == false)
        ignore_older: 72h
        paths:
          - /var/log/auth.log*
          - /var/log/secure*
        exclude_files:
          - \.gz$
        multiline:
          pattern: ^\s
          match: after
        tags:
          - system-auth
        allow_deprecated_use: true
        processors:
          - add_locale: null
          - rename:
              fields:
                - from: message
                  to: event.original
              ignore_missing: true
              fail_on_error: false
          - syslog:
              field: event.original
              ignore_missing: true
              ignore_failure: true
      - id: logfile-system.syslog-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.syslog
          type: logs
        condition: >-
          ${host.os_version} != "12 (bookworm)" and (${host.os_platform} !=
          "amzn" or ${host.os_version} != "2023") and (${host.os_platform} !=
          "sles" and startsWith(${host.os_version}, "15") == false)
        paths:
          - /var/log/messages*
          - /var/log/syslog*
          - /var/log/system*
        exclude_files:
          - \.gz$
        multiline:
          pattern: ^\s
          match: after
        processors:
          - add_locale: null
        allow_deprecated_use: true
        tags: null
        ignore_older: 72h
  - id: journald-system-aa4e6012-e859-46a2-950c-029ece86e67b
    name: system-2
    revision: 1
    type: journald
    use_output: default
    meta:
      package:
        name: system
        version: 2.5.2
    data_stream:
      namespace: default
    package_policy_id: aa4e6012-e859-46a2-950c-029ece86e67b
    streams:
      - id: journald-system.auth-aa4e6012-e859-46a2-950c-029ece86e67b
        type: journald
        data_stream:
          dataset: system.auth
          type: logs
        facilities:
          - 4
          - 10
        condition: >-
          ${host.os_version} == "12 (bookworm)" or (${host.os_platform} ==
          "amzn" and ${host.os_version} == "2023") or (${host.os_platform} ==
          "sles" and startsWith(${host.os_version}, "15") == true)
        tags: null
      - id: journald-system.syslog-aa4e6012-e859-46a2-950c-029ece86e67b
        type: journald
        data_stream:
          dataset: system.syslog
          type: logs
        facilities:
          - 0
          - 1
          - 2
          - 3
          - 5
          - 6
          - 7
          - 8
          - 9
          - 11
          - 12
          - 15
        condition: >-
          ${host.os_version} == "12 (bookworm)" or (${host.os_platform} ==
          "amzn" and ${host.os_version} == "2023") or (${host.os_platform} ==
          "sles" and startsWith(${host.os_version}, "15") == true)
        tags: null
  - id: winlog-system-aa4e6012-e859-46a2-950c-029ece86e67b
    name: system-2
    revision: 1
    type: winlog
    use_output: default
    meta:
      package:
        name: system
        version: 2.5.2
    data_stream:
      namespace: default
    package_policy_id: aa4e6012-e859-46a2-950c-029ece86e67b
    streams:
      - id: winlog-system.application-aa4e6012-e859-46a2-950c-029ece86e67b
        name: Application
        data_stream:
          dataset: system.application
          type: logs
        condition: ${host.platform} == 'windows'
        ignore_older: 72h
      - id: winlog-system.security-aa4e6012-e859-46a2-950c-029ece86e67b
        name: Security
        data_stream:
          dataset: system.security
          type: logs
        condition: ${host.platform} == 'windows'
        ignore_older: 72h
        tags: null
        include_xml: true
      - id: winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b
        name: System
        data_stream:
          dataset: system.system
          type: logs
        condition: ${host.platform} == 'windows'
        ignore_older: 72h
  - id: system/metrics-system-aa4e6012-e859-46a2-950c-029ece86e67b
    name: system-2
    revision: 1
    type: system/metrics
    use_output: default
    meta:
      package:
        name: system
        version: 2.5.2
    data_stream:
      namespace: default
    package_policy_id: aa4e6012-e859-46a2-950c-029ece86e67b
    streams:
      - id: system/metrics-system.cpu-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.cpu
          type: metrics
        metricsets:
          - cpu
        cpu.metrics:
          - percentages
          - normalized_percentages
        period: 10s
      - id: system/metrics-system.diskio-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.diskio
          type: metrics
        metricsets:
          - diskio
        diskio.include_devices: null
        period: 10s
      - id: system/metrics-system.filesystem-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.filesystem
          type: metrics
        metricsets:
          - filesystem
        period: 1m
        processors:
          - drop_event.when.regexp:
              system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - id: system/metrics-system.fsstat-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.fsstat
          type: metrics
        metricsets:
          - fsstat
        period: 1m
        processors:
          - drop_event.when.regexp:
              system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - id: system/metrics-system.load-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.load
          type: metrics
        metricsets:
          - load
        condition: ${host.platform} != 'windows'
        period: 10s
      - id: system/metrics-system.memory-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.memory
          type: metrics
        metricsets:
          - memory
        period: 10s
      - id: system/metrics-system.network-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.network
          type: metrics
        metricsets:
          - network
        period: 10s
        network.interfaces: null
      - id: system/metrics-system.process-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.process
          type: metrics
        metricsets:
          - process
        period: 10s
        process.include_top_n.by_cpu: 5
        process.include_top_n.by_memory: 5
        process.cmdline.cache.enabled: true
        process.cgroups.enabled: false
        process.include_cpu_ticks: false
        processes:
          - .*
      - id: >-
          system/metrics-system.process.summary-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.process.summary
          type: metrics
        metricsets:
          - process_summary
        period: 10s
      - id: >-
          system/metrics-system.socket_summary-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.socket_summary
          type: metrics
        metricsets:
          - socket_summary
        period: 10s
      - id: system/metrics-system.uptime-aa4e6012-e859-46a2-950c-029ece86e67b
        data_stream:
          dataset: system.uptime
          type: metrics
        metricsets:
          - uptime
        period: 10s
signed:
  data: >-
    eyJpZCI6ImY2MDI5MjdmLWRkMjMtNGE5Ny1iOTJmLWIzOWVmOGMyYjE4OSIsImFnZW50Ijp7ImZlYXR1cmVzIjp7fSwicHJvdGVjdGlvbiI6eyJlbmFibGVkIjpmYWxzZSwidW5pbnN0YWxsX3Rva2VuX2hhc2giOiJ4QkVibTJxWnFJOGZKMFZ1NUo0NEgycEVvZkpUSU9VWERMUDE0cGhOaWlJPSIsInNpZ25pbmdfa2V5IjoiTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFT2VjQkhUblVtRXVUOGh3NVNlWlUvSnlSMXFuMCtoN0tjbktpYkVtbzZENjhNS2hZQzdpZXB3UUh1NHVEN05kcnVFWkFvN0RIQmQzTG80TDZFdDBJT0E9PSJ9fSwiaW5wdXRzIjpbeyJpZCI6ImxvZ2ZpbGUtc3lzdGVtLWFhNGU2MDEyLWU4NTktNDZhMi05NTBjLTAyOWVjZTg2ZTY3YiIsIm5hbWUiOiJzeXN0ZW0tMiIsInJldmlzaW9uIjoxLCJ0eXBlIjoibG9nZmlsZSJ9LHsiaWQiOiJqb3VybmFsZC1zeXN0ZW0tYWE0ZTYwMTItZTg1OS00NmEyLTk1MGMtMDI5ZWNlODZlNjdiIiwibmFtZSI6InN5c3RlbS0yIiwicmV2aXNpb24iOjEsInR5cGUiOiJqb3VybmFsZCJ9LHsiaWQiOiJ3aW5sb2ctc3lzdGVtLWFhNGU2MDEyLWU4NTktNDZhMi05NTBjLTAyOWVjZTg2ZTY3YiIsIm5hbWUiOiJzeXN0ZW0tMiIsInJldmlzaW9uIjoxLCJ0eXBlIjoid2lubG9nIn0seyJpZCI6InN5c3RlbS9tZXRyaWNzLXN5c3RlbS1hYTRlNjAxMi1lODU5LTQ2YTItOTUwYy0wMjllY2U4NmU2N2IiLCJuYW1lIjoic3lzdGVtLTIiLCJyZXZpc2lvbiI6MSwidHlwZSI6InN5c3RlbS9tZXRyaWNzIn1dfQ==
  signature: >-
    MEUCIQDxaeJrjBOZ+nGmRyY5HFFFzL2IvrLWiXXe0paUfavn4wIgbDKuKepBw1aZWOzUt6lvqyhsB9JbZuMHCHzuCZhwsYQ=
secret_references: []
namespaces:
  - default

elastic-agent.yml

fleet:
  enabled: true

elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 06-08-2025 12:57:50
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["vixx-ubuntu"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 192.168.100.2

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

Hi @v_i_x_x

Couple things

have you tried running the elastic-agent status and inspect command from the client box?

Also You can turn up the logging at the bottom right of that screen

Also I think those logs are easier to see / debug in Discover

Go there and set the KQL bar the data_stream.dataset : "elastic_agent.filebeat"

Perhaps this will provide more information

Also if you install and Agent without Defend just windows / system does the same issue present itself?

Hello @stephenb , thank you for the assistance appreciate it.
Yeah I did a fresh install without Defend and same issue, I only have 1 integration now => the windows integration.

After restarting the agent on my windows machine

PS C:\Users\Vixx\Desktop\elastic> & "C:\Program Files\Elastic\Agent\elastic-agent.exe" status
┌─ fleet
│  └─ status: (Connected)
└─ elastic-agent
   └─ status: (HEALTHY) Running

After running

 & "C:\Program Files\Elastic\Agent\elastic-agent.exe" diagnostics

I got this => now i am not an expert but inside winlog-default => there is a file called beat-rendered-config.yml

apm: {}
features:
    features:
        fqdn:
            enabled: false
inputs:
    - data_stream:
        dataset: system.application
        type: logs
      id: winlog-system.application-aa4e6012-e859-46a2-950c-029ece86e67b
      ignore_older: 72h
      index: logs-system.application-default
      name: Application
      processors:
        - add_fields:
            fields:
                input_id: winlog-system-aa4e6012-e859-46a2-950c-029ece86e67b
            target: '@metadata'
        - add_fields:
            fields:
                dataset: system.application
                namespace: default
                type: logs
            target: data_stream
        - add_fields:
            fields:
                dataset: system.application
            target: event
        - add_fields:
            fields:
                stream_id: winlog-system.application-aa4e6012-e859-46a2-950c-029ece86e67b
            target: '@metadata'
        - add_fields:
            fields:
                id: d4fa0e12-905c-4faa-8f54-fee1769fc411
                snapshot: false
                version: 9.1.0
            target: elastic_agent
        - add_fields:
            fields:
                id: d4fa0e12-905c-4faa-8f54-fee1769fc411
            target: agent
      type: winlog
    - data_stream:
        dataset: system.security
        type: logs
      id: winlog-system.security-aa4e6012-e859-46a2-950c-029ece86e67b
      ignore_older: 72h
      include_xml: true
      index: logs-system.security-default
      name: Security
      processors:
        - add_fields:
            fields:
                input_id: winlog-system-aa4e6012-e859-46a2-950c-029ece86e67b
            target: '@metadata'
        - add_fields:
            fields:
                dataset: system.security
                namespace: default
                type: logs
            target: data_stream
        - add_fields:
            fields:
                dataset: system.security
            target: event
        - add_fields:
            fields:
                stream_id: winlog-system.security-aa4e6012-e859-46a2-950c-029ece86e67b
            target: '@metadata'
        - add_fields:
            fields:
                id: d4fa0e12-905c-4faa-8f54-fee1769fc411
                snapshot: false
                version: 9.1.0
            target: elastic_agent
        - add_fields:
            fields:
                id: d4fa0e12-905c-4faa-8f54-fee1769fc411
            target: agent
      type: winlog
    - data_stream:
        dataset: system.system
        type: logs
      id: winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b
      ignore_older: 72h
      index: logs-system.system-default
      name: System
      processors:
        - add_fields:
            fields:
                input_id: winlog-system-aa4e6012-e859-46a2-950c-029ece86e67b
            target: '@metadata'
        - add_fields:
            fields:
                dataset: system.system
                namespace: default
                type: logs
            target: data_stream
        - add_fields:
            fields:
                dataset: system.system
            target: event
        - add_fields:
            fields:
                stream_id: winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b
            target: '@metadata'
        - add_fields:
            fields:
                id: d4fa0e12-905c-4faa-8f54-fee1769fc411
                snapshot: false
                version: 9.1.0
            target: elastic_agent
        - add_fields:
            fields:
                id: d4fa0e12-905c-4faa-8f54-fee1769fc411
            target: agent
      type: winlog
outputs:
    elasticsearch:
        api_key: <REDACTED>
        bulk_max_size: 1600
        compression_level: 1
        hosts:
            - https://192.168.100.2:9200
        idle_connection_timeout: 3s
        preset: balanced
        queue:
            mem:
                events: 3200
                flush:
                    min_events: 1600
                    timeout: 10s
        ssl:
            ca_trusted_fingerprint: 10697B6F2A9AD002E9282D865103C980554602D1E8E189B8AB81D0FF7D045517
        type: elasticsearch
        worker: 1

but i don't see any data stream starting with winlog -> I have no clue if this is normal or not.

In the elastic agent log file on the client i see the same errors im seeing on the portal.

Inspect command

PS C:\Users\Vixx\Desktop\elastic> & "C:\Program Files\Elastic\Agent\elastic-agent.exe" inspect
agent:
  download:
    sourceURI: https://artifacts.elastic.co/downloads/
  features: null
  headers: null
  id: d4fa0e12-905c-4faa-8f54-fee1769fc411
  logging:
    event_data:
      to_files: true
      to_stderr: false
    level: debug
  monitoring:
    enabled: true
    http:
      buffer: null
      enabled: false
      host: localhost
      port: 6791
    logs: true
    metrics: true
    namespace: default
    pprof: null
    traces: true
    use_output: default
  protection:
    enabled: false
    signing_key: <REDACTED>
    uninstall_token_hash: <REDACTED>
fleet:
  access_api_key: <REDACTED>
  agent:
    id: d4fa0e12-905c-4faa-8f54-fee1769fc411
  enabled: true
  enrollment_token_hash: <REDACTED>
  host: localhost:5601
  hosts:
  - https://192.168.100.2:8220
  protocol: http
  replace_token_hash: <REDACTED>
  ssl:
    certificate_authorities:
    - C:\Users\Vixx\Desktop\elastic\fleet-ca.crt
    renegotiation: never
    verification_mode: full
  timeout: 10m0s
host:
  id: 8c20ae6c-9428-4e8f-b774-305332834138
id: f602927f-dd23-4a97-b92f-b39ef8c2b189
inputs:
- data_stream:
    namespace: default
  id: winlog-system-aa4e6012-e859-46a2-950c-029ece86e67b
  meta:
    package:
      name: system
      version: 2.5.2
  name: system-2
  package_policy_id: aa4e6012-e859-46a2-950c-029ece86e67b
  revision: 2
  streams:
  - condition: ${host.platform} == 'windows'
    data_stream:
      dataset: system.application
      type: logs
    id: winlog-system.application-aa4e6012-e859-46a2-950c-029ece86e67b
    ignore_older: 72h
    name: Application
  - condition: ${host.platform} == 'windows'
    data_stream:
      dataset: system.security
      type: logs
    id: winlog-system.security-aa4e6012-e859-46a2-950c-029ece86e67b
    ignore_older: 72h
    include_xml: true
    name: Security
    tags: null
  - condition: ${host.platform} == 'windows'
    data_stream:
      dataset: system.system
      type: logs
    id: winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b
    ignore_older: 72h
    name: System
  type: winlog
  use_output: default
- data_stream:
    namespace: default
  id: system/metrics-system-aa4e6012-e859-46a2-950c-029ece86e67b
  meta:
    package:
      name: system
      version: 2.5.2
  name: system-2
  package_policy_id: aa4e6012-e859-46a2-950c-029ece86e67b
  revision: 2
  streams:
  - cpu:
      metrics:
      - percentages
      - normalized_percentages
    data_stream:
      dataset: system.cpu
      type: metrics
    id: system/metrics-system.cpu-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - cpu
    period: 10s
  - data_stream:
      dataset: system.diskio
      type: metrics
    diskio:
      include_devices: null
    id: system/metrics-system.diskio-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - diskio
    period: 10s
  - data_stream:
      dataset: system.filesystem
      type: metrics
    id: system/metrics-system.filesystem-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - filesystem
    period: 1m
    processors:
    - drop_event:
        when:
          regexp:
            system:
              filesystem:
                mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
  - data_stream:
      dataset: system.fsstat
      type: metrics
    id: system/metrics-system.fsstat-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - fsstat
    period: 1m
    processors:
    - drop_event:
        when:
          regexp:
            system:
              fsstat:
                mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
  - condition: ${host.platform} != 'windows'
    data_stream:
      dataset: system.load
      type: metrics
    id: system/metrics-system.load-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - load
    period: 10s
  - data_stream:
      dataset: system.memory
      type: metrics
    id: system/metrics-system.memory-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - memory
    period: 10s
  - data_stream:
      dataset: system.network
      type: metrics
    id: system/metrics-system.network-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - network
    network:
      interfaces: null
    period: 10s
  - data_stream:
      dataset: system.process
      type: metrics
    id: system/metrics-system.process-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - process
    period: 10s
    process:
      cgroups:
        enabled: false
      cmdline:
        cache:
          enabled: true
      include_cpu_ticks: false
      include_top_n:
        by_cpu: 5
        by_memory: 5
    processes:
    - .*
  - data_stream:
      dataset: system.process.summary
      type: metrics
    id: system/metrics-system.process.summary-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - process_summary
    period: 10s
  - data_stream:
      dataset: system.socket_summary
      type: metrics
    id: system/metrics-system.socket_summary-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - socket_summary
    period: 10s
  - data_stream:
      dataset: system.uptime
      type: metrics
    id: system/metrics-system.uptime-aa4e6012-e859-46a2-950c-029ece86e67b
    metricsets:
    - uptime
    period: 10s
  type: system/metrics
  use_output: default
output_permissions:
  default:
    _elastic_agent_checks:
      cluster:
      - monitor
    _elastic_agent_monitoring:
      indices:
      - names:
        - logs-elastic_agent.apm_server-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.apm_server-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.auditbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.auditbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.cloud_defend-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.cloudbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.cloudbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.elastic_agent-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.endpoint_security-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.endpoint_security-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.filebeat_input-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.filebeat_input-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.filebeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.filebeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.fleet_server-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.fleet_server-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.heartbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.heartbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.metricbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.metricbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.osquerybeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.osquerybeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.packetbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.packetbeat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.pf_elastic_collector-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.pf_elastic_symbolizer-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.pf_host_agent-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-elastic_agent.status_change-default
        privileges:
        - auto_configure
        - create_doc
    aa4e6012-e859-46a2-950c-029ece86e67b:
      indices:
      - names:
        - logs-system.application-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-system.security-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - logs-system.system-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.cpu-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.diskio-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.filesystem-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.fsstat-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.load-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.memory-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.network-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.process-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.process.summary-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.socket_summary-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-system.uptime-default
        privileges:
        - auto_configure
        - create_doc
  fleetserver-output-6adb209b-6310-45c9-b1c2-ea06d6baf1bb: null
outputs:
  default:
    api_key: <REDACTED>
    hosts:
    - https://192.168.100.2:9200
    preset: balanced
    ssl:
      ca_trusted_fingerprint: 10697B6F2A9AD002E9282D865103C980554602D1E8E189B8AB81D0FF7D045517
    type: elasticsearch
  fleetserver-output-6adb209b-6310-45c9-b1c2-ea06d6baf1bb:
    api_key: <REDACTED>
    ssl:
      certificate_authorities:
      - |-
        -----BEGIN CERTIFICATE-----
        MIIFajCCA1KgAwIBAgIVAO2dzIdCsmxt5njR01VI+pqLFoceMA0GCSqGSIb3DQEB
        CwUAMDwxOjA4BgNVBAMTMUVsYXN0aWNzZWFyY2ggc2VjdXJpdHkgYXV0by1jb25m
        aWd1cmF0aW9uIEhUVFAgQ0EwHhcNMjUwODA2MTI1NzU0WhcNMjgwODA1MTI1NzU0
        WjA8MTowOAYDVQQDEzFFbGFzdGljc2VhcmNoIHNlY3VyaXR5IGF1dG8tY29uZmln
        dXJhdGlvbiBIVFRQIENBMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA
        qlxv7Vf4bmIc7GpZoaM8qNvXD0P0vZ9IPsUGIaUXjbg9PdLkhgocQcNh04KWDB+0
        fml7Kfr5zusmbe3r+f9Qni0zMzc8yxQTgXZdYPDx9eWj1RjRIdNWmPRx9Bf8llQz
        476kf9FZi3a6O8BkVrQ99OWiq3mmGkEpPWuY5H5pcGqIZdi5TBQGk0mwipr9vFXy
        VOwBRnRnYv11BeWyWKbPpr4IWeiE/WpIxTEnyBJuoSYFJlGYjiBlkFyDQUrljStU
        epn2tlokD/XL7KHfWmd6N6xSXaqX//f2MB7aPYZnI7up1v8xfP2XbCqANz2drsuD
        LYNoOC3u4xYWJEO2qYeP7oDD7G21UO7UJjXNXbe2VcWSYgbD1IKK8llYIB7VVDmA
        BmVOVCyQZrOsJaA1VnGP1cpQCANYGHjbjAJnauWQ6B9WQ1jTYf2BctenwMXMnJ+i
        BeTC5cqU7MJVXmejbix28TVw/q/92wzBrNpu9VoL07fTAApW9Cvr3C4K7LToUZfg
        ow0IqBNF8VwP7UybLgJGgCOrZ8wwTRh78TxFV3dNZf1x9fwWVymEg3Yqm23U42k4
        +ct7VdWqBRbAawWRZ+om6eVpOt+Slbs5coyPiXYhtMk0JPBX+fcAbL6W1snPhzTx
        1lXGeLSS2BU51WYxTfNciA1k6faocytKjNnt18Ao3oECAwEAAaNjMGEwHQYDVR0O
        BBYEFAJI5MLEBXw8fai9MeupG3vtwWSaMB8GA1UdIwQYMBaAFAJI5MLEBXw8fai9
        MeupG3vtwWSaMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMA0GCSqG
        SIb3DQEBCwUAA4ICAQAPw0dK7yZKw/UrxUS55xxaJr0b2H/pKYwQp6tkzDNznA1k
        xw+65Xtn/5qx9XHIQpi1YgvXfMrVVkXzOeal5vi3UKL7UlZ2HUkX1yr/Jikq5Z3J
        7xYqqQAZsX3HqbZiP6xyyac15DOCHQJmVt1GjfHvYQIrNnpTBXfHm017oMA/K3Dv
        EbMfrlS8kmlB/FscXgXbtKjiWw+3izy7lnWQL9w2otmBTgJ8SWf9N1YJuquCLw+r
        Mxh1vM6vhaesFQL3TCwrQJgpH5TYu2d0lWWjuRy92LcN1ceWCytqnpdBSJNybCLc
        41iQxext1jUPNs8CdtA/osIM7z7XbFBmomkdKTPZ1snlWgZuaAKk5IYpq5jrCz+L
        zPrNYGxSFQw9vt17SQlgt73m9a29xsqrtC6AvtA0QwMXDFjS58ID9pc7UDKWbl4H
        KRhU98F493qe/U369NVb4j+77KKNkiL5EZ+r2TJTb+4UUBkhTPBomwwI343NahFO
        A1jMzySu0A0pQeknNFPYqZVE2mNk+aM+n4aJpyD51xqIvreVk+dpHKdbqaJCSJud
        S/l9/NnVPhFne22cYIkx+qC46zCfmSuncnq3FesBBevGwR7NAyyu+Px+ANKkJxmC
        Z9XY1TAh1NSpCfaTmu9PhFksgI0Y5tpBtBx5ihDpjMfB7jbT47B8rHhoZaQyzw==
        -----END CERTIFICATE-----
    type: elasticsearch
path:
  config: C:\Program Files\Elastic\Agent
  data: C:\Program Files\Elastic\Agent\data
  home: C:\Program Files\Elastic\Agent\data\elastic-agent-9.1.0-2bcf7b
  logs: C:\Program Files\Elastic\Agent
revision: 2
runtime:
  arch: amd64
  native_arch: amd64
  os: windows
  osinfo:
    family: windows
    major: 10
    minor: 0
    patch: 0
    type: windows
    version: "10.0"
secret_paths: []
signed:
  data: eyJpZCI6ImY2MDI5MjdmLWRkMjMtNGE5Ny1iOTJmLWIzOWVmOGMyYjE4OSIsImFnZW50Ijp7ImZlYXR1cmVzIjp7fSwicHJvdGVjdGlvbiI6eyJlbmFibGVkIjpmYWxzZSwidW5pbnN0YWxsX3Rva2VuX2hhc2giOiJ4QkVibTJxWnFJOGZKMFZ1NUo0NEgycEVvZkpUSU9VWERMUDE0cGhOaWlJPSIsInNpZ25pbmdfa2V5IjoiTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFT2VjQkhUblVtRXVUOGh3NVNlWlUvSnlSMXFuMCtoN0tjbktpYkVtbzZENjhNS2hZQzdpZXB3UUh1NHVEN05kcnVFWkFvN0RIQmQzTG80TDZFdDBJT0E9PSJ9fSwiaW5wdXRzIjpbeyJpZCI6IndpbmxvZy1zeXN0ZW0tYWE0ZTYwMTItZTg1OS00NmEyLTk1MGMtMDI5ZWNlODZlNjdiIiwibmFtZSI6InN5c3RlbS0yIiwicmV2aXNpb24iOjIsInR5cGUiOiJ3aW5sb2cifSx7ImlkIjoic3lzdGVtL21ldHJpY3Mtc3lzdGVtLWFhNGU2MDEyLWU4NTktNDZhMi05NTBjLTAyOWVjZTg2ZTY3YiIsIm5hbWUiOiJzeXN0ZW0tMiIsInJldmlzaW9uIjoyLCJ0eXBlIjoic3lzdGVtL21ldHJpY3MifV19
  signature: MEYCIQDzMan0g1y6TGp8oL2TNUeXRXoGHXYcmMeaQJNcALJx1QIhAM9YOLC+bclBcz8IqcYh34qxjAicIi3hxiuYpBjSLST0

Hmm let me check internally ... not my area of expertise... see if I can get someone...

The dataset is system.security so you would not see them under windows, yeah confusing the input / datastrea id is not related...

the data stream would be

logs-system.security-default

appreciate it, thank you.

The dataset is system.security so you would not see them under windows, yeah confusing the input / datastrea id is not related...

the data stream would be

logs-system.security-default

Here is my windows

So do you have that Data Stream?

no, I don't have it. I only have the ones related to my ubuntu server (which is where elastic, kibana and fleet are)

and the indices is empty is this normal ?

and on my windows policy


nor the indices and nor the data streams exists.

I am using 9.1 version btw

well indices were hidden, I see the one related to ubuntu as well not the windows

Are you doing Fleet Managed or Stand Alone

And Sorry EXACTLY which integrations do you have enabled can you show me?

I am confused at this point...

I would narrow down then start to add back

Example Show


I would start with one like system logs and then start to add

I did Fleet Managed agent.
Yeah I only have at this point only 1 enabled, just because i was desprate to make it work somehow but same issue.

1 Like

I am at a point where I don't really know what to do to workaround this lol.

Assume you are using the latest version of the Integration

OK Good / Bad... Mine is working and I am not getting the panic etc...

What version of Windows?

Can you turn up the Debug and Capture some more logs.... which I showed above

Also with your Screen Shot we can see the whole message

I will see if I can get someone else...

Thank you, appreciate it.

Yes latest version of integration and windows

echo "deb [signed-by=/usr/share/keyrings/elastic.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-9.x.list

PS C:\Users\Vixx\Desktop\elastic> systeminfo

Host Name: WIN11
OS Name: Microsoft Windows 11 Pro
OS Version: 10.0.26100 N/A Build 26100

Agent set to Debug from here unless its wrong.

After restarting Agent

In Discover

Full Json message if needed.

{
  "_index": ".ds-logs-elastic_agent.filebeat-default-2025.08.06-000001",
  "_id": "AZiAT5NDsC2pQT-WA4Jd",
  "_version": 1,
  "_source": {
    "agent": {
      "name": "Win11",
      "id": "d4fa0e12-905c-4faa-8f54-fee1769fc411",
      "type": "filebeat",
      "ephemeral_id": "dc989db7-a34d-41e9-a922-837135d0a009",
      "version": "9.1.0"
    },
    "service.name": "filebeat",
    "log": {
      "file": {
        "path": "C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-9.1.0-2bcf7b\\logs\\elastic-agent-20250806-5.ndjson",
        "vol": "3931281995",
        "idxlo": "2032",
        "idxhi": "589824",
        "fingerprint": "6f75f7590df56d174404076b150de5ae1e36f12c72edaa1dd915eb4f59582274"
      },
      "offset": 96673,
      "source": "winlog-default"
    },
    "elastic_agent": {
      "id": "d4fa0e12-905c-4faa-8f54-fee1769fc411",
      "version": "9.1.0",
      "snapshot": false
    },
    "message": "Input 'winlog' failed with: input winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b failed: input panic with: name winlog-system_system-aa4e6012-e859-46a2-950c-029ece86e67b already used\ngoroutine 197 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x5e\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).runSource.func1()\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:172 +0x58\npanic({0x891a180?, 0xc0030bc410?})\n\truntime/panic.go:792 +0x132\ngithub.com/elastic/elastic-agent-libs/monitoring.panicErr(...)\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:287\ngithub.com/elastic/elastic-agent-libs/monitoring.(*Registry).Add(0xc00015ab80, {0xc0030a8200?, 0xc0030c8120?}, {0xaa33860, 0xc0030ca080}, 0x0?)\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:185 +0xca\ngithub.com/elastic/elastic-agent-libs/monitoring.(*Registry).NewRegistry(0xc00015ab80, {0xc0030a8200, 0x39}, {0x0?, 0xaa0af10?, 0x1?})\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:94 +0x155\ngithub.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry({0x9b5d6aa, 0x6}, {0xc00177f7c0, 0x39}, 0x0)\n\tgithub.com/elastic/beats/v7/libbeat/monitoring/inputmon/input.go:73 +0x36f\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.newInputMetrics({0xc002ca4404, 0x6}, {0xc00177f7c0?, 0xc002000f50?})\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/metrics.go:79 +0x56\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.(*winEventLog).Open(0xc001cd7040, {{0x0, 0x0}, 0x0, {0x0, 0x0, 0x0}, {0x0, 0x0}})\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/wineventlog.go:167 +0xda\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.Run({0xaa3f240, 0xc002cbb080}, {0xaad6d88, 0xc002000f50}, {0xaaf56a0, 0xc001cd7040}, {{0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/runner.go:80 +0x331\ngithub.com/elastic/beats/v7/filebeat/input/winlog.winlogInput.Run({}, {0xc00309cfb0, {0xc002cb8100, 0x74}, {0xc00177f7c0, 0x39}, {0x9b5d6aa, 0x6}, {{0x9b667c6, 0x8}, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/filebeat/input/winlog/input.go:103 +0x3ff\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).runSource(_, {0xc00309cfb0, {0xc002cb8100, 0x74}, {0xc00177f7c0, 0x39}, {0x9b5d6aa, 0x6}, {{0x9b667c6, 0x8}, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:196 +0x483\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).Run.func1()\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:151 +0x2f2\ngithub.com/elastic/go-concert/unison.(*MultiErrGroup).Go.func1()\n\tgithub.com/elastic/go-concert@v0.3.0/unison/multierrgroup.go:42 +0x66\ncreated by github.com/elastic/go-concert/unison.(*MultiErrGroup).Go in goroutine 196\n\tgithub.com/elastic/go-concert@v0.3.0/unison/multierrgroup.go:40 +0x76\n",
    "log.logger": "input.winlog",
    "log.origin": {
      "file.line": 164,
      "function": "github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1",
      "file.name": "compat/compat.go"
    },
    "input": {
      "type": "filestream"
    },
    "component": {
      "binary": "filebeat",
      "id": "winlog-default",
      "type": "winlog",
      "dataset": "elastic_agent.filebeat"
    },
    "@timestamp": "2025-08-06T16:55:44.100Z",
    "ecs": {
      "version": "8.0.0"
    },
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "elastic_agent.filebeat"
    },
    "host": {
      "hostname": "Win11",
      "os": {
        "build": "26100.4770",
        "kernel": "10.0.26100.4768 (WinBuild.160101.0800)",
        "name": "Windows 11 Pro",
        "family": "windows",
        "type": "windows",
        "version": "10.0",
        "platform": "windows"
      },
      "ip": [
        "fe80::c862:115e:3f89:499e",
        "192.168.100.100",
        "fe80::bbbe:8b18:205a:2477",
        "172.27.198.127",
        "fe80::95d1:5917:4087:19b1",
        "172.22.0.1"
      ],
      "name": "win11",
      "id": "8c20ae6c-9428-4e8f-b774-305332834138",
      "mac": [
        "00-15-5D-01-64-02",
        "00-15-5D-01-64-05",
        "00-15-5D-47-7D-58"
      ],
      "architecture": "x86_64"
    },
    "log.level": "error",
    "id": "winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b",
    "event": {
      "agent_id_status": "verified",
      "ingested": "2025-08-06T16:55:56Z",
      "dataset": "elastic_agent.filebeat"
    }
  },
  "fields": {
    "elastic_agent.version": [
      "9.1.0"
    ],
    "component.binary": [
      "filebeat"
    ],
    "host.os.name.text": [
      "Windows 11 Pro"
    ],
    "host.hostname": [
      "Win11"
    ],
    "host.mac": [
      "00-15-5D-01-64-02",
      "00-15-5D-01-64-05",
      "00-15-5D-47-7D-58"
    ],
    "component.id": [
      "winlog-default"
    ],
    "host.os.version": [
      "10.0"
    ],
    "host.os.name": [
      "Windows 11 Pro"
    ],
    "log.level": [
      "error"
    ],
    "agent.name": [
      "Win11"
    ],
    "host.name": [
      "win11"
    ],
    "id": [
      "winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b"
    ],
    "event.agent_id_status": [
      "verified"
    ],
    "host.os.type": [
      "windows"
    ],
    "log.source": [
      "winlog-default"
    ],
    "input.type": [
      "filestream"
    ],
    "log.offset": [
      96673
    ],
    "data_stream.type": [
      "logs"
    ],
    "host.architecture": [
      "x86_64"
    ],
    "log.origin.function": [
      "github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1"
    ],
    "agent.id": [
      "d4fa0e12-905c-4faa-8f54-fee1769fc411"
    ],
    "ecs.version": [
      "8.0.0"
    ],
    "agent.version": [
      "9.1.0"
    ],
    "host.os.family": [
      "windows"
    ],
    "log.file.vol": [
      "3931281995"
    ],
    "log.logger": [
      "input.winlog"
    ],
    "host.os.build": [
      "26100.4770"
    ],
    "host.ip": [
      "fe80::c862:115e:3f89:499e",
      "192.168.100.100",
      "fe80::bbbe:8b18:205a:2477",
      "172.27.198.127",
      "fe80::95d1:5917:4087:19b1",
      "172.22.0.1"
    ],
    "agent.type": [
      "filebeat"
    ],
    "host.os.kernel": [
      "10.0.26100.4768 (WinBuild.160101.0800)"
    ],
    "component.dataset": [
      "elastic_agent.filebeat"
    ],
    "elastic_agent.snapshot": [
      false
    ],
    "host.id": [
      "8c20ae6c-9428-4e8f-b774-305332834138"
    ],
    "log.origin.file.line": [
      164
    ],
    "service.name": [
      "filebeat"
    ],
    "elastic_agent.id": [
      "d4fa0e12-905c-4faa-8f54-fee1769fc411"
    ],
    "data_stream.namespace": [
      "default"
    ],
    "log.file.idxhi": [
      "589824"
    ],
    "log.file.idxlo": [
      "2032"
    ],
    "message": [
      "Input 'winlog' failed with: input winlog-system.system-aa4e6012-e859-46a2-950c-029ece86e67b failed: input panic with: name winlog-system_system-aa4e6012-e859-46a2-950c-029ece86e67b already used\ngoroutine 197 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x5e\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).runSource.func1()\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:172 +0x58\npanic({0x891a180?, 0xc0030bc410?})\n\truntime/panic.go:792 +0x132\ngithub.com/elastic/elastic-agent-libs/monitoring.panicErr(...)\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:287\ngithub.com/elastic/elastic-agent-libs/monitoring.(*Registry).Add(0xc00015ab80, {0xc0030a8200?, 0xc0030c8120?}, {0xaa33860, 0xc0030ca080}, 0x0?)\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:185 +0xca\ngithub.com/elastic/elastic-agent-libs/monitoring.(*Registry).NewRegistry(0xc00015ab80, {0xc0030a8200, 0x39}, {0x0?, 0xaa0af10?, 0x1?})\n\tgithub.com/elastic/elastic-agent-libs@v0.20.0/monitoring/registry.go:94 +0x155\ngithub.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry({0x9b5d6aa, 0x6}, {0xc00177f7c0, 0x39}, 0x0)\n\tgithub.com/elastic/beats/v7/libbeat/monitoring/inputmon/input.go:73 +0x36f\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.newInputMetrics({0xc002ca4404, 0x6}, {0xc00177f7c0?, 0xc002000f50?})\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/metrics.go:79 +0x56\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.(*winEventLog).Open(0xc001cd7040, {{0x0, 0x0}, 0x0, {0x0, 0x0, 0x0}, {0x0, 0x0}})\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/wineventlog.go:167 +0xda\ngithub.com/elastic/beats/v7/winlogbeat/eventlog.Run({0xaa3f240, 0xc002cbb080}, {0xaad6d88, 0xc002000f50}, {0xaaf56a0, 0xc001cd7040}, {{0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/winlogbeat/eventlog/runner.go:80 +0x331\ngithub.com/elastic/beats/v7/filebeat/input/winlog.winlogInput.Run({}, {0xc00309cfb0, {0xc002cb8100, 0x74}, {0xc00177f7c0, 0x39}, {0x9b5d6aa, 0x6}, {{0x9b667c6, 0x8}, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/filebeat/input/winlog/input.go:103 +0x3ff\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).runSource(_, {0xc00309cfb0, {0xc002cb8100, 0x74}, {0xc00177f7c0, 0x39}, {0x9b5d6aa, 0x6}, {{0x9b667c6, 0x8}, ...}, ...}, ...)\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:196 +0x483\ngithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor.(*managedInput).Run.func1()\n\tgithub.com/elastic/beats/v7/filebeat/input/v2/input-cursor/input.go:151 +0x2f2\ngithub.com/elastic/go-concert/unison.(*MultiErrGroup).Go.func1()\n\tgithub.com/elastic/go-concert@v0.3.0/unison/multierrgroup.go:42 +0x66\ncreated by github.com/elastic/go-concert/unison.(*MultiErrGroup).Go in goroutine 196\n\tgithub.com/elastic/go-concert@v0.3.0/unison/multierrgroup.go:40 +0x76\n"
    ],
    "component.type": [
      "winlog"
    ],
    "event.ingested": [
      "2025-08-06T16:55:56.000Z"
    ],
    "@timestamp": [
      "2025-08-06T16:55:44.100Z"
    ],
    "log.origin.file.name": [
      "compat/compat.go"
    ],
    "host.os.platform": [
      "windows"
    ],
    "data_stream.dataset": [
      "elastic_agent.filebeat"
    ],
    "log.file.path": [
      "C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-9.1.0-2bcf7b\\logs\\elastic-agent-20250806-5.ndjson"
    ],
    "agent.ephemeral_id": [
      "dc989db7-a34d-41e9-a922-837135d0a009"
    ],
    "log.file.fingerprint": [
      "6f75f7590df56d174404076b150de5ae1e36f12c72edaa1dd915eb4f59582274"
    ],
    "event.dataset": [
      "elastic_agent.filebeat"
    ]
  }
}

@stephenb I added the full json message which contains the full error message, and version of windows and filebeat and ect.

1 Like

If it can be helpfull