Elastic-agent to logstash mapper_parsing_exception with windows system integration

Elastic Stack Elastic Agent
1

Hi,
I am trying to send windows events via an elastic-agent (8.6.0) (with fleet in 8.6.0) to logstash (8.6.0).
the eleastic-agent is configured with an agent policy that has a system integration configured as follows:

image
image667×625 46.8 KB

and the pipeline:

input {
  elastic_agent {
    port => "5047"
    ssl => true
    ssl_key => '/etc/pki/logstash.pkcs8.key'
    ssl_certificate => '/etc/pki/logstash.crt'
  }
}

filter {
  if ([input][type] =~ "winlog") {
    mutate {
      remove_field => ["[winlog][logon][id]",
                       "[winlog][provider_guid]",
                       "[winlog][record_id]",
                       "[winlog][event_data][LogonType ]",
                       "[winlog][event_data][AccessList]",
                       "[winlog][event_data][AccessMask]",
                       "[winlog][event_data][HandleId]",
                       "[winlog][event_data][ElevatedToken]",
                       "[winlog][event_data][ResourceAttributes]",
                       "[winlog][event_data][ImpersonationLevel]",
                       "[winlog][event_data][KeyType]",
                       "[winlog][event_data][SamAccountName]",
                       "[winlog][event_data][KeyLength]",
                       "[winlog][event_data][Operation]",
                       "[winlog][event_data][Binary ]",
                       "[winlog][event_data][NewState]",
                       "[winlog][event_data][SubjectLogonId]",
                       "[winlog][event_data][CountNew]",
                       "[winlog][event_data][CountOld]",
                       "[winlog][event_data][DwordVal]",
                       "[winlog][event_data][DriverNameLength]",
                       "[winlog][event_data][FailureNameLength]",
                       "[winlog][event_data][ExtraStringLength]",
                       "[winlog][event_data][AddressLength]",
                       "[winlog][event_data][ProcessNameLength]",
                       "[winlog][keywords]",
                       "[winlog][process][thread][id]",
                       "[host][os][family]",
                       "[host][os][type]",
                       "[host][os][platform]"
                      ]
    }
  }
}

output {
  elasticsearch {
    hosts => ["https://host1:9200","https://host2:9200","https://host3:9200"]
    user => 'user'
    password => "${PASSWORD}"
    cacert => '/etc/pki/ca.crt'
    ilm_rollover_alias  => "win-logs"
    ilm_pattern         => "{now/d}-000001"
    ilm_policy          => "20-days-default"
  }
}

when i enable the system and security options, i get the following error in logstash:

"error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [data_stream.dataset] of type [constant_keyword] in document. Preview of field's value: 'system.system'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"[constant_keyword] field [data_stream.dataset] only accepts values that are equal to the value defined in the mappings [system.security], but got [system.system]"}}}}

Could someone help me to understand what could cause this error or/and how to solve it?

Thanks in advance.

2

Hello,

From the error message it seems like the system.security data stream only accepts "system.security" as dataset name, and it received "system.system".
Do you see these settings in the agent policy Advanced options of the data streams?

The System integration docs are here: System | Elastic docs

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.