Getting timestamp field from XML attribute

Elastic Stack Logstash
7

Thanks for the explanation - very helpful! My config now looks like this:

input {
  file {
    path => [ "C:/temp/SCAP/*.xml" ]
    start_position => "beginning"
    codec => multiline {
      pattern => "^ZsExDrC" 
      what => "previous" 
      negate => true 
      auto_flush_interval => 2
      max_lines => 50000
    }
  }
}

filter {
  xml {
    namespaces => {
    "cdf" => "http://checklists.nist.gov/xccdf/1.2" 
    "xsi" => "http://www.w3.org/2001/XMLSchema-instance" 
    "dc" => "http://purl.org/dc/elements/1.1/"
    }
    source => "message"
    target => "doc"
    xpath => { 
    "/cdf:Benchmark/cdf:title/text()" => "benchmark"
    "/cdf:Benchmark/cdf:plain-text[@id='release-info']/text()" => "release-info"
    "/cdf:Benchmark/cdf:TestResult/@start-time" => "[@metadata][timestamp]"
    "/cdf:Benchmark/cdf:TestResult/cdf:target/text()" => "host.name"
    "/cdf:Benchmark/cdf:TestResult/cdf:target-address[normalize-space()][1]/text()" => "host.ip"
    "/cdf:Benchmark/cdf:TestResult/cdf:target-facts/cdf:fact[@name='urn:scap:fact:asset:identifier:os_name']/text()" => "host.os.name"
    "/cdf:Benchmark/cdf:TestResult/cdf:target-facts/cdf:fact[@name='urn:scap:fact:asset:identifier:processor']/text()" => "host.cpu"
    "/cdf:Benchmark/cdf:TestResult/cdf:target-facts/cdf:fact[@name='urn:scap:fact:asset:identifier:processor_mhz']/text()" => "host.cpu.speed"
    "/cdf:Benchmark/cdf:TestResult/cdf:target-facts/cdf:fact[@name='urn:scap:fact:asset:identifier:physical_memory']/text()" => "host.memory"
    "/cdf:Benchmark/cdf:TestResult/cdf:score[1]/text()" => "vulnerability.score.base"
    }
  }
  date { match => [ "[@metadata][timestamp][0]", "YYYY-MM-dd'T'HH:mm:ss" ] }
}

output {
  elasticsearch {
  hosts => ["localhost:9200"]
  index => "scap-results-%{+YYYY.MM.dd}"
  }
}

However, I'm still not getting any timestamp field or any other parsed fields. It just ingests into an index with zero documents. I do get this warning in logstash-plain.log (I added line breaks to make it more legible):

[2022-01-18T13:17:56,173][WARN ][logstash.outputs.elasticsearch][scap-results]
[fad371fb3a2f1bced415913c622407598fadd3ce093c68958e81938693f4259c] 
Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil,
:_index=>"scap-results-2022.01.17", :routing=>nil}, {
"@timestamp"=>2022-01-17T16:15:01.000Z, "host.ip"=>["192.168.40.208"],
"message"=>"<?xml version=\"1.0\" ....<the full XML document>...............
"tags"=>["multiline"], "benchmark"=>["Windows 10 Security Technical Implementation Guide"], "host.cpu"=>["AMD A6-5400K APU with Radeon(tm) HD Graphics   "], "host"=>"Finlandia", "host.memory"=>["16384"]...........
"error"=>{"type"=>"illegal_argument_exception", 
"reason"=>"can't merge a non object mapping [doc.Value.value] with an object mapping"}}}}

From that, you can see that it is in fact parsing the fields (for instance, the host.ip, host.cpu, etc. as well as the @timestamp), but it is not able to index it. I found this related discussion, but am not clear on how to use that solution, since that post seems more to do with machine learning jobs. In another discussion, @xeraa said:

Either have a concrete value or a subdocument in a field, but don't mix them.

But I don't understand this. My fields have concrete values (from either the XML element or attribute).