Failed to parse field [[observer.hostname]]

[eO2H_LG]

Hello All,

Hoping somebody can assist me with the following error. It seems some of my DHCP data is failing to ingest due to a parsing issue with the observer.hostname field.

Below is the error output from logstash-plain.log

[2022-01-06T11:27:33,527][WARN ][logstash.outputs.elasticsearch][main][0333efb347b760d89aa286d374f72b4afb134c9060bf796e12c73a5540111601] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"mcafee-1.0", :routing=>nil}, {"host.os.family"=>"windows", "network.transport"=>"udp", "timestamp"=>"01/06/22,11:27:08", "network.protocol"=>"dhcpv4", "event.category"=>"network, network_traffic", "event.type"=>"connection, protocol", "agent"=>{"hostname"=>"DHCP-SERVER-1", "id"=>"aace8503-d37a-40b7-806a-4e4830c64dff", "ephemeral_id"=>"1e604d93-9c73-4871-ae70-25941c8dc08f", "name"=>"DHCP-SERVER-1", "type"=>"filebeat", "version"=>"7.16.2"}, "process.name"=>"microsoft.dhcp", "dhcpv4.client_ip"=>"192.168.5.42", "dhcpv4.id"=>"31", "input"=>{"type"=>"log"}, "log"=>{"offset"=>6294498, "file"=>{"path"=>"C:\\Windows\\Stsyem32\\dhcp\\DhcpSrvLog-Thu.log"}}, "dhcpv4.option.hostname"=>"Computer1.domain.com", "event.dataset"=>"dhcpv4", "network.type"=>"ipv4", "message"=>"31,01/06/22,11:27:08,DNS Update Failed,192.168.5.42,Computer1.domain.com,,,0,6,,,,,,,,,9017", "dhcpv4.transaction_id"=>"0", "dhcpv4.option.message"=>"9017", "host.os.platform"=>"windows", "observer.hostname"=>{"name"=>"DHCP-SERVER-1"}, "dhcpv4.option.message_type"=>"DNS Update Failed", "event.kind"=>"event", "@timestamp"=>2022-01-06T16:27:08.000Z, "ecs"=>{"version"=>"1.12.0"}, "dhcpv4.op_code"=>"No Quarantine Information Probation Time", "dhcpv4.hardware_type"=>"Ethernet", "host.os.name"=>"windows", "@version"=>"1", "tags"=>["beats_input_codec_plain_applied", "DHCP", "_grokparsefailure"]}], :response=>{"index"=>{"_index"=>"mcafee-1.0", "_type"=>"_doc", "_id"=>"randomID", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [observer.hostname] of type [text] in document with id 'randomID'. Preview of field's value: '{name=DHCP-SERVER-1}'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:913"}}}}}

[Badger]

See this answer. Once you have indexed a document in which [observer.hostname] is a string, any event in which [observer.hostname] is an object (a hash) will be rejected.

[eO2H_LG]

Hi Badger,

Thank you so much for the link. My question is as all of this data can be re-ingested again is there a way to convert or force [observer.hostname] to avoid these errors each time the index rolls over.

[Badger]

That post discusses a couple of ways to do that.