"message" filed missing from windows event logs

Elastic Stack Beats
1

Hi,

I am sending all the event logs to Logstash using Winlogbeat. Here is part of Winlogbeat.yml,

winlogbeat.event_logs:
  - name: Application
    level: error, warning, info

output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

Issue is that - I can see error message in Windows event viewer but there is no "message" field at all for some events when checked in Kibana. I am not able to find what the issue is. Here is Logstash config file,

input {
	beats {
		port => 5044
	}
}

filter{
	
	mutate{
		add_field =>{"app_name" => "***"}
		add_field =>{"time" => "%{@timestamp}"}
		rename => {"[beat][name]" => "source_host"}
		remove_field => ["event_data", "tags", "[beat][hostname]", "[beat][version]"]
	}
	
	
}

output {
	http {
       url=> "http://*****"
        http_method=> "post"
        format=> "json"
    }

}

Can someone please help me with the issue?

2

Winlogbeat will add a message_error field if it cannot provide a message.

3

Hi Andrew,

There is no message_error field either. It says level:error in kibana but there's no message or message_error field.

I know that Logstash by default would add message field unless we explicitly drop that field but here I don't see message field at all.

4

What version of Winlogbeat are you running? And what OS version?

If you can turn on debug logging this will give you some more detail about the events on the Winlogbeat side. The log will then contain the raw XML received from Windows so we can see if it has any rendering errors in it (these are what will be added to message_error).

logging.level: debug
logging.selectors: [eventlog, eventlog_detail, publish]
5

I am using winlogbeat 5.5.0

Here is the xml,

<Event xmlns='http://example.com'><System><Provider Name='test'/><EventID Qualifiers='1000'>0</EventID><Level>4</Level><Task>0</Task><Keywords>xxxx</Keywords><TimeCreated SystemTime='2018-07-12T16:56:54.000000000Z'/><EventRecordID>4153686</EventRecordID><Channel>Application</Channel><Computer>xxxxxx</Computer><Security/></System><EventData><Data>Timestamp Local: 7/12/2018 16:56:54 AM
Message: some message
Category: General
Priority: 3
EventId: 0
Severity: Information
Title:example
Machine: xxxxx
Application Domain: xxxxx
Process Id: 14335536
Process Name: c:\windows\system32\inetsrv\w3wp.exe
Win32 Thread Id: 324556
Thread Name: 
Extended Properties: xxxx
SessionID - xxxx
MemberId - xxxx
ClientID - xxxx
CampaignID - xxx
</Data></xxxx><RenderingInfo Culture='en-US'><Message></Message><Level>Information</Level><Task></Task><Opcode>Info</Opcode><Channel></Channel><Provider></Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>

Please let me know the issue..

6

It looks like there is no message contained in the event from Windows. Nor is there a rendering error. There might be an issue with the application that is logging the message (in particular the message text file).

As a workaround you could use Logstash to copy the event_data.Data value to message when you see this event ID value in the Application log and it does not have a message.

7

Could you please help me with this?

8

I used to drop event_data field using mutate filter in Logstash

mutate{
       remove_field => ["event_data"]
}

Now, I am not dropping this field, so in some documents I see just one filed as event_data.param1(which contains the message) but in some cases I see lot of fields event_data.param1 to event_data.param30. In this case event_data.param1 would contain an integer(not the message).

Please say if I am missing anything here. My goal is to combine all these fields into a single field "message" instead of having 30 different fields.

Thanks

9

Assuming the application creates just this one event with the problem, then I'd put a condition around the logic so that it does not affect other events.

filters {
  if ![message] and [event_data][param1] and [log_name] == "Application" and [source_name] == "test" and [event_id] == 0 {
    mutate {
      rename => {
        "[event_data][param1]" => "message"
      }
    }
  }
}

This may need some tweaking since I am not working off the real data.

10

Or if this is affecting all events being logged by this one application then you could do a hack like this to convert the parameters logged by the application to a json string and put that into message. I must say that is the opposite of what most users do; mostly they want to have each piece of data is its own field in order to do various types of analysis or aggregations.

filter {
  if ![message] and [event_data][param1] and [log_name] == "Application" and [source_name] == "test" {
    json_encode {
      source => "event_data"
      target => "message"
    }
  }
}
11

Thanks Andrew. Will try it :slight_smile:

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.