Message filed is missing when Winlogbeat is used

Elastic Stack Beats
1

Hi,

I am using Winlogbeat to collect Windows event logs and sending output to Logstash. But the problem is that I am missing message filed and instead I am seeing event_data filed. And inside event_data field there is another field named param1. Also I can see the multiple events with the same record_number. Can someone please explain what it means?

Thanks

2

If you could give examples of what you are seeing (copy the JSON out of Kibana for the each example event) that would be helpful towards us providing an explanation.

If message is missing then there is probably a message_error field saying why.

event_data contains the raw data written by the Application that created the event. param1 is the first parameter logged in the event by the Application. Some apps use positional paramters rather than named parameters so you end up with param1, param2, ..., paramN. See Legacy Winlogbeat alias fields | Winlogbeat Reference [8.11] | Elastic

Each event log has its own record_number counter. So an event from the Application log can have the same record_number as an event from the Security log.

3

First example:(copied from logstash output)

{
    "computer_name" => "mycomputer",
            "level" => "Error",
         "log_name" => "Application",
    "record_number" => "142921",
       "event_data" => {
        "param1" => "2017-08-16 23:07:18.911 -0400 catalina-exec-478   ERROR: com.software.controller.dataserver.DataServerSessi
onService - Error response. code:401 type:DataServerNoAuth msg:No Server
 user found."
    },
             "type" => "wineventlog",
             "tags" => [
        [0] "beats_input_raw_event"
    ],
         "event_id" => 4096,
       "@timestamp" => 2017-08-17T03:07:18.000Z,
             "beat" => {
        "hostname" => "mycomputer",
            "name" => "mycomputer",
         "version" => "5.5.0"
    },
         "@version" => "1",
             "host" => "mycomputer",
      "source_name" => "Server (Dataserver)"
}

Second Example:(copied from logstash output)

{
    "computer_name" => "mycomputer",
         "keywords" => [
        [0] "Classic"
    ],
            "level" => "Error",
         "log_name" => "Application",
    "record_number" => "158968",
       "event_data" => {
        "param1" => "hr=0x80072EE2",
        "param2" => "00010001(0x00000000, 04:07:55:564 error at line 4"
    },
          "message" => "License acquisition failure",
             "type" => "wineventlog",
             "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
        
       "@timestamp" => 2017-09-10T08:08:16.000Z,
         "event_id" => 8200,
    "provider_guid" => "{C8C9-472C-A5F9-F2BDFEA0F156}",
         "@version" => "1",
             "beat" => {
        "hostname" => "mycomputer",
            "name" => "mycomputer",
         "version" => "5.5.0"
    },
             "host" => "mycomputer",
      "source_name" => "Microsoft-Windows-Security-SPP"
}

I don't see any error :frowning:

Is there any way where I can combine all the param fields into a single field?

4

The second example event looks normal (it has a message).

That first one is missing the message so I'm wondering what shows up in the Windows Event Viewer for this message? Also could you add include_xml: true to the config for the Application log. This should show what information Windows is providing to Winlogbeat for that event. There will be an additional field called xml in each event from the Application log. Can you please share an event where it's missing message and has xml.

winlogbeat.event_logs:
  - name: Application
    include_xml: true

That's typically what the message field does, but does it based on a template like User [param1] logged in at [param2]. With Logstash you should be able to concatenate the fields as you wish. I'd probably try writing a ruby filter to iterate over each event_data field and join the values.

5 
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Server (Dataserver)" /> 
  <EventID Qualifiers="0">4096</EventID> 
  <Level>2</Level> 
  <Task>5</Task> 
  <Keywords>0x80000000000</Keywords> 
  <TimeCreated SystemTime="2017-08-17T03:07:18.000000000Z" /> 
  <EventRecordID>142902</EventRecordID> 
  <Channel>Application</Channel> 
  <Computer>mycomputer</Computer> 
  <Security /> 
  </System>
- <EventData>
  <Data>2017-08-16 23:07:18.693 -0400 catalina-exec-478 ERROR WZUIZqwYJFMAAIAkfo4AAAOe: com.software.controller.dataserver.DataServerSessionService - Error response. code:401 type:DataServerNoAuth msg:No Server user found.</Data> 
  </EventData>
  </Event>

I have added include_xml: true to the config. I don't see any new logs now. How to make winlogbeat read the older logs?

In my case, message field is entirely different from what param1 and param2 have. Is it acceptable?

6

There's a file in C:\ProgramData\winlogbeat\.winlogbeat.yml that tells Winlogbeat where to resume from. You can delete that file and it will start from the beginning.

7 
  {
    "computer_name" => "mycomputer",
         "log_name" => "Application",
            "level" => "Error",
    "record_number" => "140750",
       "event_data" => {
        "param1" => "2017-08-16 23:06:48.036 -0400 catalina-exec-476   ERROR WZU
ISKwYJFMAAIAkbZkAAAOU: com.software.controller.dataserver.DataServerSessi
onService - Error response. code:401 type:DataServerNoAuth msg:No Server
 user found."
    },
             "type" => "wineventlog",
             "tags" => [
        [0] "beats_input_raw_event"
    ],
       "@timestamp" => 2017-08-17T03:06:48.000Z,
         "event_id" => 4096,
              "xml" => "<Event xmlns='http://schemas.microsoft.com/win/2004/08/e
vents/event'><System><Provider Name='Server (Dataserver)'/><EventID Qual
ifiers='0'>4096</EventID><Level>2</Level><Task>5</Task><Keywords>0x8000000000000
0</Keywords><TimeCreated SystemTime='2017-08-17T03:06:48.000000000Z'/><EventReco
rdID>140750</EventRecordID><Channel>Application</Channel><Computer>mycomputer</Computer><Security/></System><EventData><Data>2017-08-16 2
3:06:48.036 -0400 catalina-exec-476   ERROR WZUISKwYJFMAAIAkbZkAAAOU: com.software.controller.dataserver.DataServerSessionService - Error response. code:
401 type:DataServerNoAuth msg:No Server user found.\r\n</Data></EventDat
a></Event>",
             "beat" => {
        "hostname" => "mycomputer",
            "name" => "mycomputer",
         "version" => "5.5.0"
    },
         "@version" => "1",
             "host" => "mycomputer",
      "source_name" => "Server (Dataserver)"
}
8

Thanks for posting the XML as seen from Winlogbeat. Based on that information it is the application, "Server (Dataserver), that does not include a message. Is this event from Tableau? I saw this for 4096: https://community.spiceworks.com/windows_event/show/19885-tableau-server-dataserver-4096

9

Hi Andrew,

Yes this a tableau event. For security reasons I didn't include the complete error details. So, Server itself is dropping the message field. Is it acceptable?

10

I wouldn't say it's dropping the message, but more like the software authors didn't include it when they wrote logging for that event.

11

Makes sense. May be the two events that I posted earlier could be of two different kinds(one had message field and the other didn't).

12

Yes, the two events you posted previously were from separate sources. One was from "Server (Dataserver)" (Tableau) and the other from "Microsoft-Windows-Security-SPP". The source_name identifies the "thing" that logged the event. See our field docs for definitions of each field.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.