Event messages are splitted in event_data.param xy? How to fix that?

Elastic Stack Beats
1

Hello there,

I have installed ELKwin from https://www.itefix.net/elkwin - very nice solution to get the ELK-Stack running in seconds. If I'm honest, I am very new to the ELK-Stack and it is not so easy to start...

On the ELK-server I collect all the Windows event logs from all other Windows-Servers. That is running well.

Now I have installed Winlogbeat (5.0.0 alpha5) to send all that collected events to Logstash - that is also running. Winlogbeat 1.3.1 did not show all messages correctly, which I have used before.

But I have one problem - the message data is splitted into several fields, from event_data.param 1 - event_data.param 30.

How can I fix that? Or is that normal?

Here's my winlogbeat.yml:

winlogbeat.registry_file: C:/Elkwin_Free_x64/winlogbeat/.winlogbeat.yml

winlogbeat.event_logs:
- name: ForwardedEvents
  level: critical, error, warning
  forwarded: true

output.elasticsearch:

hosts: ["localhost:9200"]

logging.to_files: true
logging.files:

path: C:/Elkwin_Free_x64/winlogbeat/Logs

rotateeverybytes: 10485760 # = 10MB

keepfiles: 7

Thank you very much for your help!

2

Assuming that the events were forwarded from the original host to the collector in "RenderedText" format, then they should have a message field that contains the full text of the event. The event_data.* fields are the raw data that was provided by the application that logged the event. This is included in the event published by Winlogbeat so that you don't have to grok the message field to extract data needed for other analysis you might want to do.

To debug the issue I would add the include_xml: true configuration option so that the raw XML event that was forwarded by the original host is included. Then you can check this XML to see if it includes a RenderingInfo field with a message.

You should also enable debug logging (logging.level: debug) and check the log for problems (or post it to pastebin or gist and I can take a look).

As a general note about your config, I also recommend adding a forwarded tag events read from the ForwardedEvents log so that you can easily tell that they were forwarded. WIthout this tag it can be difficult to know since the log_name field gets set to the original log name.

It appears that the indentation is wrong in the config you posted.

winlogbeat.registry_file: 'C:/Elkwin_Free_x64/winlogbeat/.winlogbeat.yml'

winlogbeat.event_logs:
- name: ForwardedEvents
  level: critical, error, warning
  forwarded: true
  include_xml: true
  tags: [forwarded]

output.elasticsearch:
  hosts: ["localhost:9200"]

logging.level: debug
logging.to_files: true
logging.files:
  path: 'C:/Elkwin_Free_x64/winlogbeat/Logs'
  rotateeverybytes: 10485760
  keepfiles: 7
3

Thank you Andrew!

I have changed the config with yours and will check the results when some logs were created.
Good to know with the forwarded tag!

Now it's also clear to me why the data will be splitted into the "event_data.*" fields. That's true, it could be very nice to have some fields already "groked" to do some further analysis.

I will let you know about the results as soon as I have some logs here.

EDIT:

I think all is running well, perhaps I didn't look at this in detail... There is much data in the event text itself and also in the details (message). Maybe I fooled myself.

I have uploaded my text, it's too much data for here: http://textuploader.com/dsh18

So it seems to be right how the data is shown and groked in its fields. Am I right?

The log is showing no errors at this time.

EDIT again:

I have another problem. There are levels which mean the same, Error and error. But Kibana makes 2 of it which looks like in my screenshot. One pie is not shown, it's the gap in the middle. Don't know why.

log-per-server-kibana.JPG1215×170 13.2 KB 

{
      "type": "pie",
    "params": {
    "shareYAxis": true,
    "addTooltip": true,
    "addLegend": true,
    "isDonut": false,
    "spyPerPage": 10
  },
  "aggs": [
    {
      "id": "1",
      "type": "count",
      "schema": "metric",
      "params": {}
    },
    {
      "id": "2",
      "type": "terms",
      "schema": "split",
      "params": {
        "field": "computer_name",
        "exclude": {
          "pattern": "domain.local"
        },
        "size": 5,
        "order": "desc",
        "orderBy": "1",
        "row": false
      }
    },
    {
      "id": "3",
      "type": "terms",
      "schema": "segment",
      "params": {
        "field": "level",
        "size": 5,
        "order": "desc",
        "orderBy": "1"
      }
    }
  ],
  "listeners": {}
}

Is it possible to transform the error in Error? And what issue could it be that the pie of one server is not shown? Logs are available.

4

I know the problem with the missing pie, but didn't know how to solve.

The pie is missing because "level" is empty in so many events.

It is displayed in the normal view of the Windows Eventviewer, but when I switch over to the XML view it is empty or not set.

It seems to be a Windows problem - anybody knows about?

Is it possible to check the level numbers at the top and add a field with the level string?

For example look for <Level>3</Level> => add <Level>Warning</Level> under RenderingInfo?

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Microsoft-Windows-CertificateServicesClient-AutoEnrollment" Guid="{F0DB7EF8-B6F3-4005-9937-FEB77B9E1B43}" EventSourceName="AutoEnrollment" /> 
  <EventID Qualifiers="32768">64</EventID> 
  <Version>0</Version> 
  <Level>3</Level> 
  <Task>0</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x80000000000000</Keywords> 
  <TimeCreated SystemTime="2016-09-22T20:15:40.000000000Z" /> 
  <EventRecordID>151292</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="0" ThreadID="0" /> 
  <Channel>Application</Channel> 
  <Computer>server.domain.local</Computer> 
  <Security /> 
  </System>
- <EventData>
  <Data Name="Context">local system</Data> 
  <Data Name="ObjId">7a 05 23 ee 00 20 5c 35 51 c8 11 cf f7 aa 18 c8 52 58 9c bb</Data> 
  </EventData>
- <RenderingInfo Culture="en-US">
  <Message /> 
  <Level /> 
  <Task /> 
  <Opcode /> 
  <Channel /> 
  <Provider /> 
  <Keywords /> 
  </RenderingInfo>
  </Event>

If this is not working, do you know another possibility to get all eventdata out of the Windows Eventlog?

5

Sorry, for not responding earlier. I missed the notification for your response.

I'd say it's probably a Windows issue that a Level string isn't provided even though there is a numeric Level value. I think the level values are pretty much static so Winlogbeat could map the numeric value to a string when it's not provided. I suggest openning an enhancement request for this feature.

You could normalize the level value in Logstash with a mutate filter and convert them to all lowercase.

6

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.