Winlogbeat 8.0.0 parsing with module

Elastic Stack Beats
,
1

Hello everyone

I have an ELK 8.0.0 series. I am testing the build of windows logs using winlogbeat 8.0.0. As I understand from the elastic documentation, the parsing of raw windows events will be by means of Security, PowerShell, Sysmon modules(Modules | Winlogbeat Reference [8.0] | Elastic). Based on this information, I collected a config.

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: Security
  - name: System
  - name: ForwardedEvents
    tags: ["forwarder"]

setup.template.settings:
  index.number_of_shards: 1

max_procs: 1
keystore.path: ${path.home}/winlogbeat.keystore

output.elasticsearch:
  hosts: ["https://server1:9200", "https://server2:9200"]
  protocol: https
  index: "winlogbeat-nvs"
  username: "***"
  password: "***"
  ssl:
    enabled: true
    verification_mode: full
    supported_protocol: [TLSv1.2, TLSv1.3]
    certificate_authorities: ${path.home}/rootCA.cer

setup.ilm.enabled: false
setup.template.overwrite: true

output.elasticsearch.pipeline: winlogbeat-%{[agent.version]}-routing

setup.template.name: 'winlogbeat'
setup.template.pattern: 'winlogbeat'
setup.template.enabled: true

But I get incomplete parsing of logs in cabana, and almost all fields are unknown field.

image
image708×736 31.6 KB

Experts, please tell me, is this a full-fledged parsing or am I missing something?

2

Someone can help me?

3

Nobody faced such a problem?

4

Studying the problem, I found in the elasticsearch.yml config that the ingest role was missing from the assigned roles. I thought that I had finally found the root of the problem, but I hurried to rejoice by specifying ingest in node.roles, the problem was not solved

My cluster consists of 3 nodes:
1 - nodes.roles: [master, data, remoting_cluster_client]
2 - nodes.roles: [master, data, remoting_cluster_client]
3 - nodes.roles: [data, remoting_cluster_client, ml, ingest]

Added a new user with full rights cluster_privilege and kibana - no result

5

What specific version are you using for Winlogbeat?

Did you restarted your cluster after adding the ingest role?

Did you load the ingest pipelines?

Did you load the template for your index?

6

Hello, leandrojmp! Thanks for the answer!

I'm using winlogbeat 8.0.0
Yes, I rebutted each node separately, as it says here

Yes, pipelines are loaded automatically, I see them in the kibana section of ingest pipelines

pipilenes
pipilenes843×623 66.4 KB

Yes, template is created automatically when creating an index. This can be seen from the winlogbeat config, which is located above

7

Even if I delete winlogbeat pipelines, nothing changes, the data in kibana remains in the same form. It seems that the parsing mechanism itself does not turn on...

8

You deleted the ingest pipelines and still got data in your cluster? Winlogbeat should fail if a ingest pipeline that it uses while sending data does not exists.

Have you tried sending it without changing the index name? Using the default one.

Do you have any errors in Winlogbeat logs?

Also, I'm not sure this is correct, this would not match your winlogbeat-nvs index, try to change it to winlogbeat-nvs.

Can you share the template winlogbeat ?

9

You deleted the ingest pipelines and still got data in your cluster? Winlogbeat should fail if a ingest pipeline that it uses while sending data does not exists.

Yes, I am deleting pipelines, but the data is still being written to the index. at that time, no errors are recorded in the winlogbeat log

Have you tried sending it without changing the index name? Using the default one.

yes, I tried writing to the standard index that is created after starting winlogbeat (index name: winlogbeat-8.0.0), but nothing has changed. There are no errors in the winlogbeat log either, even with debug logging mode enabled

Can you share the template winlogbeat ?

Yes, I can. It's too big, I put it in my git, please take a look

10

I didn't find the root of the problem, reinstalled the cluster and connected winlogbeat again, everything worked

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.