Error in Powershell during Config

Elastic Stack Beats
1

I have been following the tutorial to get Winlogbeat installed, and I got stuck on this bit:
https://www.elastic.co/guide/en/beats/winlogbeat/master/winlogbeat-template.html#load-template-manually.
Whenever I run this command:

PS > .\winlogbeat.exe setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

I get this error:

Get-Process : Parameter cannot be processed because the parameter name 'E' is ambiguous. Possible matches include:
-ErrorAction -ErrorVariable.
At line:1 char:40
+ PS > .\winlogbeat.exe setup --template -E output.logstash.enabled=fal ...
+                                        ~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.GetProcessCommand

Any clues?

2

When I try a different method: https://www.elastic.co/guide/en/beats/winlogbeat/1.3/winlogbeat-template.html

I ran this code:

PS C:\Program Files\Winlogbeat> Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri http://localhost:9200/_template/winlogbeat?pretty

and get this error:

Get-Process : A positional parameter cannot be found that accepts argument 'Invoke-WebRequest'.
At line:1 char:1
+ PS .\winlogbeat.exe> Invoke-WebRequest -Method Put -InFile winlogbeat ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

if I remove the path to winlogbeat I get this:

Get-Process : A parameter cannot be found that matches parameter name 'Method'.
At line:1 char:22
+ PS Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Ur ...
+                      ~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand
3

I am also getting this error when doing .\winlogbeat.exe

Program 'winlogbeat.exe' failed to run: The specified executable is not a valid application for this OS platform.At
line:1 char:2
+  C:\Software\winlogbeat\winlogbeat.exe test config -c .\winlogbeat.ym ...
+  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At line:1 char:2
+  C:\Software\winlogbeat\winlogbeat.exe test config -c .\winlogbeat.ym ...
+  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed
4

Even when setting up Kibana dashboard I get an error:

PS > .\winlogbeat.exe setup --dashboards

Get-Process : A positional parameter cannot be found that accepts argument '--dashboards'.
At line:1 char:1
+ PS > .\winlogbeat.exe setup --dashboards
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand
5

I had to download the JSON file from the web (github) then pasted into the file. I then ran the command:

Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri http://localhost:9200/_template/winlogbeat?pretty

and worked fine.
Now I am trying to set the Kibana dashboard, but it either says:

Get-Process : A parameter cannot be found that matches parameter name 'dashboards'.
At line:1 char:25
+ PS > .\winlogbeat setup -dashboards
+                         ~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

or it says:

Program 'winlogbeat.exe' failed to run: The specified executable is not a valid application for this OS platform.At line:1 char:1
+ .\winlogbeat setup --dashboards
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At line:1 char:1
+ .\winlogbeat setup --dashboards
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed

When installed the dashboard for Logstash output using this command:

PS > .\winlogbeat.exe setup -e `
  -E output.logstash.enabled=false `
  -E output.elasticsearch.hosts=['localhost:9200'] `
  -E output.elasticsearch.username=winlogbeat_internal `
  -E output.elasticsearch.password=YOUR_PASSWORD `
  -E setup.kibana.host=localhost:5601

I get this error:

Get-Process : Parameter cannot be processed because the parameter name 'e' is ambiguous. Possible matches include:
-ErrorAction -ErrorVariable.
At line:1 char:29
+ PS > .\winlogbeat.exe setup -e `
+                             ~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.GetProcessCommand

Also for some reason my winlogbeat did not come with a scripts folder for me to install dashboards as stated here:
https://www.elastic.co/guide/en/beats/winlogbeat/5.0/winlogbeat-sample-dashboards.html

Also when trying to run winlogbeat I get this error:

Start-Service : Service 'winlogbeat (winlogbeat)' cannot be started due to the following error: Cannot start service winlogbeat on computer '.'.
At line:1 char:1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

This is my config file:

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: Security
  - name: System

setup.template.settings:
  index.number_of_shards: 3

setup.dashboards.enabled: true

setup.kibana:
  host: "IP:5601"

output.logstash:
  hosts: ["IP:5000"]
6

Hello ?

7

I see that you are mixing resources and documentation from wildly different versions of Beats. From master (beta), 1.3, 5.0. There is a lot of evolution and breaking changes so its normal that some procedures can't be followed.

First things first, which version of Winlogbeat are you using?

I suggest you use the latest relase, 6.3.1 at this moment, and stick to the latest documentation.

Latest Winlogbeat : https://www.elastic.co/downloads/beats/winlogbeat

Latest Documentation: https://www.elastic.co/guide/en/beats/winlogbeat/current/index.html

8

Thank you!
I reinstalled and followed the doc and it worked, I was able to start winlogbeat.
However, now on Kibana dashboard when I select winlogbeat-* I get a red bar at the top saying:

No matching indices found: No indices match pattern "winlogbeat-*"

any clue of what is going on? could it be configuration ?

I get the same for Filebeat

9

You need to run winlogbeat first, so it sends events to Elasticsearch. Until then, the index will not exist.

10

I have ran winlogbeat with Start-Service winlogbeat
I am using Logstash so would it be sent to Logstash and the indices will be named logstash?

For that red bar, the Error is:

Error: No matching indices found: No indices match pattern "winlogbeat-*"
    at http://10.130.233.242:5601/bundles/commons.bundle.js:3:918983
    at processQueue (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:134252)
    at http://10.130.233.242:5601/bundles/vendors.bundle.js:133:135201
    at Scope.$digest (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:146077)
    at Scope.$apply (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:148856)
    at done (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:101124)
    at completeRequest (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:106024)
    at XMLHttpRequest.xhr.onload (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:106783)
11

Did you set up the Beats Input module in logstash?

https://www.elastic.co/guide/en/logstash/6.3/plugins-inputs-beats.html

If so, it will be using the winlogbeat indices.

12

Yes I had a file within conf.d named logstash-beats.conf and it has this config:

input {
  beats {
    port => 5000
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}
13

Then, if beats is configured to forward events to LS, you should be seeing events in Kibana. Can you check that winlogbeat is forwarding events and logstash is receiving them?

14

https://pastebin.com/raw/J3iDDv9v

here is the logs from winlogbeat

It seems like the client is having issues with Logstash, but I have no idea why because all the configs seems to be fine

seems like filebeat and metricbeat log file has a similar output.

15

Are you sure there is no firewall blocking the 5000 port in the logstash server?

16

Not sure if blocking, but it wasn't in the list. So, I just added it to the list, do I have to restart everything or?

This is what I get from the logs now:

2018-07-10T11:11:37.737+0100 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":2156,"time":{"ms":16}},"total":{"ticks":21624,"time":{"ms":109},"value":21624},"user":{"ticks":19468,"time":{"ms":93}}},"info":{"ephemeral_id":"e5347566-6fef-4735-add4-f90bd7973a0e","uptime":{"ms":3000172}},"memstats":{"gc_next":37620576,"memory_alloc":31959552,"memory_total":824947000,"rss":389120}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"batches":4,"failed":4156,"total":4156},"read":{"errors":1},"write":{"bytes":738146}},"pipeline":{"clients":3,"events":{"active":4119,"retry":6204}}}}}}

This is the same on filebeat and metricbeat

17

It will retry sending events after a while, but if you're testing you can stop winlogbeat, remove the file in C:\ProgramData\winlogbeat\.winlogbeat.yml and launch it again, so it starts sending events from the beginning of the event log.

18

you mean delete the log file right not the actual config file

19

In C:\ProgramData\winlogbeat\ you should have a file starting with a dot, .winlogbeat.yml which is the registry file where winlogbeat stores the bookmarks to remember the already-read position into an event log between executions. It you remove it, it will start reading events from the beginning of the event log.

20

So I just made a new directory to send logs to get fresh ones and it still gives No matching indices found for filebeat, metric and winlogbeat.
Logstash outputs this:

`

version:1 host:10.130.233.242 tags:_grokparsefailure @timestamp:July 10th 2018, 11:39:20.326 message:\xD5 (\u0006x}\xD2*^\xEA$\xD6\e@\x9C&\u0011\xADi6\xA66\xC7k\xE7N\xF8)\xABMa\u0014^\xD2\xD2W\x90\x92\xA4\xEE5\xED>\x83\x80\xA8δ\xD8/g\xBA}\xDA*bJO\xE7:H4s\xEA\xB9\u001A;\xA2\xBB\x85\xC6v\x98v\xC1\u0013?C\x88\xF3\x84\xB5¾\u0006\x8Af\vI\xAD\xCE?\,\xBD\x89?\xAEZ\xE2\eW\xFA[\xC0\xB3C\vT\xD2\e\xB0\xFB\xACՎ\u0015\xC7.\xD5,\u0014\xE0\x82Z\xAF\xDEQP\x9EO\xBF\u0004\xC8>G\x88\xF3\u0000Jn\x81\x81\xE7\xE9Uq?T\u001A\xD7I\x95E\xC2gY\x8A#!si,\x9B66\u0000\xC0\xEA\xF3\xE8df\u0002+\2\x8B\xDC@x\xAD\xAC\bS\xA1=\xFE\u001A\xA7\xB7A\xC3c\xC7\xEF<\u0001\xB1/XE\x8C\u0019b\xE4y\xC3\u0003e\xE8\xDA\aĞC_D\x9B\xD0\u0018m\xA3b\x88\x8AԢ\x97i\xF7\xABc\u001EƲ\xA4\a\x87\x8D\xAD\xABW\xE0\u0001\xB3\xE7\xADb6\b\u001E\xABΆۃ\x8C\xA4\x9B\x9F\xB4]`\xEA[U_D\xBFd4\xA13\xE4\u000F port:32,800 type:random_logs _id:BrHGg2QBHw24IkY_MwGV _type:doc _index:logstash-2018.07.10 _score: -

and

@version:1 host:10.130.233.242 tags:_grokparsefailure @timestamp:July 10th 2018, 11:39:51.303 message:\xDD\u007F\u000F$\xB3R\x8BP\xD7#\xAFL\x8B\xF3\x92T\xBA\xCD-\u0012\u000Fu\x82\f\xBC\xFE;\xC2s\x98$L5o\u0000\xCBE\x91\x84 port:32,810 type:random_logs _id:0bHGg2QBHw24IkY_sA4S _type:doc _index:logstash-2018.07.10 _score:

The errors on the winlogbeat log is:

2018-07-10T11:22:57.345+0100	ERROR	logstash/async.go:235	Failed to publish events caused by: client is not connected
|2018-07-10T11:23:59.535+0100|ERROR|logstash/async.go:235|Failed to publish events caused by: read tcp IP:55144->IP:5000: i/o timeout|
|---|---|---|---|
|2018-07-10T11:23:59.539+0100|ERROR|logstash/async.go:235|Failed to publish events caused by: read tcp IP:55144->IP:5000: i/o timeout|
|2018-07-10T11:23:59.540+0100|ERROR|logstash/async.go:235|Failed to publish events caused by: read tcp IP:55144->IP:5000: i/o timeout|
2018-07-10T11:24:00.602+0100	ERROR	pipeline/output.go:92	Failed to publish events: client is not connected

I will try

I dont have any file starting with a dot (.winlogbeat.yml) I do have just winlogbeat.yml. The only file I have that starts with a dot is .build_hash