Error in Powershell during Config


(Marcos Felix) #1

I have been following the tutorial to get Winlogbeat installed, and I got stuck on this bit:
https://www.elastic.co/guide/en/beats/winlogbeat/master/winlogbeat-template.html#load-template-manually.
Whenever I run this command:

PS > .\winlogbeat.exe setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

I get this error:

Get-Process : Parameter cannot be processed because the parameter name 'E' is ambiguous. Possible matches include:
-ErrorAction -ErrorVariable.
At line:1 char:40
+ PS > .\winlogbeat.exe setup --template -E output.logstash.enabled=fal ...
+                                        ~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.GetProcessCommand

Any clues?


(Marcos Felix) #2

When I try a different method: https://www.elastic.co/guide/en/beats/winlogbeat/1.3/winlogbeat-template.html

I ran this code:

PS C:\Program Files\Winlogbeat> Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri http://localhost:9200/_template/winlogbeat?pretty

and get this error:

Get-Process : A positional parameter cannot be found that accepts argument 'Invoke-WebRequest'.
At line:1 char:1
+ PS .\winlogbeat.exe> Invoke-WebRequest -Method Put -InFile winlogbeat ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

if I remove the path to winlogbeat I get this:

Get-Process : A parameter cannot be found that matches parameter name 'Method'.
At line:1 char:22
+ PS Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Ur ...
+                      ~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

(Marcos Felix) #3

I am also getting this error when doing .\winlogbeat.exe

Program 'winlogbeat.exe' failed to run: The specified executable is not a valid application for this OS platform.At
line:1 char:2
+  C:\Software\winlogbeat\winlogbeat.exe test config -c .\winlogbeat.ym ...
+  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At line:1 char:2
+  C:\Software\winlogbeat\winlogbeat.exe test config -c .\winlogbeat.ym ...
+  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed

(Marcos Felix) #4

Even when setting up Kibana dashboard I get an error:

PS > .\winlogbeat.exe setup --dashboards

Get-Process : A positional parameter cannot be found that accepts argument '--dashboards'.
At line:1 char:1
+ PS > .\winlogbeat.exe setup --dashboards
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

(Marcos Felix) #5

I had to download the JSON file from the web (github) then pasted into the file. I then ran the command:

Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri http://localhost:9200/_template/winlogbeat?pretty

and worked fine.
Now I am trying to set the Kibana dashboard, but it either says:

Get-Process : A parameter cannot be found that matches parameter name 'dashboards'.
At line:1 char:25
+ PS > .\winlogbeat setup -dashboards
+                         ~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

or it says:

Program 'winlogbeat.exe' failed to run: The specified executable is not a valid application for this OS platform.At line:1 char:1
+ .\winlogbeat setup --dashboards
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At line:1 char:1
+ .\winlogbeat setup --dashboards
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed

When installed the dashboard for Logstash output using this command:

PS > .\winlogbeat.exe setup -e `
  -E output.logstash.enabled=false `
  -E output.elasticsearch.hosts=['localhost:9200'] `
  -E output.elasticsearch.username=winlogbeat_internal `
  -E output.elasticsearch.password=YOUR_PASSWORD `
  -E setup.kibana.host=localhost:5601

I get this error:

Get-Process : Parameter cannot be processed because the parameter name 'e' is ambiguous. Possible matches include:
-ErrorAction -ErrorVariable.
At line:1 char:29
+ PS > .\winlogbeat.exe setup -e `
+                             ~~
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.GetProcessCommand

Also for some reason my winlogbeat did not come with a scripts folder for me to install dashboards as stated here:
https://www.elastic.co/guide/en/beats/winlogbeat/5.0/winlogbeat-sample-dashboards.html

Also when trying to run winlogbeat I get this error:

Start-Service : Service 'winlogbeat (winlogbeat)' cannot be started due to the following error: Cannot start service winlogbeat on computer '.'.
At line:1 char:1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

This is my config file:

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: Security
  - name: System

setup.template.settings:
  index.number_of_shards: 3

setup.dashboards.enabled: true

setup.kibana:
  host: "IP:5601"

output.logstash:
  hosts: ["IP:5000"]

(Marcos Felix) #6

Hello ?


(Adrian Serrano) #7

I see that you are mixing resources and documentation from wildly different versions of Beats. From master (beta), 1.3, 5.0. There is a lot of evolution and breaking changes so its normal that some procedures can't be followed.

First things first, which version of Winlogbeat are you using?

I suggest you use the latest relase, 6.3.1 at this moment, and stick to the latest documentation.

Latest Winlogbeat : https://www.elastic.co/downloads/beats/winlogbeat

Latest Documentation: https://www.elastic.co/guide/en/beats/winlogbeat/current/index.html


(Marcos Felix) #8

Thank you!
I reinstalled and followed the doc and it worked, I was able to start winlogbeat.
However, now on Kibana dashboard when I select winlogbeat-* I get a red bar at the top saying:

No matching indices found: No indices match pattern "winlogbeat-*"

any clue of what is going on? could it be configuration ?

I get the same for Filebeat


(Adrian Serrano) #9

You need to run winlogbeat first, so it sends events to Elasticsearch. Until then, the index will not exist.


(Marcos Felix) #10

I have ran winlogbeat with Start-Service winlogbeat
I am using Logstash so would it be sent to Logstash and the indices will be named logstash?

For that red bar, the Error is:

Error: No matching indices found: No indices match pattern "winlogbeat-*"
    at http://10.130.233.242:5601/bundles/commons.bundle.js:3:918983
    at processQueue (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:134252)
    at http://10.130.233.242:5601/bundles/vendors.bundle.js:133:135201
    at Scope.$digest (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:146077)
    at Scope.$apply (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:148856)
    at done (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:101124)
    at completeRequest (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:106024)
    at XMLHttpRequest.xhr.onload (http://10.130.233.242:5601/bundles/vendors.bundle.js:133:106783)

(Adrian Serrano) #11

Did you set up the Beats Input module in logstash?

https://www.elastic.co/guide/en/logstash/6.3/plugins-inputs-beats.html

If so, it will be using the winlogbeat indices.


(Marcos Felix) #12

Yes I had a file within conf.d named logstash-beats.conf and it has this config:

input {
  beats {
    port => 5000
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

(Adrian Serrano) #13

Then, if beats is configured to forward events to LS, you should be seeing events in Kibana. Can you check that winlogbeat is forwarding events and logstash is receiving them?


(Marcos Felix) #14

https://pastebin.com/raw/J3iDDv9v

here is the logs from winlogbeat

It seems like the client is having issues with Logstash, but I have no idea why because all the configs seems to be fine

seems like filebeat and metricbeat log file has a similar output.


(Adrian Serrano) #15

Are you sure there is no firewall blocking the 5000 port in the logstash server?


(Marcos Felix) #16

Not sure if blocking, but it wasn't in the list. So, I just added it to the list, do I have to restart everything or?

This is what I get from the logs now:

2018-07-10T11:11:37.737+0100 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":2156,"time":{"ms":16}},"total":{"ticks":21624,"time":{"ms":109},"value":21624},"user":{"ticks":19468,"time":{"ms":93}}},"info":{"ephemeral_id":"e5347566-6fef-4735-add4-f90bd7973a0e","uptime":{"ms":3000172}},"memstats":{"gc_next":37620576,"memory_alloc":31959552,"memory_total":824947000,"rss":389120}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"batches":4,"failed":4156,"total":4156},"read":{"errors":1},"write":{"bytes":738146}},"pipeline":{"clients":3,"events":{"active":4119,"retry":6204}}}}}}

This is the same on filebeat and metricbeat


(Adrian Serrano) #17

It will retry sending events after a while, but if you're testing you can stop winlogbeat, remove the file in C:\ProgramData\winlogbeat\.winlogbeat.yml and launch it again, so it starts sending events from the beginning of the event log.


(Marcos Felix) #18

you mean delete the log file right not the actual config file


(Adrian Serrano) #19

In C:\ProgramData\winlogbeat\ you should have a file starting with a dot, .winlogbeat.yml which is the registry file where winlogbeat stores the bookmarks to remember the already-read position into an event log between executions. It you remove it, it will start reading events from the beginning of the event log.


(Marcos Felix) #20

So I just made a new directory to send logs to get fresh ones and it still gives No matching indices found for filebeat, metric and winlogbeat.
Logstash outputs this:

`

version:1 host:10.130.233.242 tags:_grokparsefailure @timestamp:July 10th 2018, 11:39:20.326 message:\xD5 (\u0006x}\xD2*^\xEA$\xD6\e@\x9C&\u0011\xADi6\xA66\xC7k\xE7N\xF8)\xABMa\u0014^\xD2\xD2W\x90\x92\xA4\xEE5\xED>\x83\x80\xA8δ\xD8/g\xBA}\xDA*bJO\xE7:H4s\xEA\xB9\u001A;\xA2\xBB\x85\xC6v\x98v\xC1\u0013?C\x88\xF3\x84\xB5¾\u0006\x8Af\vI\xAD\xCE?\,\xBD\x89?\xAEZ\xE2\eW\xFA[\xC0\xB3C\vT\xD2\e\xB0\xFB\xACՎ\u0015\xC7.\xD5,\u0014\xE0\x82Z\xAF\xDEQP\x9EO\xBF\u0004\xC8>G\x88\xF3\u0000Jn\x81\x81\xE7\xE9Uq?T\u001A\xD7I\x95E\xC2gY\x8A#!si,\x9B66\u0000\xC0\xEA\xF3\xE8df\u0002+\2\x8B\xDC@x\xAD\xAC\bS\xA1=\xFE\u001A\xA7\xB7A\xC3c\xC7\xEF<\u0001\xB1/XE\x8C\u0019b\xE4y\xC3\u0003e\xE8\xDA\aĞC_D\x9B\xD0\u0018m\xA3b\x88\x8AԢ\x97i\xF7\xABc\u001EƲ\xA4\a\x87\x8D\xAD\xABW\xE0\u0001\xB3\xE7\xADb6\b\u001E\xABΆۃ\x8C\xA4\x9B\x9F\xB4]`\xEA[U_D\xBFd4\xA13\xE4\u000F port:32,800 type:random_logs _id:BrHGg2QBHw24IkY_MwGV _type:doc _index:logstash-2018.07.10 _score: -

and

@version:1 host:10.130.233.242 tags:_grokparsefailure @timestamp:July 10th 2018, 11:39:51.303 message:\xDD\u007F\u000F$\xB3R\x8BP\xD7#\xAFL\x8B\xF3\x92T\xBA\xCD-\u0012\u000Fu\x82\f\xBC\xFE;\xC2s\x98$L5o\u0000\xCBE\x91\x84 port:32,810 type:random_logs _id:0bHGg2QBHw24IkY_sA4S _type:doc _index:logstash-2018.07.10 _score:

The errors on the winlogbeat log is:

2018-07-10T11:22:57.345+0100	ERROR	logstash/async.go:235	Failed to publish events caused by: client is not connected
|2018-07-10T11:23:59.535+0100|ERROR|logstash/async.go:235|Failed to publish events caused by: read tcp IP:55144->IP:5000: i/o timeout|
|---|---|---|---|
|2018-07-10T11:23:59.539+0100|ERROR|logstash/async.go:235|Failed to publish events caused by: read tcp IP:55144->IP:5000: i/o timeout|
|2018-07-10T11:23:59.540+0100|ERROR|logstash/async.go:235|Failed to publish events caused by: read tcp IP:55144->IP:5000: i/o timeout|
2018-07-10T11:24:00.602+0100	ERROR	pipeline/output.go:92	Failed to publish events: client is not connected

I will try

I dont have any file starting with a dot (.winlogbeat.yml) I do have just winlogbeat.yml. The only file I have that starts with a dot is .build_hash