Error in Winlogbeat logs

Elastic Stack Beats
1

Winlogbeat has been working fine for a week now, today I turned my computer on and it just wouldnt load todays logs.
I checked the logs and this is what it says:

2018-07-18T09:32:27.192+0100	ERROR	instance/beat.go:691	Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory C:\software\winlogbeat\kibana: Failed to import index-pattern: Failed to load directory C:\software\winlogbeat\kibana/6/index-pattern:
  error loading C:\software\winlogbeat\kibana\6\index-pattern\winlogbeat.json: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];. Response: {"objects":[{"id":"winlogbeat-*","type":"index-pattern","error":{"message":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}}]}

any idea?

2

Your Elasticsearch index has turned read-only, that's why Winlogbeat can't ship the logs.

It might be that Elasticsearch is running out of disk, or other reasons.

Have a look at FORBIDDEN/12/index read-only / allow delete (api)]

3

Yes, i just ran across that thread, I have cleared my logs and will test then. Will update the threat if anything.

Thanks

4

It is working now, I had to delete old logs

5

Hey, I came back after an hour and this came up on monitoring:

We couldn't activate monitoring
Here might be some things to check

No Monitoring data could be found for the selected time period, but we could not find the cluster setting that makes the data unavailable.

There may be data available for a different time period than we have selected. Try adjusting the time filter controls to a time range where the Monitoring data is expected.

We are refreshing the search for data in the background. If cluster data is found, we will redirect to the cluster overview page.

and this:

Monitoring Request Failed

Unable to find the cluster in the selected time range. UUID: BZrLFZApRyWIsTznQpzRfg

HTTP 404

I can see data if the time period is longer than 4 hours, however there are no recent logs being shipped. When I restarted winlogbeat I got this error:

Start-Service : Failed to start service 'winlogbeat (winlogbeat)'.
At line:1 char:1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController)
   ServiceCommandException
    + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand

After running this command on PS:
.\winlogbeat.exe -c winlogbeat.yml -e -v -d "*"

2018-07-18T14:09:04.894+0100	ERROR	instance/beat.go:691	Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory C:\software\winlogbeat\kibana: Failed to import index-pattern: Failed to load directory C:\software\winlogbeat\kibana/6/index-pattern:
  error loading C:\software\winlogbeat\kibana\6\index-pattern\winlogbeat.json: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];. Response: {"objects":[{"id":"winlogbeat-*","type":"index-pattern","error":{"message":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}}]}

to solve this issue ^: I ran this

 PUT _settings
    {
    "index": {
    "blocks": {
    "read_only_allow_delete": "false"
    }
    }
    }

Any clues as to why I am getting this?

6

edit1: I restart winlogbeat and now it cannot start on windows. This seems to be a routine. Everytime after I run something if I restart it automatically gives this error:

2018-07-18T15:39:38.075+0100 ERROR instance/beat.go:691 Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory C:\software\winlogbeat\kibana: Failed to import index-pattern: Failed to load directory C:\software\winlogbeat\kibana/6/index-pattern:
error loading C:\software\winlogbeat\kibana\6\index-pattern\winlogbeat.json: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];. Response: {"objects":[{"id":"winlogbeat-*","type":"index-pattern","error":{"message":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}}]}

I don't know how to fix this error without having to delete the logs. It's not running out of memory:

image
image3324×846 230 KB

and I already ran this on dev tools:

PUT winlogbeat-*/_settings
{
"index": {
"blocks": {
"read_only_allow_delete": "false"
}
}
}

7

filebeat setup wants to setup different things, some of them being the index templates and pipelines, which need to be done in Elasticsearch. If you want to setup only the dashboards, try adding --dashboards. Then elasticsearch output won't be required.

10

Im sorry, seems like I got the posts mixed up. I am having this issue with winlogbeat not filebeat. Well, I was having with both. So, I just deleted filebeat for now and trying to work with winlogbeat.

If I do as you said, I get this:

image
image.png1275×153 11 KB

or if I run a different command:
image
image.png1290×209 15.8 KB

11

Marcos, it looks like the root cause here is that your ES instance has been put back into a read-only state. Have you checked the ES logs?

Are you running low on disk space? That can cause this situation. You may have to re-enable writes to that index if it has been closed due to low disk space.

12

I have checked the elastic logs and it is indeed full, but I dont understand those logs. Are they the winlogbeats logs ive been submitting? Also, how can I delete these logs? would it be safe to ?
You mentioned re-enabling writes to that index if its has been closed cause of low disk space, how do I go on about doing this?

Thanks

14

I got it to work now, increase disk space. Winlogbeat is running but for some reason it isn't forwarding any logs to Kibana, there is nothing on the winlogbeat logs and it starts and stops just fine. Any clues?

edit1: fixed, I had to delete the logs so it would start logging again.

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.