No indices match pattern "winlogbeat-*"

Elastic Stack Beats
1

Hi,

I have configured elasticsearch, logstash and kibana in Server Cent OS which IP is (10.200.14.36) and both working perfectly.

Also, with filebeat in another Client Cent OS machine, I can able to get index and logs into Server Cent OS machine.

I have setup winlogbeat 7.4 in windows 10 and want to ship logs to elasticsearch in Server Cent OS. However, I am receiving "No indices match pattern " error in kibana as well in kibana it enlisted winlogbeat in index pattern.

Please find attached screenshots for more clarification....

Screenshot_3
Screenshot_3.png724×377 27.4 KB
Screenshot_4

Following is configuration of my winlogbeat:

#======================= Winlogbeat specific options ===========================

https://go.es.io/WinlogbeatConfig

winlogbeat.event_logs:

  • name: Application
    ignore_older: 72h

  • name: System

  • name: Security
    processors:

    • script:
      lang: javascript
      id: security
      file: ${path.home}/module/security/config/winlogbeat-security.js

  • name: Microsoft-Windows-Sysmon/Operational
    processors:

    • script:
      lang: javascript
      id: sysmon
      file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js

#==================== Elasticsearch template settings ==========================

setup.template.settings:
index.number_of_shards: 1
#index.codec: best_compression
#_source.enabled: false

#================================ General =====================================

The name of the shipper that publishes the network data. It can be used to group

all the transactions sent by a single shipper in the web interface.

#name:

The tags of the shipper are included in their own field with each

transaction published.

#tags: ["service-X", "web-tier"]

Optional fields that you can specify to add additional information to the

output.

#fields:

env: staging

#============================== Dashboards =====================================

These settings control loading the sample dashboards to the Kibana index. Loading

the dashboards is disabled by default and can be enabled either by setting the

options here or by using the setup command.

setup.dashboards.enabled: true

The URL from where to download the dashboards archive. By default this URL

has a value which is computed based on the Beat name and version. For released

versions, this URL points to the dashboard archive on the artifacts.elastic.co

website.

#setup.dashboards.url:

#============================== Kibana =====================================

Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.

This requires a Kibana endpoint configuration.

setup.kibana:

Kibana Host

Scheme and port can be left out and will be set to the default (http and 5601)

In case you specify and additional path, the scheme is required: http://localhost:5601/path

IPv6 addresses should always be defined as: https://[2001:db8::1]:5601

#host: "localhost:5601"
host: "10.200.14.36:5601"

Kibana Space ID

ID of the Kibana Space into which the dashboards should be loaded. By default,

the Default Space will be used.

#space.id:

#================================ Outputs =====================================

Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:

Array of hosts to connect to.

#hosts: ["localhost:9200"]
hosts: ["10.200.14.36:9200"]

Optional protocol and basic auth credentials.

#protocol: "https"
#username: "elastic"
#password: "changeme"

#----------------------------- Logstash output --------------------------------
#output.logstash:

The Logstash hosts

#hosts: ["localhost:5044"]

Optional SSL. By default is off.

List of root certificates for HTTPS server verifications

#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

Certificate for SSL client authentication

#ssl.certificate: "/etc/pki/client/cert.pem"

Client Certificate Key

#ssl.key: "/etc/pki/client/cert.key"

#================================ Processors =========================

Configure processors to enhance or manipulate events generated by the beat.

processors:

  • add_host_metadata: ~
  • add_cloud_metadata: ~

#================================ Logging =================================

Sets log level. The default log level is info.

Available log levels are: error, warning, info, debug

#logging.level: debug

At debug level, you can selectively enable logging only for some components.

To enable all selectors use ["*"]. Examples of other selectors are "beat",

"publish", "service".

#logging.selectors: ["*"]

Please help me..try to resolve it but don't understand the actual problem.....

2

As well, following is my log file for winlogbeat after starting the service:

2019-10-24T01:37:39.232-0400 INFO instance/beat.go:607 Home path: [C:\Program Files\Winlogbeat] Config path: [C:\Program Files\Winlogbeat] Data path: [C:\ProgramData\winlogbeat] Logs path: [C:\ProgramData\winlogbeat\logs]
2019-10-24T01:37:39.239-0400 INFO instance/beat.go:615 Beat ID: 5d3af6dd-7cb1-47a2-8c90-344960754f9e
2019-10-24T01:37:39.249-0400 INFO [beat] instance/beat.go:903 Beat info {"system_info": {"beat": {"path": {"config": "C:\Program Files\Winlogbeat", "data": "C:\ProgramData\winlogbeat", "home": "C:\Program Files\Winlogbeat", "logs": "C:\ProgramData\winlogbeat\logs"}, "type": "winlogbeat", "uuid": "5d3af6dd-7cb1-47a2-8c90-344960754f9e"}}}
2019-10-24T01:37:39.875-0400 INFO [beat] instance/beat.go:912 Build info {"system_info": {"build": {"commit": "f940c36884d3749901a9c99bea5463a6030cdd9c", "libbeat": "7.4.0", "time": "2019-09-27T07:53:03.000Z", "version": "7.4.0"}}}
2019-10-24T01:37:39.875-0400 INFO [beat] instance/beat.go:915 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":16,"version":"go1.12.9"}}}
2019-10-24T01:37:39.880-0400 INFO [beat] instance/beat.go:919 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-10-23T19:59:26.23-04:00","name":"DESKTOP-KP8OOMO","ip":["fe80::f05b:7e64:3d34:5cd8/64","10.200.14.39/24","::1/128","127.0.0.1/8"],"kernel_version":"10.0.18362.418 (WinBuild.160101.0800)","mac":["00:0c:29:5f:b1:55"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"18362.418"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"27c5681f-aedc-4d6c-896f-6ec875861761"}}}
2019-10-24T01:37:39.884-0400 INFO [beat] instance/beat.go:948 Process info {"system_info": {"process": {"cwd": "C:\WINDOWS\system32", "exe": "C:\Program Files\Winlogbeat\winlogbeat.exe", "name": "winlogbeat.exe", "pid": 3968, "ppid": 856, "start_time": "2019-10-24T01:37:38.833-0400"}}}
2019-10-24T01:37:39.884-0400 INFO instance/beat.go:292 Setup Beat: winlogbeat; Version: 7.4.0
2019-10-24T01:37:39.884-0400 INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'winlogbeat-7.4.0' as ILM is enabled.
2019-10-24T01:37:39.885-0400 INFO elasticsearch/client.go:170 Elasticsearch url: http://10.200.14.36:9200
2019-10-24T01:37:39.885-0400 INFO [publisher] pipeline/module.go:97 Beat name: DESKTOP-KP8OOMO
2019-10-24T01:37:39.885-0400 INFO beater/winlogbeat.go:69 State will be read from and persisted to C:\ProgramData\winlogbeat.winlogbeat.yml
2019-10-24T01:37:39.904-0400 WARN [cfgwarn] registered_domain/registered_domain.go:58 BETA: The registered_domain processor is beta.
2019-10-24T01:37:39.904-0400 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2019-10-24T01:37:39.904-0400 INFO kibana/client.go:117 Kibana url: http://10.200.14.36:5601
2019-10-24T01:37:40.059-0400 INFO kibana/client.go:117 Kibana url: http://10.200.14.36:5601
2019-10-24T01:37:41.575-0400 INFO instance/beat.go:777 Kibana dashboards successfully loaded.
2019-10-24T01:37:41.575-0400 INFO instance/beat.go:422 winlogbeat start running.
2019-10-24T01:37:42.250-0400 INFO add_cloud_metadata/add_cloud_metadata.go:87 add_cloud_metadata: hosting provider type not detected.
2019-10-24T01:37:42.324-0400 WARN beater/eventlogger.go:108 EventLog[Microsoft-Windows-Sysmon/Operational] Open() error. No events will be read from this source. The specified channel could not be found.
2019-10-24T01:37:43.211-0400 INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(http://10.200.14.36:9200))
2019-10-24T01:37:47.218-0400 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://10.200.14.36:9200)): Get http://10.200.14.36:9200: dial tcp 10.200.14.36:9200: connectex: No connection could be made because the target machine actively refused it.
2019-10-24T01:37:47.219-0400 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://10.200.14.36:9200)) with 1 reconnect attempt(s)
2019-10-24T01:37:52.298-0400 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://10.200.14.36:9200)): Get http://10.200.14.36:9200: dial tcp 10.200.14.36:9200: connectex: No connection could be made because the target machine actively refused it.
2019-10-24T01:37:52.298-0400 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://10.200.14.36:9200)) with 2 reconnect attempt(s)
2019-10-24T01:38:00.500-0400 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://10.200.14.36:9200)): Get http://10.200.14.36:9200: dial tcp 10.200.14.36:9200: connectex: No connection could be made because the target machine actively refused it.
2019-10-24T01:38:00.500-0400 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://10.200.14.36:9200)) with 3 reconnect attempt(s)
2019-10-24T01:38:00.500-0400 INFO [publisher] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-10-24T01:38:00.500-0400 INFO [publisher] pipeline/retry.go:168 done
2019-10-24T01:38:09.907-0400 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":359,"time":{"ms":359}},"total":{"ticks":2296,"time":{"ms":2296},"value":2296},"user":{"ticks":1937,"time":{"ms":1937}}},"handles":{"open":331},"info":{"ephemeral_id":"d62ce8bd-ff4c-4687-8fa9-a231cdc8f274","uptime":{"ms":30707}},"memstats":{"gc_next":33469232,"memory_alloc":28566832,"memory_total":105728144,"rss":60780544},"runtime":{"goroutines":33}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":3,"events":{"active":2176,"published":2176,"retry":50,"total":2176}}},"msg_file_cache":{"ApplicationHits":903,"ApplicationMisses":20,"ApplicationSize":20,"SecurityHits":756,"SecurityMisses":2,"SecuritySize":2,"SystemHits":479,"SystemMisses":24,"SystemSize":24},"system":{"cpu":{"cores":16}}}}}
2019-10-24T01:38:16.500-0400 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://10.200.14.36:9200)): Get http://10.200.14.36:9200: dial tcp 10.200.14.36:9200: connectex: No connection could be made because the target machine actively refused it.
2019-10-24T01:38:16.500-0400 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://10.200.14.36:9200)) with 4 reconnect attempt(s)
2019-10-24T01:38:16.500-0400 INFO [publisher] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-10-24T01:38:16.500-0400 INFO [publisher] pipeline/retry.go:191 done
2019-10-24T01:38:16.500-0400 INFO [publisher] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-10-24T01:38:16.500-0400 INFO [publisher] pipeline/retry.go:168 done

3

Seems like Winlog beat can not connect to Elasticsearch. Check networking and firewall rules and that you have setup Elasticsearch to allow external connections (not bound to localhost).

4

Hi Christian,

In firewall port 9200 is already open, same for CentOS and Windows firewall...

However, Problem resolved, I need to add "http.host: 10.200.14.36 & http.port: 9200" in my elasticsearch YAML file which can receive winloagbeat logs input (due to it connect to elasticsearch with http://10.200.14.36:9200 as per the logs...)

Thank you for your help

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.