ELK on Windows Server 2016 with Filebeat on Windows 10 client - not working

[JY_DT]

Hello,

I've setup ELK 7.4.2 on Windows Server 2016, with some of these references:

Demystifying ELK stack

http://robwillis.info/2019/05/installing-elk-7-elasticsearch-logstash-and-kibana-windows-server-2016/

Installing the ELK Stack on Windows | Logz.io

and I've setup Filebeat on my Windows 10 client machine.
It looks like all the installations are fine, and I'm able to start all services.
But no data seems to be inserted into Elasticsearch. I'm not sure what's going wrong.

On the server machine:

1. When I enter this in a browser:
   http://localhost:9200/

I see the details correctly.

2. And on this:
   http://localhost:9200/_cat/indices

I see these indexes:
green open .kibana_task_manager_1 MNLTzyjNQk2_DBwLbLAUjQ 1 0 2 0 13.3kb 13.3kb
green open kibana_sample_data_ecommerce MD4yCUShR4KJi0qzrWo1lQ 1 0 4675 0 4.4mb 4.4mb
green open .apm-agent-configuration bqpM6GzVS7mJJNtxbWrVvg 1 0 0 0 283b 283b
green open .kibana_1 9S6hqlJnS2KxsKPYh1viyw 1 0 74 7 954.3kb 954.3kb

3. On viewing Kibana, in the browser:
   http://127.0.0.1:5601/ and then selecting "Index Patterns", I see only

kibana_sample_data_ecommerce

and I don't see anything related to filebeat*

On the Client machine:

1. Filebeat is installed and configured correctly.
   I've enabled logstash output with the correct IP and port address.

What could be wrong? I should be able to see the indices getting created and then be able to query for data.

Thanks,
Jy

[MiTschMR]

Hi @JY_DT

Please share your configuration for Filebeat and Logstash with us. Without them it is more of a quiz.

Cheers
MiTschMR

[JY_DT]

Sure, is there a way to upload a file. But anyway, here are the configs:

1. Filebeat:

'''
filebeat.inputs:

• type: log
  enabled: true
  paths:
  - C:\ProgramData\DiagramOne\STool\Trace*

output.logstash:
hosts: ["MyserverIP:5044"]
bulk_max_size: 1024

'''

These are the only things I've changed.

[JY_DT]

And for Logstash on the Server:
The logstash.conf file contains:
'''
input {
beats {
port => 5044
type => "log"
}
}

output {
elasticsearch {
hosts => "http://localhost:9200"
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
'''

[MiTschMR]

Hi @JY_DT

Please format your code and check it in the preview window. It can be hard to read and people might skip your post. Use triple-backsticks ``` above and below the text. It should then look like this:

Formatted text

Thank you very much.

[MiTschMR]

Please post the content of your filebeat log file. You can find the file under C:\ProgramData\filebeat\logs.

[JY_DT]

'''

2019-11-26T18:40:43.330+0530	INFO	instance/beat.go:607	Home path: [C:\Program Files\Filebeat] Config path: [C:\Program Files\Filebeat] Data path: [C:\Program Files\Filebeat\data] Logs path: [C:\Program Files\Filebeat\logs]
2019-11-26T18:40:43.335+0530	INFO	instance/beat.go:615	Beat ID: 6487577c-a9c3-4918-8bdb-520ff8aa9a53

'''

[MiTschMR]

Hi @JY_DT

As you can see in the filebeat log file, the path for the logs is not C:\ProgramData\filebeat, it is C:\Program Files\Filebeat\logs. Please share the content of the log file from there.

[JY_DT]

Hi,

I'm actually trying this now:
Load the index template in Elasticsearch:
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html
and I get these errors in the log file.

'''
2019-11-28T15:44:23.248+0530 INFO [publisher] pipeline/module.go:97 Beat name: Something
2019-11-28T15:44:23.250+0530 INFO elasticsearch/client.go:170 Elasticsearch url: http://192.168.103.84:9200
2019-11-28T15:44:25.269+0530 ERROR elasticsearch/elasticsearch.go:260 Error connecting to Elasticsearch at http://192.168.103.84:9200: Get http://192.168.103.84:9200: dial tcp 192.168.103.84:9200: connectex: No connection could be made because the target machine actively refused it.
2019-11-28T15:44:25.269+0530 ERROR instance/beat.go:878 Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://192.168.103.84:9200: Get http://192.168.103.84:9200: dial tcp 192.168.103.84:9200: connectex: No connection could be made because the target machine actively refused it.]
'''

[JY_DT]

Please note that on the m/c where ELK is installed has the firewall disabled, and I've also added firewall rules to allow incoming to ELK. So, I'm not sure why this error occurs.

Also, I'm able to ping the ELK m/c from my client m/c.
Thanks.