ERR Failed to publish events caused by: read tcp 127.0.0.1:50248->127.0.0.1:5601: i/o timeout

Elastic Stack Beats
1

To visualize Windows Event Logs I've setup an ELK server running all 3 components on the same box. Think for testing and learning ELK should be doable.
Logs forwarded are account lockouts (Event ID 4740) from Domain Controllers.

I was able to pull in and visualize some of these forwarded logs but all of a sudden it stoped. I also noticed the Logstash service restarted automatically on a short but regular interval.

Re-installing the Logstash service fixed this and it is now in a permanent running state. But not log information appears to be shipped.

Looking at the winlogbeat log files the error from the subject of this request for help is shown:
ERR Failed to publish events caused by: read tcp 127.0.0.1:50248->127.0.0.1:5601: i/o timeout

When I enable the debug option for winlogbeat is see the forwarded events it is picking up from the Forwarded Events log on the same server.

Any help or ideas are welcome to solve the error shown

I already read many posts here but none appear to fix my issue.

Platform is Windows Server 2016, 8GB RAM, 2 CPU's
Latest ELK downloads.
Telnet to port 5601 is fine, although slow in response.
Java Platform SE binary takes all the CPU.

winlogbeat.yml
winlogbeat

logstash config.json
json

2

Port 5601 is usually used by Kibana, but you have configured the beats inout in Logstash to use it, which may be causing the problem. I would recommend using a different port for the Logstas beats input and update Winlogbeat to use tis.

3

Hello Christian
Thanks for your reply, how to change the port for logstash?

4

Change in the Logstash config.

5

Changed the port:
ERR Connecting error publishing events (retrying): dial tcp 127.0.0.1:5000: connectex: No connection could be made because the target machine actively refused it.

6

Ok I changed the port to 9300 assuming that is the correct one to use.

Yet another error showed.
ERR Failed to publish events caused by: EOF

7

You should not use the 9300 port. Winlogbeat should connect to Logstash on port 5000 (update winlogbeat.yml). The beats input should listen to port 5000 and the Elasticsearch output should connect to Elasticsearch on port 9200 (both updated in config.json). Please share your configuration files again once you have updated them.

8

Logstash config file
json

Winlogbeat
winlogbeat

9

Yes, that looks fine as far as I can see.

10

2017-09-26T05:45:03-07:00 INFO Home path: [D:\Program Files\winlogbeat-5.6.0-windows-x86_64] Config path: [D:\Program Files\winlogbeat-5.6.0-windows-x86_64] Data path: [C:\ProgramData\winlogbeat] Logs path: [D:\Program Files\winlogbeat-5.6.0-windows-x86_64\logs]
2017-09-26T05:45:03-07:00 INFO Setup Beat: winlogbeat; Version: 5.6.0
2017-09-26T05:45:03-07:00 INFO Max Retries set to: 3
2017-09-26T05:45:03-07:00 INFO Activated logstash as output plugin.
2017-09-26T05:45:03-07:00 INFO Publisher name: gdcpwsec01
2017-09-26T05:45:04-07:00 INFO Metrics logging every 30s
2017-09-26T05:45:04-07:00 INFO Flush Interval set to: 1s
2017-09-26T05:45:04-07:00 INFO Max Bulk Size set to: 2048
2017-09-26T05:45:04-07:00 INFO State will be read from and persisted to C:\ProgramData\winlogbeat.winlogbeat.yml
2017-09-26T05:45:04-07:00 INFO winlogbeat start running.
2017-09-26T05:45:06-07:00 ERR Connecting error publishing events (retrying): dial tcp 127.0.0.1:5000: connectex: No connection could be made because the target machine actively refused it.
2017-09-26T05:45:10-07:00 ERR Connecting error publishing events (retrying): dial tcp 127.0.0.1:5000: connectex: No connection could be made because the target machine actively refused it.

11

Have all components started up OK? Can you connect to the ports using telnet?

12

Yes services started properly.

But can't connect on port 5000 using telnet.
Microsoft Telnet> o
( to ) 127.0.0.1 5000
Connecting To 127.0.0.1...Could not open connection to the host, on port 5000: Connect failed
Microsoft Telnet>

13

That probably means that Logstash have not started ups correctly. Can you see anything in the Logstash logs?

14

Found it.

I had Logstash installed in "Program Files\Logstash", changed to a folder with no " " and now it runs.

I wonder why it had worked before. Mayb during my first try is was ingesting directly in ES.

Anyway thanks for your help Christion. Really appreciate it.

Regards.

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.