Checking if the output.logstash is going to valid host

Elastic Stack Beats
1

My goal is to verify that the output.logstash is actually being sent to the proper host. My filebeat configuration is set up like so

filebeat.inputs:

- input_type: log
  paths:
    - /var/log/*.log
    - /var/log/*/*.log

tags: ["security", "bastion"]

processors:
 - add_cloud_metadata:

output.logstash:
  hosts: ["myhost:5000"]
  ssl.certificate_authorities: ["/etc/pki/beats/logstashCA.crt"]
  ssl.certificate: "/etc/pki/beats/beats.crt"
  ssl.key: "/etc/pki/beats/beats.key"

The logstash instance is up and running, but I'd like to know how to test whether or not the data is actually being sent from the machine running filebeat to the machine running logstash. Does anyone have a suggestion for how to test this?

2

See here

filebeat test output

3

Error initializing beat: could not initialize the keystore: open /var/lib/filebeat/filebeat.keystore: permission denied

4

Thats is a different issue all together.

That is usually a directory / file permission issue...

Does the user you are running the test command have permission to access those files / directory?

Did you install with a package if so those files are probably owned by the filebeat

you may need to run the above command with sudo

sudo filebeat test output

5

Thank you for the suggestion. Looks like things are up and running, but I'm getting security... WARN server's certificate chain verification is disabled

logstash: myinstance:5000...
connection...
parse host... OK
dns lookup... OK
addresses: x.xxx.xxx.xxx, xx.xx.xxx.xx, xx.xx.xx.xxx
dial up... OK
TLS...
security... WARN server's certificate chain verification is disabled
handshake... OK
TLS version: TLSv1.2
dial up... OK
talk to server... OK

6

Certs... not my area of expertise.

Filebeat logstash outpput ssl

Configuration options for SSL parameters like the root CA for Logstash connections. See SSL for more information. To use SSL, you must also configure the Beats input plugin for Logstash to use SSL/TLS.

Do you have the beats input plugin on logstash using SSL?

7

ls /var/lib/logstash/plugins/inputs/

s3

8

No it is a directory on your unix filesystem

9

What exactly do you mean by that?

10

You asked....

/var/lib/logstash/plugins/inputs/ is a filesystem directory on a Unix OS.

Perhaps if you are not familiar with Unix Operating Systems and Filesystems you could reach out to a peer that is.

This is fundamental Unix OS Admin stuff: Filesystems, Directories, Permissions etc.. etc..

and I am not clear why you are asking about it that is directory in the first place?

11

Thank you, I'm trying to understand how to validate whether or not this is an ssl issue. I was listing out the inputs. The filebeat test output shows that this instance is connected. How would you test if this is being processed on the logstash side of things?

12

The ssl certificate for logstash is valid. I just checked.

13

The filebeat output is common for the 2 different filebeat inputs.

You would test logstash by running beats -> logstash -> output (Elasticsearch or Console etc)

In your logstash pipeline config did you enable ssl in the beats input? Try that

input {
  beats {
    port => 5044
    ssl => true
  }
}
15

That shows that it's running.

16

Apologies @EvanGertis

I am not sure how to help, I am not sure how to respond to 1 line statements perhaps someone else will be able to help.

17

[INFO ] 2021-09-14 21:27:06.698 [[main]-pipeline-manager] beats - Beats inputs: Starting input listener {:address=>"0.0.0.0:5000"} the pipeline started and it's running.

18

solved: I was specifying an IP address for logstash instead of the DNS name

1 Like
19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.