Logstash data ouput to elasticsearch calculation

Elastic Stack Logstash
1

Hello,

I am trying to find out how to see how much data is being outputted to elasticsearch from logstash servers. The logstash servers in question are hosted on VMWare at the moment.

The logstash server hosted on AWS, I can see how much data is coming out on the networkout within cloudwatch metrics.

How can I get a similar result on the logstash boxes themselves?
Many thanks.

2

Could you just use the index sizes on elastic as a guide?

3

Hi,

Thanks for getting back to me.

Sorry could you elaborate more on that?

Thanks.

Regards,

Kevin

4

Are you splitting your indices based on days? If so, you could use cat API to view the size of the indices by visiting <ELASTIC_HOST>:9200/_cat/indices. Alternatively you could monitor the size of your index over a time period to get a rough idea of data rates.

5

I see thank you. I will give that a go. In this case I would not have to install metric beat on logstash server?

6

That would not require that, no. That can give some useful metrics so I would still recommend considering that as an option as well but it wouldn't provide this data as far as I am aware.

7

if you are using basic license, you could leverage the monitoring tab inside elasticsearch. just configure your logstash to use monitoring

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: ["1.2.2.3"]

After that the logstash node should pop up and give you information on how many events received and emitted are.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.