Need help in listing syslog data (10000 +)

[nisaxena (Nitin)]

Hi There,

I am new to ELK and need help. I am working on syslog data collected from all zonal rsyslog servers. Now I am putting some visualization to this data. there are around 10000+ unique hosts data collected everyday.
Now, I wish to create a dashboard to keep a track of

a. List of hosts reported their logs everyday .... I am able to plot it but in numbers .. how can I get the list of hosts ?
b. I also wish to pull the list of hosts not reported their logs and/or the list of extra hosts reported their logs ?

Thanks,
Nitin

[warkolm (Mark Walkom)]

That's a lot of hosts to just list out in a big table. Given you want to also show hosts that haven't reported, why do you need the list of ones that have?

For the missing hosts, you can use Alerting and then something like this.

[nisaxena (Nitin)]

Hi Mark,

My setup is big, there are 10000+ hosts reporting logs everyday and this number will keep on growing.

Need suggestion on how should I keep this data for easy listing / retrieval in visualisation.

A simple one for now, How can I visualise this data and get the list (not number) of all unique reported host on a day ?

So far I have created a single host, will a cluster pair help in this scenario ? Will it help in fast listing of data ?

Thanks,
Nitin

[warkolm (Mark Walkom)]

Why do you want to see over 10000 hosts in a table like that though? That is not really usable.

[nisaxena (Nitin)]

Hi Mark,

I am talking about visualization of data here at vertical / horizontal bar or any other visualization.

My Requirement, From the single Index logs-* ( logs-YYYY-MM-DD ) I just wanted to see list of unique reported hosts.

Thanks,