Topbeat change settings when to gather stats

Elastic Stack Beats
1

I find a strange problem in my ELK setup. I have topbeat configured to gather statistics every 3600s using:
input:

In seconds, defines how often to read server statistics

period: 3600

And it sends output to log stash which in turn sends to elastic search and then indexed and stored. However, I find that the indices are created everyday only once. Below is the indices files i saw:
ls -ltr|grep topbeat
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 19 16:23 topbeat-2016.05.19
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 20 10:23 topbeat-2016.05.20
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 20 21:23 topbeat-2016.05.21
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 21 21:23 topbeat-2016.05.22
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 22 21:23 topbeat-2016.05.23
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 23 21:23 topbeat-2016.05.24
drwxr-xr-x 8 elasticsearch elasticsearch 4096 may 24 21:23 topbeat-2016.05.25

It seems to get created at 21:23 everyday, any clue as to what settings governs this appreciated.

2

What timezone is being used for the ls output?

The new indices will be created when the first event of the new day is reported. And the new day starts at 00:00 UTC (not your local timezone).

3

Its ART(Argentina timezone) . How do i check which timezone is set at topbeat level?

4

On querying elastic search, I see that the latest data send by topbeat->log stash->elastic search, is that of 21:23 of the previous day. Say i run a query on May 26th 12 pm, the latest data i will find is May 25th 21:23 . How do i configure topbeat to fetch more near real time statistics. I have already given period:3600. So i thought every 3600s, system statistics would be gather which is not apparently happening. Any help please

5

ART timezone is UTC-3 which means 21:23 ART is equivalent to 00:23 UTC. So at the 23rd minute of every hour Topbeat reports metrics, and since 00:23 UTC is a new day, a new daily index is created.

Topbeat doesn't care about the host's timezone, it uses UTC.

Fetching metrics every one hour is not real-time IMO. The default reporting period for Topbeat is 10 seconds. Try running with the default value for a bit.

6

Cant i configure so that the index creation happens like every 30 minutes or so instead of index being created for just once a day?
As per current settings, only metrics for the previous day per hour is available. To get todays metrics per hour loaded in elastic search , I will have to wait till 21:23 ART

7

Thanks andrew for the help. It seems to be working now. Apparently topbeat was crashing due to "divide by zero" error. I had changed output bulk_max_size to 0 which was causing the issue. Changed it back to 1024 and that did the trick.