How to check per day logs size?

I want to check per day log size.It was very easy earlier because logstash created index on daily basis,but now we are using ILM policy , so it create new index when an index reaches to specific gb or it create new index after some period of time by default after 30 days
So i want to know how much log data we are getting per day.

GET /_cat/indices?v&pretty
health status index                                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   log-wlb-security-2021.04.06-000001    dmYSzsHASOaJ4eqkB3lxww   1   1     225579         7173    392.6mb        196.7mb
green  open   log-pb-icmp-2021.04.05-000001         NwzA1kw-TFm37IRcKknAGQ   1   1     437700            0    337.2mb        168.5mb
green  open   .items-default-000001                 DSJGqk3HQXOzoKGbvNgcpw   1   1          0            0       416b           208b
green  open   log-pb-flow-2021.04.05-000001         _HTx3o3iTEW25cP-anki-A   1   1   37767666            0     39.2gb         19.6gb
green  open   log-pb-tls-2021.05.05-000002          wiWn9yhrQ8yHwbV_zSLmsw   1   1      56450            0    256.9mb        127.9mb
green  open   .monitoring-kibana-7-mb-2021.05.05    wvTv-2nYSZC9QtehP86uVQ   1   1      17280            0      6.7mb          3.4mb
green  open   metricbeat-7.11.2-2021.03.18-000001   gHKZB662T6CRRWWwfVLiBA   1   1    4287891            0      2.9gb          1.4gb
green  open   .monitoring-kibana-7-mb-2021.05.07    lAsBPwJvTlugQwR7fUX9nw   1   1      17280            0      6.6mb          3.3mb
green  open   .monitoring-kibana-7-mb-2021.05.06    aUtahbGVTSGp2mB62JpojQ   1   1      17280            0      6.6mb          3.3mb
green  open   .monitoring-es-7-mb-2021.05.05        PDJvWlTUQd2of9ermOqPOg   1   1   63118153            0     92.4gb         46.2gb
green  open   log-pb-dns-2021.04.05-000001          y51OL9YBTBiZ4bFY-Kdupg   1   1     236493            0      455mb        226.7mb
green  open   .apm-custom-link                      treHQkxpSJ2XfGmd2Ierkg   1   1          0            0       416b           208b
green  open   .monitoring-kibana-7-mb-2021.05.09    sOZfPCawQYu2Q0V1WXOlEg   1   1      17280            0      6.5mb          3.2mb
green  open   log-wlb-sysmon-2021.04.06-000001      IAHprh30T6-rh2hxB67TLA   1   1   84049138        52926      100gb           50gb
green  open   .kibana_task_manager_1                2td3Ng_PQTaOo9fG3-N7Mg   1   1          9       657048    269.6mb        134.8mb
green  open   .monitoring-kibana-7-mb-2021.05.08    P-97Az_cSP2eZ94A17ySbw   1   1      17280            0      6.6mb          3.3mb
green  open   .monitoring-es-7-mb-2021.05.06        BKeoiWBvQv6S8CHKBXflAA   1   1   70087476            0     97.4gb         48.7gb
green  open   log-wlb-application-2021.05.06-000002 YRDLYvzAShuR3BgUu4jqyA   1   1      29455           41     24.9mb         12.5mb
green  open   .monitoring-es-7-mb-2021.05.07        lJTLdN0PSvCT5mNBrsDcRw   1   1   71872429            0     98.7gb         49.3gb
green  open   log-wlb-powershell-2021.04.06-000001  RqE2bkC2TDenqKEu-2dSnw   1   1       6450            0       34mb         16.9mb
green  open   .monitoring-es-7-mb-2021.05.08        D0YbvuDaR_KVsLqhBy0OVw   1   1   72268063            0    100.4gb         50.2gb
green  open   .monitoring-es-7-mb-2021.05.09        VmTxMW9jQMyMslvOwvQ4mg   1   1   72268143            0    100.2gb         50.1gb
green  open   log-wlb-wmiactivity-2021.05.06-000002 3sQ4DIm4QHGnYR2Jc_0fNA   1   1      22903           30       19mb          9.4mb
green  open   log-wlb-security-2021.05.06-000002    p9jMsvPaTxK1hg5Wy8xlyg   1   1      43670         2276     86.3mb         43.2mb
green  open   .apm-agent-configuration              6_CEmX6lT5ex-cDT3SDX7A   1   1          0            0       416b           208b
green  open   log-pb-icmp-2021.05.05-000002         liHOxFn6T5GMojxEunTNRA   1   1      56324            0     49.9mb           25mb
green  open   log-wlb-system-2021.05.06-000002      V0HoF0QWRgSKdusDCuczeg   1   1       1784            0      3.4mb          1.7mb
green  open   log-pb-dns-2021.05.05-000002          AA7i7xrdSsCfA73gBJncSA   1   1      44962            0    107.1mb         53.4mb
green  open   .monitoring-es-7-mb-2021.05.10        w3ppFwogQmCcEqxPs8M1Pw   1   1   72268307            0    101.5gb         50.7gb
green  open   log-wlb-powershell-2021.05.06-000002  zf705sGTS4u3XAJubdudPw   1   1        984            0      6.7mb          3.3mb
green  open   log-pb-tls-2021.04.05-000001          ZGu9rPQiT765hT_L0wun5g   1   1     254845            0    969.4mb        489.4mb
green  open   .monitoring-es-7-mb-2021.05.11        AF2ATrcQSGeVJTGbovch5A   1   1   32145937            0     47.4gb         24.6gb
green  open   log-wlb-application-2021.04.06-000001 yiS284-qRSeRawUw3KlLTw   1   1     164502           57    137.4mb         68.8mb
green  open   log-pb-http-2021.04.05-000001         OjapOmTCQa2k4xxgzA3-4w   1   1     159927            0    192.3mb         96.2mb
green  open   metricbeat-7.10.1-2021.04.29-000001   Nmb-DKXYQg6ZUgKGVY4U7g   1   1        567            0    201.9kb        100.3kb
green  open   log-pb-flow-2021.05.05-000002         CkJzzRBxQ1m1BDD3Rkqcjg   1   1    6754373            0      7.2gb          3.6gb
green  open   log-wlb-wmiactivity-2021.04.06-000001 Bj38q2I-RaKrvXb4i9CRhg   1   1     157641           49    132.5mb         66.3mb
green  open   log-wlb-system-2021.04.06-000001      tOBz6Pd8SBS67Vzo1JD-fA   1   1      11902            0       18mb          8.9mb
green  open   log-pb-http-2021.05.05-000002         6PcQu2Q6Q0q5-yqotcGD6Q   1   1      12820            0     24.7mb         12.6mb
green  open   .kibana-event-log-7.11.2-000002       JvM-kPorS5WgmubJ6s2upQ   1   1          0            0       416b           208b
green  open   .kibana-event-log-7.11.2-000001       5qA-0J8ETnq-J93JWnkJhA   1   1         29            0     52.7kb         26.3kb
green  open   .monitoring-logstash-7-mb-2021.05.11  O5CqO8NXRN6Ltsc8eO_Qww   1   1    1006869            0      138mb         69.1mb
green  open   .monitoring-logstash-7-mb-2021.05.10  pLPGAg-lTE-6IcOPa0NhKw   1   1    3547773            0    478.7mb        239.5mb
green  open   log-wlb-sysmon-2021.04.26-000002      p5ROhVjwT0Gpl3pgwu2bRA   1   1   84829663        79510      100gb           50gb
green  open   .lists-default-000001                 BdqP7KeqQ_iF1NHtGK8NtQ   1   1          0            0       416b           208b
green  open   log-wlb-sysmon-2021.04.26-000003      QSzhCbMPRQ2Yynils14EFA   1   1   31481947        41761     37.1gb         18.5gb
green  open   .kibana_1                             f58Dkr-PQIq4ozxf1jgvLw   1   1       1133         1145      4.9mb          2.4mb
green  open   .security-7                           Yja8FWWVSOGGqlT657YHMw   1   1         64            0      308kb          154kb
green  open   .monitoring-logstash-7-mb-2021.05.06  Bez0JXyWRpywYvSQrNobmw   1   1    3450338            0    468.8mb        234.6mb
green  open   metricbeat-7.11.2-2021.04.17-000002   r0ErrpsPS_iEOX02p-67kw   1   1    3370687            0      2.4gb          1.2gb
green  open   .monitoring-logstash-7-mb-2021.05.05  kcFCduT_T2yT64JK3DsqjA   1   1    3414156            0    465.6mb        232.8mb
green  open   .monitoring-logstash-7-mb-2021.05.08  g7swP2f2Roes1nvA2qm6Ag   1   1    3533763            0    475.7mb        237.9mb
green  open   log-pfsense-2021.04.12-000001         UWcklbIBSBCnPMWvBXd2sA   1   1   19806757            0      5.4gb          2.7gb
green  open   .monitoring-logstash-7-mb-2021.05.07  oXYjQ7_VRZGnexLnFbaGsw   1   1    3515315            0    476.6mb        238.3mb
green  open   .monitoring-logstash-7-mb-2021.05.09  -iSAeEXXR7OdPbPcwtGTDg   1   1    3533763            0    475.7mb        237.9mb
green  open   .monitoring-kibana-7-mb-2021.05.10    lXONhFLjQuyEARUnrUrCMQ   1   1      17280            0      6.7mb          3.3mb
green  open   .async-search                         50VWnbEfT32aMoE8WHvigQ   1   1         42           18     61.4kb         31.1kb
green  open   .monitoring-kibana-7-mb-2021.05.11    IQIKODhhQw-mAmxMd8Wo6w   1   1       4882            0        2mb            1mb

You raise a good point, as there is no easy way to grab this using ILM as you highlight.

You could find the start+end dates for each index, then figure out the average daily generation based on the size of the index.

FWIW I raised Show per day size stats for ILM indices · Issue #100009 · elastic/kibana · GitHub to see if we can do something here.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.