How to export `alerts` as backup?

fzyzcjy · January 21, 2021, 10:28am

Hi thanks for the lib! I am trying to monitor my server by using monitoring and alerting. When, for example, CPU is too high or memory is almost full, I want to send an email to myself. Thus I use Kibana Alerting (is it correct to use this?).

The problem is, how can I backup those alerts I created? I have tried to backup those "saved objects" but there seems no alerts.

Thanks!

weltenwort · January 21, 2021, 10:49am

Hi @fzyzcjy,

that functionality is still being worked on (see https://github.com/elastic/kibana/issues/50266). It's non-trivial because of the security implications of exporting the captured credentials.

fzyzcjy · January 25, 2021, 6:15am

Hmm thanks all the same!

fzyzcjy · January 30, 2021, 3:34am

Hi, how can I backup/restore those alerts currently? Thanks

weltenwort · February 3, 2021, 10:08am

Hi @fzyzcjy, I'm trying to find out if there is an option aside from backup up the .kibana system index directly.

fzyzcjy · February 6, 2021, 1:05am

Hi is there any updates? We know backup is very important, so I cannot let the elastic stack run without any backups!

stephenb · February 6, 2021, 2:06am

Hi @fzyzcjy just to be clear

With respect to Alerts

The Alerts are backed up as part of the normal snapshot and restore mechanism.
As @weltenwort said Import / Export of Alerts Saved Object is underway (working through security mechanism) . We do not have an ETA for that yet but it is a highly requested feature.
Also and API for directly Creating, Update, Deleting Retrieving alerts is also underway (nearly complete in Docs Stage). I do not have an exact ETA but that should be coming soon as well.

fzyzcjy · February 6, 2021, 2:31am

Sounds wonderful! However, I do not see the backuped alerts:

(the policy:)

Am I missing something? Do I need to tune some configuration to backup? Or is it actually backuped (and just not shown?)

Thanks!

stephenb · February 6, 2021, 6:13pm

It appears that your current snapshot policies is only backing up 4 specific indices. Your snapshot policy is backing up a very limited set of indices and it does not appear it is backing up any of the system indices which may or may not be risky depending on your overall strategy. You will need to create a policy that backs up the kibana system indices at the very least, perhaps you should consider snapshot policy that covers all the system indices ... in general they tend to be small in comparison to data indices.

My snapshot policy is backing up everything

Including the .kibana system indicies

fzyzcjy · February 6, 2021, 3:15am

Thanks very much! In my case, I should backup system index (though not done yet), and I want to backup those 4 indices shown above (post/user_metadata/...), but I do not want to backup filebeat & metricbeat as they are too large. (By the way, is it a good or bad idea to backup filebeat/metricbeat data?)

Thus, should I select each and every one of those system indices one by one in this panel? Or is there any automatic way (like specifying an index pattern - which index pattern should I write down, maybe .* or something else?)

Thanks!

stephenb · February 6, 2021, 3:27am

See Here... so you should be able to use the .* syntax but I would run and test...

indices
(Optional, string) A comma-separated list of data streams and indices to include in the snapshot. Multi-index syntax is supported.

fzyzcjy · February 6, 2021, 3:49am

Thanks very much!

fzyzcjy · February 6, 2021, 3:58am

Hmm wait a bit... Some indices are quite huge!

Firstly, .* causes the following to be backup:

.apm-agent-configuration
.apm-custom-link
.async-search
.items-default-000001
.kibana-event-log-7.10.2-000001
.kibana-event-log-7.9.0-000001
.kibana-event-log-7.9.0-000002
.kibana-event-log-7.9.0-000003
.kibana_1
.kibana_2
.kibana_security_session_1
.kibana_task_manager_1
.kibana_task_manager_2
.lists-default-000001
.monitoring-beats-7-mb-2021.01.31
.monitoring-beats-7-mb-2021.02.01
.monitoring-beats-7-mb-2021.02.02
.monitoring-beats-7-mb-2021.02.03
.monitoring-beats-7-mb-2021.02.04
.monitoring-beats-7-mb-2021.02.05
.monitoring-beats-7-mb-2021.02.06
.monitoring-es-7-mb-2021.01.31
.monitoring-es-7-mb-2021.02.01
.monitoring-es-7-mb-2021.02.02
.monitoring-es-7-mb-2021.02.03
.monitoring-es-7-mb-2021.02.04
.monitoring-es-7-mb-2021.02.05
.monitoring-es-7-mb-2021.02.06
.monitoring-kibana-7-2021.01.31
.monitoring-kibana-7-2021.02.01
.monitoring-kibana-7-2021.02.02
.monitoring-kibana-7-2021.02.03
.monitoring-kibana-7-2021.02.04
.monitoring-kibana-7-2021.02.05
.monitoring-kibana-7-2021.02.06
.monitoring-kibana-7-mb-2021.01.31
.monitoring-kibana-7-mb-2021.02.01
.monitoring-kibana-7-mb-2021.02.02
.monitoring-kibana-7-mb-2021.02.03
.monitoring-kibana-7-mb-2021.02.04
.monitoring-kibana-7-mb-2021.02.05
.monitoring-kibana-7-mb-2021.02.06
.security-7
.siem-signals-default-000001
.slm-history-2-000001
.slm-history-3-000001
.transform-internal-005
.transform-notifications-000002

Secondly, some indices are huge:

So I use this one instead:

Result:

Thanks for the help!