Combine indices into new index

[Diggy]

Hey, all.

The subject of my post probably isn't clear; here's what I'd like to at least try:

I have a logstash-* and filebeat-* index. In Kibana, depending on which I choose to view, of course, I'll set either logs from hosts forwarding via syslog, or those forwarding via filebeat. I was wondering if I could create a third index which contains both logstash- and filebeat-derived logs, and display that in Kibana.

Hoping you understand what I'm trying to accomplish. And, apologies if I've posted in the wrong forum.

Many thanks.

[warkolm (Mark Walkom)]

Why not just use an alias that points to both? https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-aliases.html#indices-aliases

[Diggy]

Mark, that worked great. Thanks!

[Diggy]

I've used this to create my alias and, thus, have Kibana output that combines both logstash and filebeat output:

curl -XPOST 'http://localhost:9200/_aliases' -d '
{
"actions" : [
{ "add" : { "indices" : ["logstash-", "filebeat-"], "alias" : "logs-*" } }
]
}'

However, every night at 10:00, output stops. I then have to remove and add back the alias, and am good again until 10:00 that night. I don't see anything that might cause this behavior. Does anyone have any ideas?

Thank you.

[warkolm (Mark Walkom)]

Because new indices are created every day at 0000UTC and you need to add the alias to the new indices.

You can update the various templates to include an alias if you want - https://www.elastic.co/guide/en/elasticsearch/reference/2.4/indices-templates.html#indices-templates - then it will be added automatically.