Elasticsearch for logging. HOW to configure automatic creation of the new index every day?


(Konstantin Erman) #1

Most of the guides I could find recommend creation of one index per day
when Elastic is used to store and query log files. Unfortunately not a
single guide dares to explain HOW exactly shall I configure freshly
installed Elastic to create new index every day
. Could somebody please
help me with it?

A few bits of additional info: I deal with Elastic on Windows Server (or
may be on Azure, but not any Linux) and I (plan) to send log events to
Elastic using Serilog. Any advise for those special circumstances
appreciated.

Thank you!
Konstantin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c7594fb8-7caf-4163-a2d9-b50b3c7b6994%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Kleiner) #2

Hello Konstantin,

You can use index value of name-%{+YYYY.MM.dd} in your elasticsearch
output in logstash

(link: http://logstash.net/docs/1.4.2/outputs/elasticsearch#index)

HTH,

David

On Tuesday, August 26, 2014 10:01:39 AM UTC-7, Konstantin Erman wrote:

Most of the guides I could find recommend creation of one index per day
when Elastic is used to store and query log files. Unfortunately not a
single guide dares to explain HOW exactly shall I configure freshly
installed Elastic to create new index every day
. Could somebody please
help me with it?

A few bits of additional info: I deal with Elastic on Windows Server (or
may be on Azure, but not any Linux) and I (plan) to send log events to
Elastic using Serilog. Any advise for those special circumstances
appreciated.

Thank you!
Konstantin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7c2fbf8d-1c5e-435d-945b-2e6baf012abe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Konstantin Erman) #3

Thank you, David!

I don't use Logstash mostly because I could not find an easy way to send
Serilog events to Logstash, while Elastic sink comes with Serilog in the
package.

But regardless, from your comment I realized that Indices are created at
the moment something is indexed into them and therefore there is no need to
tweak any configuration! That "one a day" behavior is achieved simply by
giving indices new name for every day. It looks trivial, but for
uninitiated take some mental effort to realize. :slight_smile:

Konstantin
On Tuesday, August 26, 2014 1:32:52 PM UTC-7, David Kleiner wrote:

Hello Konstantin,

You can use index value of name-%{+YYYY.MM.dd} in your elasticsearch
output in logstash

(link: http://logstash.net/docs/1.4.2/outputs/elasticsearch#index)

HTH,

David

On Tuesday, August 26, 2014 10:01:39 AM UTC-7, Konstantin Erman wrote:

Most of the guides I could find recommend creation of one index per day
when Elastic is used to store and query log files. Unfortunately not a
single guide dares to explain HOW exactly shall I configure freshly
installed Elastic to create new index every day
. Could somebody please
help me with it?

A few bits of additional info: I deal with Elastic on Windows Server (or
may be on Azure, but not any Linux) and I (plan) to send log events to
Elastic using Serilog. Any advise for those special circumstances
appreciated.

Thank you!
Konstantin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/48c66486-1a43-42ac-ae89-cb25408b8945%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Kleiner) #4

Quite right, I'm glad I pointed you in the right directions :slight_smile:

Cheers!

David

On Tuesday, August 26, 2014 4:26:31 PM UTC-7, Konstantin Erman wrote:

Thank you, David!

I don't use Logstash mostly because I could not find an easy way to send
Serilog events to Logstash, while Elastic sink comes with Serilog in the
package.

But regardless, from your comment I realized that Indices are created at
the moment something is indexed into them and therefore there is no need to
tweak any configuration! That "one a day" behavior is achieved simply by
giving indices new name for every day. It looks trivial, but for
uninitiated take some mental effort to realize. :slight_smile:

Konstantin
On Tuesday, August 26, 2014 1:32:52 PM UTC-7, David Kleiner wrote:

Hello Konstantin,

You can use index value of name-%{+YYYY.MM.dd} in your elasticsearch
output in logstash

(link: http://logstash.net/docs/1.4.2/outputs/elasticsearch#index)

HTH,

David

On Tuesday, August 26, 2014 10:01:39 AM UTC-7, Konstantin Erman wrote:

Most of the guides I could find recommend creation of one index per
day
when Elastic is used to store and query log files. Unfortunately
not a single guide dares to explain HOW exactly shall I configure
freshly installed Elastic to create new index every day
. Could
somebody please help me with it?

A few bits of additional info: I deal with Elastic on Windows Server (or
may be on Azure, but not any Linux) and I (plan) to send log events to
Elastic using Serilog. Any advise for those special circumstances
appreciated.

Thank you!
Konstantin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/78979624-0379-47cd-a5db-b5d4e6a36fb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5