Mapping per index

Elastic Stack Elasticsearch
1

Hello,
I want to have different indexes depending on the type, ie if the type is
syslog, I want a syslog-logstash-DATE index. That is easy to achive then I
want to have a mapping per index. That is have one template per index, is
there a way to do that. I have tried something like this:

input {
redis {
host => "127.0.0.1"
data_type => "list"
type => "redis-input"
key => "logstash"
}
syslog {
type => syslog
port => 5514
}
}
output {
stdout { }
elasticsearch {
cluster => "logstash"
index => "%{type}-logstash-%{+YYYY.MM.dd}"
template => "/etc/logstash/conf.d/%{type}-template.json"
}
}

But it does not work, is there a way to do it, besides doing it manually?

Regards

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c31a8b93-bc7b-47eb-afc1-83859f138660%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

The best way I can think of would be to use conditionals in your output,
one per type, then you can directly point to specific templates for each
type.

On 5 November 2014 15:41, Alejandro Alves alejandro.alves@gmail.com wrote:

Hello,
I want to have different indexes depending on the type, ie if the type is
syslog, I want a syslog-logstash-DATE index. That is easy to achive then I
want to have a mapping per index. That is have one template per index, is
there a way to do that. I have tried something like this:

input {
redis {
host => "127.0.0.1"
data_type => "list"
type => "redis-input"
key => "logstash"
}
syslog {
type => syslog
port => 5514
}
}
output {
stdout { }
elasticsearch {
cluster => "logstash"
index => "%{type}-logstash-%{+YYYY.MM.dd}"
template => "/etc/logstash/conf.d/%{type}-template.json"
}
}

But it does not work, is there a way to do it, besides doing it manually?

Regards

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c31a8b93-bc7b-47eb-afc1-83859f138660%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c31a8b93-bc7b-47eb-afc1-83859f138660%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZnUyGgqH%2BbvsENsO9B%3DqTccmskYSoYVkVaFEr8pctYXmQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hello,
Yes, I thought about that, but we wanted something more dynamic, so we do
not have to register every type, so if we have a new logstash shipper with
a new type, we do not have to do anything in the central logstash.
Thanks!

El miércoles, 5 de noviembre de 2014 17:56:23 UTC+13, Mark Walkom escribió:

The best way I can think of would be to use conditionals in your output,
one per type, then you can directly point to specific templates for each
type.

On 5 November 2014 15:41, Alejandro Alves <alejand...@gmail.com
<javascript:>> wrote:

Hello,
I want to have different indexes depending on the type, ie if the type is
syslog, I want a syslog-logstash-DATE index. That is easy to achive then I
want to have a mapping per index. That is have one template per index, is
there a way to do that. I have tried something like this:

input {
redis {
host => "127.0.0.1"
data_type => "list"
type => "redis-input"
key => "logstash"
}
syslog {
type => syslog
port => 5514
}
}
output {
stdout { }
elasticsearch {
cluster => "logstash"
index => "%{type}-logstash-%{+YYYY.MM.dd}"
template => "/etc/logstash/conf.d/%{type}-template.json"
}
}

But it does not work, is there a way to do it, besides doing it manually?

Regards

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c31a8b93-bc7b-47eb-afc1-83859f138660%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c31a8b93-bc7b-47eb-afc1-83859f138660%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6602e3cd-b7ee-4e8f-ad18-9174781e0877%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

I have created a pull request which creates this feature

El miércoles, 5 de noviembre de 2014 17:41:07 UTC+13, Alejandro Alves
escribió:

Hello,
I want to have different indexes depending on the type, ie if the type is
syslog, I want a syslog-logstash-DATE index. That is easy to achive then I
want to have a mapping per index. That is have one template per index, is
there a way to do that. I have tried something like this:

input {
redis {
host => "127.0.0.1"
data_type => "list"
type => "redis-input"
key => "logstash"
}
syslog {
type => syslog
port => 5514
}
}
output {
stdout { }
elasticsearch {
cluster => "logstash"
index => "%{type}-logstash-%{+YYYY.MM.dd}"
template => "/etc/logstash/conf.d/%{type}-template.json"
}
}

But it does not work, is there a way to do it, besides doing it manually?

Regards

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6a4cc76e-527a-43b7-8518-5f57972b4b05%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.