Use logstash mapping for all future indicies

Elastic Stack Elasticsearch
1

Hello there,

Is there a way to assign the mapping for the logstash index to all future indices created? I can create indices super easy, but having to define each field is a little time consuming, and so far, I have found the default one for logstash to be universally useful and time efficient :smiley: If I name my index "logstash-windowsevents" then the mapping for the logstash-* index works just fine, but I would like the data in a whole new index if possible.

Many thanks

2

Logstash uses a template - https://www.elastic.co/guide/en/elasticsearch/reference/6.1/indices-templates.html

So you can take that and modify how you wish :slight_smile:

3

Thanks, but I don't quite follow what you're saying. I do not see anything in Elasticsearch as to what template the logstash index is using. I do not wish to modify anything at all at this point in time. I want to use the exact mapping for the logstash index for all indices. Maybe there is something being said that I am not picking up or something. I'm 99% sure it's me :laughing:

4

Did you check out the _template API? That'll give you an idea of what's happening.

You will need to modify things if you want it for all indices. If you look at that link from earlier you will see the index_patterns declarative, that is where it knows what index patterns to apply the mappings to. Which is what you're trying to do :slight_smile:

5

Ah ok that makes a little more sense. Thank you for putting that into perspective. I was able to get some info from

curl -XGET localhost:9200/_template

but I cannot seem to find how to edit an existing template.

Completly spitballing it here, but I tried this

curl -XPUT localhost:9200/_template/logstash -d '
{
"template": "*"
}
'

That changed everything in the template instead of just changing what I intended. The only thing left is

{"logstash":{"order":0,"template":"-*","settings":{},"mappings":{},"aliases":{}}}

while the original had a bunch of stuff. I think I was able to fix that part by copying the original GET request and pasting that into the curl. The only thing I changed was the "template" section form "logstash-*" to just "*". I think it will work lol I will post back when I check on some data.

curl -XGET localhost:9200/_template
{"logstash":{"order":0,"template":"*","settings":{"index":{"refresh_interval":"5s"}},"mappings":{"_default_":{"dynamic_templates":[{"message_field":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit_norms":true,"type":"string"},"match_mapping_type":"string","match":"message"}},{"string_fields":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit_norms":true,"type":"string","fields":{"raw":{"ignore_above":256,"index":"not_analyzed","type":"string"}}},"match_mapping_type":"string","match":"*"}}],"_all":{"omit_norms":true,"enabled":true},"properties":{"@timestamp":{"type":"date"},"geoip":{"dynamic":true,"properties":{"ip":{"type":"ip"},"latitude":{"type":"float"},"location":{"type":"geo_point"},"longitude":{"type":"float"}}},"@version":{"index":"not_analyzed","type":"string"}}}},"aliases":{}}}
6

You are better off giving things specific names, cause if you restart/upgrade Logstash then it'll overwrite that template anyway.

7

Ah ok well I thought I had an easy solution :smile: I will create other templates and copy/paste the mapping from the logstash index. This should get me by until I get more exp with Elasticsearch. Thank you for putting all of this into a better perspective for me.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.