Dynamic index names

Elastic Stack Logstash
1

Hi.

I'm facing a problem with creating dynamic index names. I have few if statemnts like shown below to add new fields, but I can't see anything in kibana when I try to name index dynamically based on its value.

filter {
	if [host] =~ /pattern/ {
			mutate {
				add_field => {"site_code" => "code1"}
				add_field => {"region" => "region1"}
			}
		}
	else {
		mutate {
			add_field => {"site_code" => "code2"}
			add_field => {"region" => "region2"}
		}
	}
	
	Some parsing here...
}


output {
	elasticsearch {
		hosts => "http://address:9200"
		index => "syslog-%{[site_code]}-%{+YYY.MM.dd}"
		user => "name"
		password => "pass"
	}
}

I tried a different example where I'm using [host] field instead of [site_code] that I created and it works.

output {
	elasticsearch {
		hosts => "http://address:9200"
		index => "syslog-%{[host]}-%{+YYY.MM.dd}"
		user => "name"
		password => "pass"
	}
}

Is there any difference between [host] field and the [site_code] field that I created? Can I use it somehow to create dynamic indexes or am I doing something wrong?

2

Does logstash report any errors? Does the index show up at all in the index management page?

3

When using [site_code] field in index name i got this error: "Could not index event to Elasticsearch." and can't see it in index management. When using [host] field i can't see any error and index normally shows up in index management page.

4

Can you share the entire logstash line of the error when trying to use site_code? This line will say the reason why it can't index.

5

In recent versions that have ILM enabled the index option on the elasticsearch output is ignored.

6

Thank you all for your replies. I didn't know that I can't use uppercase in index name (found it in another error message). Everything works fine.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.