Logstash Http Configuration

Elastic Stack Logstash
1

Hi All,

I am trying to create multiple indices based on the input received to logstash via http. Here is my logstash configuration,

input {
http {
host => "0.0.0.0"
port => "2020"
type => ""
}
}

output{
elasticsearch{
hosts => "localhost:9200"
index => "%{type}-%{+MM.dd.YYYY}"
document_type => "data"
}
}

In this, "type" field under input tag needs to be populated based on the input request. For example, Say i am sending http request like this "http://0.0.0.0:2020/cpu" , then that cpu should get replaced in type field.

Similarly, for http request "http://0.0.0.0:2020/memory", memory should be replaced in type field.. Kindly help

2

First of all, this option is deprecated in a newer logstash version and I would suggest you to not use it and just make this indices routing per tag/custom field name, not per deprecated field.

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-document_type

You can do that by:

  • in a filter section catch that request, parse it with grok, and assign the /memory , /cpu etc. to the field you want it to be stored.
  • in the output section you can do the following data allocation as you are attempting to do.

Example:

input {
	http {
		host => "0.0.0.0"
		port => "2020"
		type => ""
	}
}

filter {
	grok {
		match => {
			"message" => "^http://%{DATA}/%{DATA:request}$"
		}
	}
}
output{
	elasticsearch{
		hosts => "localhost:9200"
		index => "%{request}-%{+MM.dd.YYYY}"
	}
}
3

@pastechecker : Thanks a lot Charlie. That's works perfectly fine for me :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.