Is it possible to have a custom document_id in Logstash?

[Kulasangar_Gowrisang (Kulasangar Gowrisangar)]

I'm having two jdbc filters (but the same connection parameters, apart from the MySQL query) in my input part where i'm trying to get data from the database and push it to the index. As I said the only difference in both of those jdbc connections is the MySQL query, where I'm trying to get data from two different tables (primary key for both is a column named id).

What i'm trying to do is to push both the data to a single index, where I'm trying to differentiate both the table records by the document_id. So I'm trying to filter both the jdbc connections by type in the output and assign document_id with a prefix so that I could search from ES using the document_id:

if [type] == "sms1"{
	elasticsearch {
	        index => "test1"
		document_type => "message_logs"
		document_id => "sms1_%{id}" <-- id is the primary key
		action => index
		hosts => "myhost"
	}
} else {
	elasticsearch {
		index => "test1"
		document_type => "message_logs"
		document_id => "sms2_%{id}" <-- id is the primary key
		action => index
		hosts => "myhost"
	}
}

Is it possible to add a prefix as such to the document_id, or is there any other way I can handle this? Any help could be appreciated.

[jsvd (João Duarte)]

I'm not fully sure I understand the question, but you could just do

output {
	elasticsearch {
		index => "test1"
		document_type => "message_logs"
		document_id => "%{type}-%{id}"
		action => index
		hosts => "myhost"
	}
}

this way you only need one output that will generate ids based on the type and id fields.

[Kulasangar_Gowrisang (Kulasangar Gowrisangar)]

My above configuration worked. Your suggestion looks even more better and clear

Thanks!