Logstash jdbc type conflicts with statement field

Elastic Stack Logstash
1

i have a conf as follows

input {
jdbc {
statement => 'select type from item'
type => "item"
}
}

output {
elasticsearch {
document_type => '%{type}'
}
}

the output type value is from statement field.

what i want is the value defined by type => "item".

any sugguestions?

2

Having type => "item" in the input declaration will set the value of field type to item, regardless of what is in your statement. This is because Logstash "decorates" the event with whatever tags you define, as well as type, after the input plugin has completed its data collection.

As data leaves an input block it becomes an event, and the decorator fields tags and type are added to the event.

Unfortunately, that makes type a semi-protected field because

document_type => '%{type}'

is already the default behavior in the Elasticsearch output plugin code. If a type field exists, that value will always be used as the document_type. Because Elasticsearch requires a "type" for mapping (and this becomes the _type metadata field in Elasticsearch), if no type field exists in a Logstash event, a default of logs will be assigned.

If you omit or comment the line type => "item", it should work, as there will be no decorator added for type in this case. The other possibility is to change your SQL to select type from item as mytype, and use the field mytype instead.

1 Like
3

Thanks
problem solved.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.