Hello guys,
I'm new in logstash. I have a question how I can return _source field from elastic input to jdbc output. This is necessary for getting main fields and _source should be stored like varchar(max)
For example: table looks like this:
DataTable(id varchar(38), timestamp datetime, source nvarchar(max))
I read a lot of documents but still not have an a picture how I can do this. My pipeline:
input
{
elasticsearch {
hosts => "192.168.9.28:9200"
index => "index*"
query => '{"query":{"bool":{"filter":[{"bool":{"should":[{"match":{"API-Action":"APIRequest"}},{"match":{"API-Action":"APIResponse"}}],"minimum_number_should_match":1}},{"range":{"timestamp":{"gte":"now-1m/m","lt":"now/m"}}}]}},"size":10000}'
docinfo => true
size => 10000
scroll => "1m"
}
}
output {
jdbc {
driver_jar_path => '/opt/jdbc/sqljdbc41.jar'
connection_string => "jdbc:sqlserver://<servername>;databaseName=<databasename>;user=<user>;password=<password>"
statement => ["INSERT INTO log ([APIAction], [Timestamp], [source]) VALUES(?, ?, ?)", "%{[@metadata][_id]}", "timestamp", "%{[@metadata][_source]}"]
codec => json
}
}
Could some one please describe where I'm wrong? Or how ww can do this? Also I read about "target" option for elastic input plugin but there are says that I can't use it on output