Hi I am using log stash to output to csv -- it is working fine.
However I am trying to write from Elasticsearch to sql server using jdbc driver which is giving error.
Can you please try to pick what I am doing. Thank in advance.
I followed the help from here.
https://discuss.elastic.co/t/logstash-data-pipeline-from-elastic-to-sql-server/123589
Input {
elasticsearch {
hosts => "localhost:9200"
index => "idx-discoveryproduct"
user => "#####"
password =>"########"
query => '
{
"query": {
"bool": {
"must": [
{
"match": {
"cop": "United States"
}
},
{
"match": {
"author": "Source Wikia"
}
}
]
}
}
}
'
}
}
output {
jdbc{
driver_jar_path => 'C:\Work\ElasticSearch\logstash-8.1.2\vendor\jar\jdbc\sqljdbc_10.2\enu/mssql-jdbc-10.2.0.jre17.jar'
connection_string => “jdbc:sqlserver://xxx.xxx.xx.xxx;databaseName=TempFeed;user=Nxxxxxx;password=xxxxx”
statement => [ “INSERT into dataFeedStage (Isbn13, Audience, Author,SessionKey,COP,timeStamp) VALUES("isbn13","title","author","audiences","12345","cop",“@timestamp”)", “client_num", “consignee_name", “@timestamp","host" ]
}
}
Error I am getting is
Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError",
:message=>"Expected one of [ \\t\\r\\n], \"#\", [A-Za-z0-9_-], '\"', \"'\", [A-Za-z_], \"-\", [0-9], \"[\", \"{\" at line 40, column 22 (byte 675) after output {\r\n\r\njdbc{\r\ndriver_jar_path => 'C:\\Work\\ElasticSearch\\logstash-8.1.2\\vendor\\jar\\jdbc\\sqljdbc_10.2\\enu/mssql-jdbc-10.2.0.jre8.jar'\r\nconnection_string => ", :backtrace=>["C:/Work/ElasticSearch/logstash-8.1.2/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "C:/Work/ElasticSearch/logstash-8.1.2/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "C:/Work/ElasticSearch/logstash-8.1.2/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute'", "C:/Work/ElasticSearch/logstash-8.1.2/logstash-core/lib/logstash/agent.rb:376:in `block in converge_state'"]}
[2022-04-14T18:45:14,189][INFO ][logstash.runner ] Logstash shut down.
[2022-04-14T18:45:14,194][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
at C_3a_.Work.ElasticSearch.logstash_minus_8_dot_1_dot_2.lib.bootstrap.environment.<main>(C:\Work\ElasticSearch\logstash-8.1.2\lib\bootstrap\environment.rb:94) ~[?:?]