Oracle Data to ES using Logstash

Elastic Stack Logstash
1

Hello everyone,

I am trying to ingest Oracle data to ES using Logstash. But I got some errors in different conditions.
Is there a procedure or best practice to do this?

Thanks

2

Welcome to the community!

Can you explain what kind of errors do you have?
Please copy:

  • error
  • .conf
  • event data, at least a sample
3

Conf File
...
input {
jdbc {
jdbc_driver_library => "/usr/share/java/ojdbc8.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:@xx.xx.xx.xx:1521/db"
jdbc_user => "user"
jdbc_password => "password"
jdbc_paging_enabled => true
tracking_column => "timestamp"
use_column_value => true
tracking_column_type => "numeric"
schedule => "*/5 * * * * *"
statement => "SELECT * from table"
}
}
output {
Elasticsearch {
hosts => ["https://yy.yy.yy.yy:9200"]
index => "test01"
user => "elastic"
password => "qwerty"
document_id => "%{id}"
SSL => true
cacert => "/etc/elasticsearch/certs/http_ca.crt"
}
stdout { codec => "rubydebug"}
}

...

Error
...
[ERROR][logstash.outputs.elasticsearch][main][3c86d10b3884197b72b9f1d472ddb8113c95053bce9c42dec049000aa78c151a] An unknown error occurred sending a bulk request to Elasticsearch (will retry indefinitely) {:message=>""\xE8" from ASCII-8BIT to UTF-8", :exception=>LogStash::Json::GeneratorError, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/json.rb:41:in `jruby_dump'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:132:in `block in bulk'", "org/jruby/RubyArray.java:2813:in `map'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:132:in `block in bulk'", "org/jruby/RubyArray.java:1989:in `each'", "org/jruby/RubyEnumerable.java:1229:in `each_with_index'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:130:in `bulk'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:380:in `safe_bulk'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:252:in `submit'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:197:in `retrying_submit'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/outputs/elasticsearch.rb:397:in `multi_receive'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:121:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:304:in `block in start_workers'"]}

...

It is just a data table containing personal info.

4

Yes it's simple, however you have different encoding.

Check this. Also just try will work without setting document_id => "%{id}"

5

Unfortunately, it doesn't work.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.