Hello,
I'm very new to all this Elasticsearch and logstash stuff.
I'm trying to import my SQLServer Database to Elastic Search, and i don't understand well how Logstash work.
I found that's i need to use the jdbc plugin.
So i tried something like this :
input { jdbc { jdbc_driver_library => "C:\Users\****\Documents\sqljdbc_6.0\enu\jre8\sqljdbc42.jar" jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc_connection_string => "jdbc:sqlserver://127.0.0.1:61672;databasename=[DB];integratedSecurity=true;" jdbc_user => "****" jdbc_password => "****" statement => "SELECT * FROM [DB].Tables" } } filter { } output { elasticsearch { hosts => "localhost" index => "index" document_type => "document" document_id => "%{id}" } stdout { codec => rubydebug } }
I'm using Jdk 8, elasticsearch 5.4.0 and logstash 5.4.0
But i get the following Error :
# # A fatal error has been detected by the Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006c1af2f5, pid=10552, tid=0x0000000000001380 # # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode windows-amd64 compressed oops) # Problematic frame: # V [jvm.dll+0x13f2f5] # # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows # # An error report file with more information is saved as: # C:\Users\lxi\Documents\BigData\logstash-5.4.0\hs_err_pid10552.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp #
after some logstash logs :
Could not find log4j2 configuration at path /Users/***/Documents/logstash-5.4.0/config/log4j2.properties. Using default config which logs to console 11:29:20.802 [main] WARN logstash.runner - --config.debug was specified, but log.level was not set to 'debug'! No config info will be logged. 11:29:21.822 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} 11:29:21.824 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"} 11:29:21.974 [[main]-pipeline-manager] WARN logstash.outputs.elasticsearch - Restored connection to ES instance {:url=>#<URI::HTTP:0x7f60d971 URL:http://localhost:9200/>} 11:29:21.976 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Using mapping template from {:path=>nil} 11:29:22.273 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_po int"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}} 11:29:22.282 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0xae7a8f0 URL://localhost>]} 11:29:22.287 [[main]-pipeline-manager] INFO logstash.pipeline - Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
Looks like a compatibility problem with jvm.dll but i don't know so much in Java or logstash.
I don't know if this is the good topic to post.
Thanks in advance.