Problem with jdbc_streaming

Elastic Stack Logstash
1

Good morning,

please i have some quetions about jdbc_streaming , i made such a configuration to extract some data using jdbc_streaming but i didn't acheive to my goal .

there is my configuration file
jdbc_streaming {
jdbc_driver_library => ".............."
jdbc_driver_class => "................."
jdbc_connection_string => "............."
jdbc_user => "........"
jdbc_password => "........"
parameters => {"variable"=>"field2"}
statement => "SELECT field5 From database2 where field4 =:variable"

target => "code"
use_cache => "true"
cache_expiration => "5"
cache_size => "100"

}
when i run logstash on mode debug i have code=>[ field5=>"valueoffield5"]

but what i want is added code to first database with value directly not code=>[ field5=>"valueoffield5"]

Please need some help

3

would you please put an example log you expect?

If your problem is just that the field you want is nested for example:

{
          "svc" : "example",
          "user_agent" : "/1.1/iOS/10.1.1/iPhone MDL",
          "ts" : "2017-05-08T15:15:21.612+02:00",
          "code" :{
              "field5":"valueoffield5"
          }    
 }

but you simply want :
{"code":"valueoffield5"}

then I think you could use the alterfilter that uses the coalesce function.

alter {
        coalesce => [
             "code", "%{[code][field5]}"]
      }

that will create the field you want with the value you want. Then you just remove the old one. It is not a pretty solution but I think it should work IF I understood correctly your problem...

5

hello! thank you for your reply
this is how look my configuration file:
jdbc_streaming
{

    jdbc_driver_library =>                                                                                               
    jdbc_driver_class =>                                                                                                                           
    jdbc_connection_string =>                                                                                                             
    jdbc_user => ""                                                                                                                                                     
    jdbc_password => ""                                                                                                                                                 
    statement => "SELECT field5 From database2 where field2 = variable"         
    parameters => {"variable"=>"field2"}                                                                                                                       
    target => "code"

}

field2 is from database1 that i extract already

actually when i run logstash without mode debug i have nothing in output, and when i run on mode debug i have
{"event"=>{"code"=>[ "field5":"valueoffield5"] } as you said
and i haven't any output in elasticsearch

6

I suggest you to use stdout{codec=> rubydebug} as output so you can test the log you are sending to whatever output you need.

I think I didn't really understand your problem then, your log has been enriched? Your problem is in the ouput filter?

As I said, put a complete configuration example you are using, put an example log you want.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.