Unable to use param in sql query and load data in local Elasticsearch index


#1

Hi folks

I use Logstash 2 and try to connect to a SQL Server database to load data.
My Logstash input config file section looks like this:

input {
  jdbc {
    jdbc_driver_library => "/opt/logstash/jtds-1.3.1.jar"
    jdbc_driver_class => "Java::net.sourceforge.jtds.jdbc.Driver"
    jdbc_connection_string => "jdbc:jtds:sqlserver://srvvppsmsql:3883/SM9DW"
    jdbc_user => "user"
    jdbc_password => "password"
    parameters => { "param1" => "INC00064174001A199877" }
    statement => "SELECT * FROM ACTIVITES WHERE ACT_ID = :param1"
  }
}

output section looks like this:

output {
    elasticsearch {
        index => "sm9"
        document_type => "activites"
        document_id => "%{ACT_ID}"
        hosts => localhost
    }
}

There is no filter section.

When I run Logstash using these parameters the system does not load any data. However it works when I remove parameters tag and use a simple query like "SELECT * FROM ACTIVITES". Do I do something wrong?

Moreover my ACTIVITES table contains more than 200 000 lines. But only 800 lines more or less are loaded. The system behavior is very surprising because when I watch the index state imported documents seems to increase and decrease with no reason during importing.

Of course log file is pretty empty.

Any Idea?


(system) #2