How to set the document id dynamically with the column value from the table

Elastic Stack Logstash
1

I want to set the id value for each document in the elastic search.

I am trying to set the account_no as an id. However, when I tried to do, the id is set as the string value "%{account_no}".

The statement document_id => "%{account_no}" is not fetching the value from the input filter.

   #jdbc.conf
    input {
        jdbc {
                # path of the jdbc driver
                jdbc_driver_library => "C:\Ashok\ojdbc8-full\ojdbc8.jar"

                # The name of the driver class 
                jdbc_driver_class => "oracle.jdbc.Driver"

                # Mysql jdbc connection string to company database
                jdbc_connection_string => "jdbc:oracle:thin:@new-connection:1234:dbsource"
            
                # user credentials to connect to the DB
                jdbc_user => "xyz"
                jdbc_password => "xyz"

                # when to periodically run statement, cron format (ex: every 30 minutes)
                schedule => "30 * * * *"

                # query parameters
                parameters => { "account_no" => "213072717" }

                # sql statement 
                statement => "SELECT * FROM accounts WHERE account_no= :account_no 
            }
    }

    output {
        elasticsearch {
          index => "bank"
          document_type => "account"
          hosts => "localhost:9200"
     document_id => "%{account_no}"
        }
    }
2

That suggests that your events do not have an [account_no] field on them. Check using

output { stdout { codec => rubydebug } }
3

You are right, the column name is in caps. Looks like it's case sensitive.

Thank you @Badger

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.