(Running on windows, latest versions of elasticsearch and logstash)
I've seen a lot of posts with issues (and fixes) about the document_id with elasticsearch, but none seem to be exactly my problem.... Below is my entire .conf
input {
jdbc {
# jdbc connection string to our database
jdbc_connection_string => "jdbc:sqlserver://****"
# The user we wish to execute our statement as
jdbc_user => "****"
jdbc_password => "****"
# The path to our downloaded jdbc driver
jdbc_driver_library => "C:\Program Files\Microsoft JDBC Driver 6.0 for SQL Server\sqljdbc_6.0\enu\sqljdbc41.jar"
# The name of the driver class for sql
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
# our query
statement => "select EXID ,Name from myTable"
}
}
output {
elasticsearch {
index => "myIndex"
document_type => "myType"
document_id => "%{EXID}"
}
}
The thing is, when I do it like this, it DOES use the EXID as the document_id......but only the first one. So it grabs the first EXID and uses it as the document_id for all of them, so at the end, I still only have one document because they all replaced each other because they were all using the same id. If I take the document_id => "%{EXID}" line out and use the auto-generated ids, everything works fine. So it is just the delayed expansion that isn't working. What am I doing wrong?