JDBC and tracking_column(s) questions

Elastic Stack Logstash
1

I have 2 questions about the jdbc and tracking columns that my google foo isn't coming up with a good answer for.

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html

(my source server is a sql server 2008)

  1. How do you use multiple tracking_column at the same time. For example, my "table" is actually a view over multiple tables, so I need to have it select based on 2 columns (commandtime and commandid). I really don't want to have to build logstash task for each of the 10 tables in my view.

My query looks like:

"Select commandid, ...
from History
WHERE CommandID > :sql_last_value
Order by CommandTime asc,CommandID asc"

  1. How do I handle auto reset of the sql_last_value? Everyday the base tables are renamed and new tables created at around midnight.

So the tracked id needs be reset to 0.
Is there an easy way to detect that the current ID isn't valid (maybe expiring after 10 minutes or something)?

1 Like
2

How do you use multiple tracking_column at the same time. For example, my "table" is actually a view over multiple tables, so I need to have it select based on 2 columns (commandtime and commandid). I really don't want to have to build logstash task for each of the 10 tables in my view.

Concatenate those two columns in your SELECT clause and use that as the tracking column.

How do I handle auto reset of the sql_last_value? Everyday the base tables are renamed and new tables created at around midnight.

Won't this be fixed if you follow my suggestion above?

1 Like