[HELP?] Can only import last entry using JDBC with Logstash and ES

Elastic Stack Logstash
1

So I'm new to ELK and I'm trying to import data from a postgres DB into ES for searching within Kibana.

I have the JDBC plugin all set up and I have a config file to import data. It runs fine but I only seem to get the last entry from the postgres database appear in kibana. I want all the data in there.

Here is my config file:

file: test.conf

input {
jdbc {
# Postgres jdbc connection string to test database, raptor_db
jdbc_connection_string => "jdbc:postgresql://localhost:5433/raptor_db"
# the user we wish to execute our statement as
jdbc_user => ""
jdbc_password => ""
# the path to our downloaded jdbc driver
jdbc_driver_library => "/postgresql-9.4.1208.jre6.jar"
# name of the driver class for Postgresql
jdbc_driver_class => "org.postgresql.Driver"
#the query
statement => "SELECT * from prefetch"
jdbc_paging_enabled => true
}

}

output {
elasticsearch {
index => "prefetch"
document_type => "prefetch_item"
document_id => "%{uid}"
hosts => "localhost:9200"
}
}

Like I said, it imports a single entry just fine but not all of them.

If someone could point me in a good direction I would be very grateful!

thanks,

Andy

2

And the uid column is unique for each row in the database?

3

This was the problem, my "document_id" field was wrong. I actually just removed it and it sorted the problem out.

thanks magnusbaeck for getting back so quickly!