Stdout codec json_lines is empty

Elastic Stack Logstash
1

Hi there,

I'm still trying to connect my postgresql database to elastic. I've got all the required files:
logstash.yml, pipelines.yml and p1.conf (in my local/etc/logstash). However I'm only getting Successfully started Logstash API endpoint and no result, if I run logstash --debug I get: stdout { codec => json_lines } ```]

So it's empty...
Here are some lines I'm also getting from the debug:

config LogStash::Inputs::Jdbc/@statement = "SELECT * from phpbb_banlist"
config LogStash::Inputs::Jdbc/@jdbc_validation_timeout = 3600
config LogStash::Inputs::Jdbc/@connection_retry_attempts = 1
config LogStash::Inputs::Jdbc/@connection_retry_attempts_wait_time = 0.5

[DEBUG][org.logstash.ackedqueue.io.MmapPageIOV2] PageIO recovery element index:0, readNextElement exception: Element invalid length

[DEBUG][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"my-pipeline_1"}

[WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][my-pipeline_1] A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team.

[INFO ][logstash.javapipeline ][my-pipeline_1] Starting pipeline {:pipeline_id=>"my-pipeline_1", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/usr/local/etc/logstash/p1.conf"], :thread=>"#<Thread:0x2dc0c9bc run>"}

[logstash.javapipeline ][my-pipeline_1] Pipeline started {"pipeline.id"=>"my-pipeline_1"}

[DEBUG][logstash.javapipeline ] Pipeline started successfully {:pipeline_id=>"my-pipeline_1", :thread=>"#<Thread:0x2dc0c9bc run>"}

[DEBUG][org.logstash.execution.PeriodicFlush][my-pipeline_1] Pushing flush onto pipeline.

[INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:"my-pipeline_1"], :non_running_pipelines=>[]}

[DEBUG][logstash.agent ] Starting puma

[DEBUG][logstash.agent ] Trying to start WebServer {:port=>9600}

[DEBUG][logstash.api.service ] [api-service] start

[INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

[DEBUG][logstash.inputs.jdbc ][my-pipeline_1] loading ~/Desktop/postgresql-42.2.9.jar

[DEBUG][logstash.inputs.jdbc ] config LogStash::Inputs::Jdbc/@use_prepared_statements = false
[DEBUG][logstash.inputs.jdbc ] config LogStash::Inputs::Jdbc/@prepared_statement_name = ""

Maybe somebody can help me to sync my postgresql database with elastic (because it's taking forever this way), I've been searching the web for a solution for a long time. I was thinking the problem is the prepared statement being empty but my .conf file is all in order and as simple as can be.

2

Also my configuration file is like the following:

input {
jdbc {
# Postgres jdbc connection string to our database,
jdbc_connection_string => "jdbc:postgresql://hostname:port/databasename"

    # The user we wish to execute our statement as
    jdbc_user => "USER"

#password for the user
jdbc_password => "PASSWORD"

jdbc_validate_connection => true

    # The path to our downloaded jdbc driver
    jdbc_driver_library => "~/Desktop/postgresql-42.2.9.jar"

    # The name of the driver class for Postgresql
    jdbc_driver_class => "org.postgresql.Driver"

    # our query
    statement => 'select * from phpbb_banlist'
}

}
output {
stdout { codec => json_lines }
}

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.