Elasticsearch Engineer I Exception

Elastic Community and Ecosystem Elastic Training
1

I'm currently working through the Elastic Engineer I (On-Demand) course and I am stuck on Lab 1.4

In trying to setup Logstash
./logstash-7.3.1/bin/logstash -f datasets/blogs_sql.conf

I get the following as output:

[2020-06-09T16:27:49,162][ERROR][logstash.inputs.jdbc     ] Unable to connect to database. Tried 1 times {:error_message=>"Java::OrgPostgresqlUtil::PSQLException: The connecti
on attempt failed."}                                                                                                                                                           
[2020-06-09T16:27:49,166][ERROR][logstash.javapipeline    ] A plugin had an unrecoverable error. Will restart this plugin.                                                     
  Pipeline_id:main                                                                                                                                                             
  Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"postgres", jdbc_password=><password>, statement=>"SELECT * from blogs", jdbc_driver_library=>"/home/elastic/postgresql-42.2.9.jar
", jdbc_connection_string=>"jdbc:postgresql://db_server:5432/", id=>"a9d0f3110d44787ac0e892416603c3654611ac23e390369c6e712c0e89499869", jdbc_driver_class=>"org.postgresql.Driv
er", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_e43e7846-d7a3-4712-b752-838f2c7ce430", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false,
 jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_
retry_attempts_wait_time=>0.5, parameters=>{"sql_last_value"=>1970-01-01 00:00:00 UTC}, last_run_metadata_path=>"/home/elastic/.logstash_jdbc_last_run", use_column_value=>fals
e, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true>                                                                     
  Error: Java::OrgPostgresqlUtil::PSQLException: The connection attempt failed.                                                                                                
  Exception: Sequel::DatabaseConnectionError                                                                                                                                   
  Stack: org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(org/postgresql/core/v3/ConnectionFactoryImpl.java:292)

This then just repeats as it continues to try and connect.
Any help would be greatly appreciated.

2

Hi Lindsay, Can you share your datasets/blogs_sql.conf?

3

Hi Mark,

Below is my datasets/blogs_sql.conf:

##############################################################################                                                                                                 
#                                                                            #                                                                                                 
# Logstash config file that ingests blog posts from a Postgres database.     #                                                                                                 
# Database has a blogs table with the following fields:                      #                                                                                                 
#   id, title, seo_title, url, author, date, category, locales, content      #                                                                                                 
#                                                                            #                                                                                                 
##############################################################################                                                                                                 
                                                                                                                                                                               
input {                                                                                                                                                                        
  jdbc {                                                                                                                                                                       
    jdbc_connection_string => "jdbc:postgresql://db_server:5432/"                                                                                                              
    jdbc_driver_class => "org.postgresql.Driver"                                                                                                                               
    jdbc_driver_library => "/home/elastic/postgresql-42.2.9.jar"                                                                                                               
    jdbc_user => "postgres"                                                                                                                                                    
    jdbc_password => "password"                                                                                                                                                
    statement => "SELECT * from blogs"                                                                                                                                         
  }                                                                                                                                                                            
}                                                                                                                                                                              
                                                                                                                                                                               
filter {                                                                                                                                                                       
  mutate {                                                                                                                                                                     
    remove_field => ["@version", "host", "message", "@timestamp", "id", "tags"]                                                                                                
  }                                                                                                                                                                            
}                                                                                                                                                                              
                                                                                                                                                                               
output {                                                                                                                                                                       
  stdout { codec => "dots"}                                                                                                                                                    
  elasticsearch {                                                                                                                                                              
    index => "blogs"                                                                                                                                                           
    document_type => "_doc"                                                                                                                                                    
  }                                                                                                                                                                            
}
4

That looks normal... let me ask around.

5

Hi Lindsay,

I have some questions to verify what you're trying to do:

  • Are you doing the labs using the Linux VM that's provided through the Strigo virtual lab environment (as opposed to trying to run Logstash locally on your own system)?
  • Have you SSHd into server1 before trying to run Logstash?

If the answer is "yes" to both questions, then there seems to be a problem with the PostgreSQL database that should automatically be running. With a bit of luck, the database will automatically start when you pause and resume the lab. Just retrying to start Logstash now may work.

If not, one thing we can try to do is replace your VM with a new one. This means you would have to start from scratch (lab 1.1) again. Would that be OK with you?

6

Hi Abdon,

Yes, I am doing the labs via the provided Strigo VM and have SSH'd onto server1 to run the command mentioned in the original post.

Unfortunately, I have tried to pause and resume my VM a few times since I first encountered this issue to no avail.

As I am only on Lab 1.4, I do not mind starting from the beginning if it might fix the issue.

7

Alright, I have replaced your VM with a new one. I hope that resolves the issue. Please let us know if it doesn't!

8

Hi Abdon and Mark,

I just went through the labs and replacing the VM fixed the issue as I am now able to successfully run ./logstash-7.3.1/bin/logstash -f datasets/blogs_sql.conf.

Thank you so much for your help.

1 Like
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.