Elastic Lab 1.4

Elastic Stack Logstash
#1

Hello,

I am working through the elastic lab 1.4. I am not using virtual lab, I am running mine locally.

For some reason I did not have the blogs_sql.conf when I installed Elastic. So, I found an example on the web, created the file, and put it into the datasets directory.

Now I am trying to run logstash with the following command from the lab manual

./logstash-7.8.0/bin/logstash -f datasets/blogs_sql.conf

I get a reflector error like this

An illegal reflective access operation has occurred

Also, I get a lot of output, but most errors are associated with permissions like the one below.

Error: unable to load /home/troy/elastic/postgresql-42.2.9.jar from :jdbc_driver_library, file not readable (please check user and group permissions for the path)
Exception: LogStash::PluginLoadingError

I have done a chmod on all the files, and the permissions should be fine. Please let me know if you have any ideas as to what the problem is.

#2

And yet logstash says they are not. I would suggest using strace (or the equivalent on your flavour of UNIX) to trace all the system calls. Write all that to a file and after the error grep for postgresql-42 in it.

You can ignore the illegal reflective access message. At some point whoever owns that code will get around to fixing it and a new version of logstash will pick up the new code.

#3

Looks like I dont even have the tables of blog data for the postgres database.

Pipeline_id:main
Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"postgres",
jdbc_password=>, statement=>"SELECT * from blogs",
jdbc_driver_library=>"/home/troy/elastic/postgresql-42.2.19.jar",
jdbc_connection_string=>"jdbc:postgresql://db_server:5432/",
id=>"1d9770372769750dbc0e037d2dd104b39ce6859f3cd417749585bd72b460cdee",
jdbc_driver_class=>"org.postgresql.Driver", enable_metric=>true,
codec=><LogStash::Codecs::Plain
id=>"plain_f3475974-8694-41d6-b097-40540228cac4", enable_metric=>true,
charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000,
jdbc_validate_connection=>false, jdbc_validation_timeout=>3600,
jdbc_pool_timeout=>5, sql_log_level=>"info",
connection_retry_attempts=>1,
connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc",
last_run_metadata_path=>"/home/troy/.logstash_jdbc_last_run",
use_column_value=>false, tracking_column_type=>"numeric",
clean_run=>false, record_last_run=>true, lowercase_column_names=>true,
use_prepared_statements=>false>

The part of the lab is on page 13 of the attached document.

I am trying to recreate the lab locally.

Thanks,

(Attachment Lab Guide Elasticsearch Engineer I(1).pdf is missing)

#4

Actually, looks like I fixed it. The ingest for the local machine is
a CSV file, which I found. Thanks.