How to define cloud id in logstash config file?

Elastic Stack Logstash
1

What I am trying to accomplish is execute sql query against our mssql database, pass that data to ES stack in cloud.

To my understanding I have to take following actions,
install jdk on windows client where i ll be sending data from,
install logstash on windows client where i ll be sending data from,
install logstash-input-jdbc on windows client where i ll be sending data from,
change logstash.conf to add cloud id and user credentials,
create data-out.conf file like below,

input {
  jdbc {
jdbc_driver_library => "sqljdbc4.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://192.168.2.126\\SQLEXPRESS2014:1433;databaseName=test
	jdbc_password => "sa@sa2015"
schedule => "0 0-59 0-23 * * *"
statement => "SELECT ID , Name, City, State,ShopName FROM dbo.Shops"
	jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
  }
}
filter {
}
output {
  stdout { codec => rubydebug }
elasticsearch { 
    protocol => "http"
		index => "shops"
		document_id => "%{id}"
}
}

create index on ES Kibana UI.

What confuses me is "elasticsearch { protocol => "http"" section. Will this "data-out.conf" find the cloud ES instance by checking against main "logstash.conf"?

2

Will this "data-out.conf" find the cloud ES instance by checking against main "logstash.conf"?

I assume you mean .yml, not .conf ? (if so...)

The Elasticsearch settings in logstash.yml are only applicable to monitoring, management, and modules. You still have to define your hosts (and credentials if needed) in the output { section of your configuration [1].

If have 1 instance/cluster of Elasticsearch , it does become a bit redundant to redefine the same connection everywhere. Also note, the output { section does not currently allow for a cloud_id , you will need to use the host name(s) directly.

[1]

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.