ERROR: Pipelines YAML file must contain an array of pipeline configs. Found "Hash"

Windows server:

I have the following pipeline.yml file

-pipeline.id: policylocalexposurehub
path.config: "config/logstash-sample.conf"
pipeline.workers: 1

error

ERROR: Pipelines YAML file must contain an array of pipeline configs. Found "Hash"

when running logstash.bat

If I ran logstash.bat with

bin\logstash.bat -f config\logstash-sample.conf

This runs fine, data appears in elasticsearch.

What is wrong with the pipelines file? I want to have multiple pipelines here but for now, just starting trying to get just one working!

logstash-sample.conf:

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}

input {  
	jdbc {
		# SqlServer jdbc connection string to your database, productdb
		#  "jdbc:sqlserver://HostName\instanceName;database=DBName;user=UserName;password=Password" 
		jdbc_connection_string => "jdbc:sqlserver://10.130.40.20;database=ApiTeamExposureHub;user=exposure;password=ebix@123"
		# The user you want to execute your statement as
		jdbc_user => nil
		# The path to your downloaded jdbc driver
		# jdbc_driver_library => "C:/Program Files/sqljdbc_6.0/enu/jre8/sqljdbc42.jar"
		jdbc_driver_library => "C:\Program Files\Microsoft JDBC DRIVER 9.2 for SQL Server\sqljdbc_9.2.1.0_enu\sqljdbc_9.2\enu\mssql-jdbc-9.2.1.jre11.jar"
		# The name of the driver class for SqlServer
		jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
		# Query for testing purpose
		statement => "portfolio.usp_Policy_SelectListForElasticsearch"
	}
}

output {
  # stdout { codec => rubydebug } #uncomment to enable debugging
  elasticsearch {
    hosts => ["https://UKEE-EXP004:9200"]
    index => "policylocalexposurehub"
    user => "elastic"
    password => "p7p8qHk9GdU0v0RLUDM9"
	ssl => true
    cacert => 'config\elasticsearch-ca.pem'
	doc_as_upsert => true
	document_id => "%{policyid}"
  }
}

The format and indentation of your pipelines.yml appears to be wrong, you need to have a space after the - and the other configurations needs to follow the same indentation.

It should looks like this:

- pipeline.id: policylocalexposurehub
  path.config: "config/logstash-sample.conf"
  pipeline.workers: 1

Logstash works when you run it using the -f parameter because this will ignore the pipelines.yml.

Thank you.

I had tried different indentations. I have now tried exactly how you have it and it seems to be working with one pipeline defined.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.