Setting up logstash config file question

A couple items here:

This is a good starting point for setting up the Elastic Stack:
https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html

For the logstash.yml, here's a list of the settings available:
https://www.elastic.co/guide/en/logstash/6.2/logstash-settings-file.html

I prefer to set the following on mine:
node.name
config.reload.automatic (This and the following allow you to make changes to the pipelines configs without restarting the service)
config.reload.interval
http.host
http.port
log.level (This is especially useful when troubleshooting issues)

In addition, if you'd like to use the disk to buffer events prior to processing, useful if you have scenarios in which your input data may outpace the speed that the data can be processed and need more space than RAM can provide, set the following minimum:
queue.type
path.queue
queue.max_bytes

For the error message you posted, there's usually a helpful bit on the first couple lines of the message. In this case, we see:

[2018-03-09T08:44:16,091][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, 
		:exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, } at line 3, column 61

So on the first line we see it failed to execute an action related to pipeline creation. Since you didn't assign an ID to your pipeline (not necessary unless you have multiple pipelines), it's tagged as 'main'. On the second line it tells you, on line 3, column 61 it expected to see one of three symbols; #, {, or } but they weren't present. So your problem is that you've got the wrong syntax on line 3, column (aka character) 61.

To make reading and creating your pipeline easier, I recommend using a proper editor like Notepad++.

1 Like