Pipelines.yml ignored on logstash

This could mean that your old pipelines.yml had some formatting error, which is one thing that could give you the error you were getting before.

I would say that you should avoid running logstash as root even for testing, when logstash runs it creates some temporary files and paths depending on your pipeline, if you run it as a root, it will create them as the root user, which could give you permissions errors when you run as the logstash user.