# Logstash Pipeline **URL:** \ **Category:** Logstash\ **Created:** [March 4, 2025, 7:54pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396 "2025-03-04T19:54:17Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 4, 2025, 7:54pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/1 "2025-03-04T19:54:17Z")
I have quite a few logstash conf files written out that have to be kicked off manually at the moment. Is there a way to cause them to run automatically? I need to basically tie what I bring in to a dashboard that constantly updates without me having to go in the back end and manually run it. Thank you ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 4, 2025, 9:11pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/2 "2025-03-04T21:11:35Z")
> [@JosephR](#): > > I have quite a few logstash conf files written out that have to be kicked off manually at the moment. Is there a way to cause them to run automatically? Can you provide more context about this? How are you running Logstash? You can Logstash as a service, I would say that this is the recommended way to run it. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 12:09pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/3 "2025-03-05T12:09:14Z")
To run them right now I am going to /logstash/bin and running the logstash -f path/to.conf --path.settings /etc/elogstash to make it run. I have logstash running as a service and it has a schedule inside of the conf file but it does not start on its own. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 12:51pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/4 "2025-03-05T12:51:45Z")
Please provide context, you need to share your configuration, it is impossible to provide any help without it. If you have a plugin that has a schedule, Logstash should executer it on the defined schedule, you just need to service to be running. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 1:26pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/6 "2025-03-05T13:26:46Z")
> [@JosephR](#): > > `schedule => "*/2 * * * *"` It is configured to run the input every 2 minutes, what is the issue here? Is the service running? You said that you run it both manually and as a service, this can conflict with each other, so you should let the service running. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 1:28pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/7 "2025-03-05T13:28:13Z")
I have to run the configuration manually as it is not recognizing the schedule inside of the conf file. logstash is running as a service. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 1:39pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/8 "2025-03-05T13:39:16Z")
> [@JosephR](#): > > I have to run the configuration manually as it is not recognizing the schedule inside of the conf file. Do you have any log from Logstash that confirms that? When you run it manually you run using which user? It is confusing when you say that Logstash is running as a service, but you also run it manually, how this works? Do you stop the service? If logstash is correctly running as a service and you try to run it manually, it will not run and says that there is an instance already running. Can you stop the service and start it again to get fresh logs and share them? ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 1:43pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/9 "2025-03-05T13:43:50Z")
I dont stop the service I run the command i put in the original post to force it to run. If I let it run on its own it gets an error code when running sudo journalctl -u logstash -f nd the reset counter just increments. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 1:44pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/10 "2025-03-05T13:44:32Z")
When I run it manually I run as root ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 1:49pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/11 "2025-03-05T13:49:31Z")
> [@JosephR](#): > > When I run it manually I run as root This may cause a lot of issues when you try to run it as a service as Logstash creates some files that would now be owned by the root user and not the `logstash` user. > [@JosephR](#): > > If I let it run on its own it gets an error code when running sudo journalctl -u logstash -f From your description I suspect that your Logstash service is not even running, probably because you run it as a root in the past and now the service cannot start. Please, try to stop and start the service and share the logs you get. I think that the log file and data path may now be owned by the root user and the service cannot start because the logstash user does not have permissions, you would need to fix those permissions. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 1:54pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/12 "2025-03-05T13:54:44Z")
All my conf files are set as logstash:logstash with 640 on them What else would I need to check? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 1:56pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/13 "2025-03-05T13:56:49Z")
> [@JosephR](#): > > What else would I need to check? Probably the log file on `/var/log/logstash/` and some data files in `/var/lib/logstash`. This is described in the error that the service generates, can you restart the service to get fresh logs as asked? You will need to check on `/var/log/messages` or `/var/log/syslog` for the issue that is making the service to not start. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 2:03pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/14 "2025-03-05T14:03:26Z")
Thge main error I am seeing at the moment is [FATAL] 2025-03-05 08:03:00.440 [main] Logstash - Logstash stopped processing because of an error: (LoadError) no such file to load -- logstash/outputs/elasticsearch ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 2:12pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/15 "2025-03-05T14:12:21Z")
This logs does not make any sense, is this the only log? Something is not right Did you edit the logstash service? Share the **full** content of the following files: `/etc/logstash/logstash.yml` and `/etc/logstash/pipelines.yml`. Also, you need to share all logs or it is pretty complicated to troubleshoot with partial information. Please run `systemctl stop logstash` than `systemctl start logstash` and check the `/var/log/messages` or `/var/log/syslog` for **every** mention to logstash and share them. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 5:00pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/17 "2025-03-05T17:00:16Z")
The `pipelines.yml` looks ok, but none of those pipelines are running if the Logstash service is not running. It is not clear what caused the error, you need to share more log lines as asked But as mentioned, running Logstash as `root` can mess up the permissions and make it not possible to run as a service without fixing it. I recommend that you check the permissions for `/var/lib/logstash`, where everything needs to be owned by the `logstash` user and the `logstash` group and also `/var/log/logstash`, where the path needs to owned by the `logstash` user and the files inside as well. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 5:08pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/18 "2025-03-05T17:08:16Z")
After doing what you said in /var/lib/logstash I have seemed to get past every error of it trying to run on its own besides this one. Logstash stopped processing because of an error: (LoadError) Could not load FFI Provider: (NotImplementedError) FFI not available: null ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 5:39pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/19 "2025-03-05T17:39:39Z")
> [@JosephR](#): > > Logstash stopped processing because of an error: (LoadError) Could not load FFI Provider: (NotImplementedError) FFI not available: null You need to share the full log, the lines that came before and after, with partial lines is not possible to troubleshoot. Please, when sharing log lines share the full log error. ---
**Author:** ![JosephR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephr/32/141754_2.png) [@JosephR](https://discuss.elastic.co/u/JosephR)\ **Post date:** [March 5, 2025, 5:43pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/20 "2025-03-05T17:43:05Z")
I am unable to share my full logs so I will have to find an alternative. Thank you ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 5, 2025, 5:52pm UTC](https://discuss.elastic.co/t/logstash-pipeline/375396/21 "2025-03-05T17:52:19Z")
This may be again related to permissions, in this case permissions for the temp dir. Check this part of the [documentation](https://www.elastic.co/guide/en/logstash/current/ts-logstash.html#ts-temp-dir).