# Unable to start Logstash via systemd on RHEL - can't find Elapsed Plugin

**URL:** <https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828>\
**Category:** Logstash\
**Created:** [January 10, 2018, 10:13am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828 "2018-01-10T10:13:56Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 10, 2018, 10:13am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/1 "2018-01-10T10:13:56Z")

</div>

Hi ELK-bods,

I'm going through my first install of ELK on Linux/RHEL and am not able to run Logstash via systemd as it seems it is not able to find/read the 'logstash-input-elapsed' plugin.

This plugin has been installed correctly and shows up in the plugin list:

```
[root@gbdocldibgdap02 6.1.0]# /usr/share/logstash/bin/logstash-plugin list elapsed
logstash-filter-elapsed
[root@gbdocldibgdap02 6.1.0]#

```

Additionally, when I run logstash directly from the command line, it works perfectly - including using the 'elapsed' plugin:

```
[2018-01-10T09:40:31,436][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.1.0"}
[2018-01-10T09:40:31,552][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-01-10T09:40:32,237][INFO][logstash.filters.elapsed] Elapsed, timeout: 60 seconds
[2018-01-10T09:40:32,239][INFO][logstash.pipeline] Starting pipeline {:pipeline_id=>"uat-esb-audit-pipeline", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125, :thread=>"#<Thread:0x348fd941@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
[2018-01-10T09:40:32,297][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=>"uat-esb-audit-pipeline"}
[2018-01-10T09:40:32,313][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["uat-esb-audit-pipeline"]}
[2018-01-10T09:40:32,644][INFO][logstash.inputs.jdbc] (0.035720s) SELECT [rowID_PK] as primary_key
                                ,[createTimestamp] as db_created
                                ,[eventTimestamp] as event_created
                        FROM [ESB_PREPROD].[esb].[esb_audit]
                        WHERE [ESB_PREPROD].[ESB].[ESB_AUDIT].[rowID_PK] = 2130648
[2018-01-10T09:40:32,692][INFO][logstash.filters.elapsed] Elapsed, 'end event' received {:end_tag=>"END", :unique_id_field=>"unique_composite_id"}
[2018-01-10T09:40:32,832][INFO][logstash.pipeline] Pipeline terminated {"pipeline.id"=>"uat-esb-audit-pipeline"}

```

However, when I try to start logstash via systemd:  
`systemctl start logstash`

I get errors which seem to point to it not being able to find/read/access the 'elapsed' plugin:

```
[2018-01-10T09:41:50,655][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-01-10T09:41:50,657][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-01-10T09:41:50,909][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.1.0"}
[2018-01-10T09:41:50,969][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-01-10T09:41:51,646][ERROR][logstash.plugins.registry] Tried to load a plugin's code, but failed. {:exception=>#<LoadError: no such file to load -- logstash/filters/elapsed>, :path=>"logstash/filters/elapsed", :type=>"filter", :name=>"elapsed"}
[2018-01-10T09:41:51,656][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:uat-esb-audit-pipeline, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any filter plugin named 'elapsed'. Are you sure this is correct? Trying to load the elapsed filter plugin resulted in this error: no such file to load -- logstash/filters/elapsed", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:192:in `lookup_pipeline_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:140:in `lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/plugin_factory.rb:82:in `plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:114:in `plugin'", "(eval):64:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:86:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:171:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:335:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:332:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:319:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

```

It may be worth pointing out that the plugin was installed as an offline pack (my machines don't currently have access to the external sites) and I installed as 'root'.

I did notice that the plugin gemspec file was installed as 'root' whereas everything else within /usr/share/logstash/vendor/bundle/jruby/2.3.0/specifications was owned by 'logstash':

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/5/a5f67a3dc6836b3c6c00025cbe3b88958f31f3f7.png)

I had to chown/chmod this file to 'logstash' user and change the permissions:

```
[root@gbdocldibgdap02 6.1.0]# chown logstash:logstash /usr/share/logstash/vendor/bundle/jruby/2.3.0/specifications/logstash-filter-elapsed-4.0.3.gemspec
[root@gbdocldibgdap02 6.1.0]# chmod 664 /usr/share/logstash/vendor/bundle/jruby/2.3.0/specifications/logstash-filter-elapsed-4.0.3.gemspec

```

As without the chown/chmod starting via systemd didn't even get to the stage of running - instead I saw this line within /var/log/messages:

`Jan 10 10:05:36 gbdocldibgdap02 logstash: Errno::EACCES: Permission denied - /usr/share/logstash/vendor/bundle/jruby/2.3.0/specifications/logstash-filter-elapsed-4.0.3.gemspec`

Can someone advise:

1. When installing plugins, is it a requirement to change the ownership/permissions of the installed plugin after installation? I would have thought the logstash-plugin executable would have handled this?

2. What might be the problem which is stopping logstash from accessing the 'elapsed' plugin when its started via systemd.

All help appreciated as I'm a little stuck at this point 😕

Cheers,  
Steve

---

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 11, 2018, 8:24am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/2 "2018-01-11T08:24:29Z")

</div>

Hi Chaps,

Can anyone out there shed any light on this - I'm trying to set up a PoC and this is holding us up at the moment.

Cheers,  
Steve

---

<div class="post-metadata">

**Author:** ![alpha\_gamma](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@alpha\_gamma](https://discuss.elastic.co/u/alpha_gamma)\
**Post date:** [January 11, 2018, 9:45am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/3 "2018-01-11T09:45:08Z")

</div>

Hi stevedearl

have u tried

initctl start logstash  
initctl status logstash  
initctl stop logstash

I have n't gone through whole blog, but hope it works

---

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 11, 2018, 3:18pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/4 "2018-01-11T15:18:17Z")

</div>

Hi alpha\_gamma,

Thanks for the comment but in my case our RHEL box is using systemd (not upstart) so I'm starting/stopping Logstash with:

```
systemctl start logstash
systemctl stop logstash

```

This is as per the Logstash documentation.

Cheers,  
Steve

---

<div class="post-metadata">

**Author:** ![alpha\_gamma](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@alpha\_gamma](https://discuss.elastic.co/u/alpha_gamma)\
**Post date:** [January 11, 2018, 3:55pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/5 "2018-01-11T15:55:28Z")

</div>

Have u tried Sudo -u with the commands,pls check

---

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 12, 2018, 12:24pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/6 "2018-01-12T12:24:17Z")

</div>

Hi alpha\_gamma,

Thanks for the update. Can you provide any additional detail on what you mean?

I'm running the Logstash start/stop scripts as root already (should have mentioned that perhaps, sorry).

From the ELK v6.1 documentation around the RPM installation and configuration process I don't see anything on having to run systemctl with any sudo for any other account details.

If I run Logstash directly it finds and uses the Elapsed plugin without any problems at all:

`sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash`

Additionally, it seems odd that I had to change the ownership of the installed plugin gemspec file.

Regards,  
Steve

---

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 16, 2018, 3:20pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/7 "2018-01-16T15:20:24Z")

</div>

Hi All,

Just another call to see if anyone can help at all with this. Even something in terms of how to switch on more logging in Logstash (when running under systemd) or an answer to my question around plugin/gemspec file ownership would be helpful.

As I say, we're a bit stuck at the moment.

cheers,  
Steve

---

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 18, 2018, 9:06am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/8 "2018-01-18T09:06:43Z")

</div>

Hi All,

I was working my way through the Logstash Ruby code which is called out within the Stacktrace I'm seeing:

```
[2018-01-18T08:32:34,703][ERROR][logstash.plugins.registry] Tried to load a plugin's code, but failed. {:exception=>#<LoadError: no such file to load -- logstash/filters/elapsed>, :path=>"logstash/filters/elapsed", :type=>"filter", :name=>"elapsed"}
[2018-01-18T08:32:34,707][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:uat-esb-audit-pipeline, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any filter plugin named 'elapsed'. Are you sure this is correct? Trying to load the elapsed filter plugin resulted in this error: no such file to load -- logstash/filters/elapsed", :backtrace=>
["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:192:in `lookup_pipeline_plugin'", 
"/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:140:in `lookup'", 
"/usr/share/logstash/logstash-core/lib/logstash/plugins/plugin_factory.rb:82:in `plugin'", 
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:114:in `plugin'", "(eval):64:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", 
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:86:in `initialize'", 
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:171:in `initialize'", 
"/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:335:in `block in converge_state'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:332:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:319:in `converge_state'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", 
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", 
"/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", 
"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

```

and noticed that the section which is called out in `/usr/share/logstash/logstash-core/lib/logstash/plugin.rb` appears to be for backwards compatibility from 6.0 to 5.x:

```
  # This is keep for backward compatibility, the logic was moved into the registry class
  # but some plugins use this method to return a specific instance on lookup
  #
  # Should I remove this now and make sure the pipeline invoke the Registry or I should wait for 6.0
  # Its not really part of the public api but its used by the tests a lot to mock the plugins.
  def self.lookup(type, name)
    LogStash::PLUGIN_REGISTRY.lookup_pipeline_plugin(type, name)
  end

```

Could this be causing an issue perhaps (I appreciate I'm clutching at straws a bit here)

Can anyone confirm if they've been able to run the Elapsed plugin successfully using systemd on RHEL6. I'm using the ELK 6.1.0 binaries installed via RPM.

Cheers,  
Steve

---

<div class="post-metadata">

**Author:** ![stevedearl](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@stevedearl](https://discuss.elastic.co/u/stevedearl)\
**Post date:** [January 19, 2018, 9:09am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/9 "2018-01-19T09:09:34Z")

</div>

Hi All,

I've managed to resolve this issue by amending the user:group ownership of various files and dirs to 'logstash:logstash'. For info, I rah the following:

```
chown -R logstash:logstash /data/logstash
chown -R logstash:logstash /var/log/logstash
chown -R logstash:logstash /etc/logstash
chown -R logstash:logstash /usr/share/logstash

```

I suspect the above is a sledgehammer approach and I've probably changed the ownership of more files than I'd need to get around the problem - however, it has solved my issue.

Steve

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 16, 2018, 9:09am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828/10 "2018-02-16T09:09:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.