# Directions for installing gssapi to use logstash webhdfs plugin with kerberos

**URL:** <https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049>\
**Category:** Logstash\
**Created:** [June 10, 2019, 8:28pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049 "2019-06-10T20:28:30Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![preddy75](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@preddy75](https://discuss.elastic.co/u/preddy75)\
**Post date:** [June 10, 2019, 8:28pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049/1 "2019-06-10T20:28:30Z")

</div>

Hi,

I am running logstash 6.4.2 (no modifications). I am getting gssapi not found in my scenario below. I am research now how to install gssapi. I would appreciate any information at all on doing this. I have found the ruby gem online for gssapi, but I already see one dependency (ffi) that I do not see in logstash, logstash has ffi-java gem. webhdfs plugin is logstash-output-webhdfs (3.0.6). Please share any information.

I am using webhdfs plugin for logstash and need to use kerberos for auth. After I got everything setup in logstash I get this error on startup: My logstash configuration is after.

[ERROR] 2019-06-10 20:07:00.873 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) no such file to load -- gssapi

I see in the documentation page for the version of logstash I am using that gssapi needs to be installed -\> [https://www.elastic.co/guide/en/logstash/6.4/plugins-outputs-webhdfs.html#plugins-outputs-webhdfs-kerberos\_keytab](https://www.elastic.co/guide/en/logstash/6.4/plugins-outputs-webhdfs.html#plugins-outputs-webhdfs-kerberos_keytab)

However, I see a bunch of threads on the web where people install it all sorts of ways and even after they install it, things do not work because some sorts of init scripts have not been run, etc.

Is there anywhere that has the proper directions to install gssapi for the logstash installation?

I am using a very vanilla logstash configuration for this:

webhdfs {  
host =\> "edited"  
port =\> 50070  
path =\> "/edited/logstash/dt=%{+YYYY-MM-dd}/logstash-%{+HH}.log"  
user =\> "edited"  
use\_kerberos\_auth =\> true  
kerberos\_keytab =\> "/edited/ca/edited.keytab"  
}

I am also using a docker container for logstash and cannot deviate from that in this environment if that makes a difference in configuration steps for gssapi.

---

<div class="post-metadata">

**Author:** ![preddy75](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@preddy75](https://discuss.elastic.co/u/preddy75)\
**Post date:** [June 13, 2019, 4:10pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049/2 "2019-06-13T16:10:05Z")

</div>

Here is an update -\> I have the following modifications (yes its still messy, but you will get the idea) to get me to the point where I get another known error:

Dockerfile:

FROM [docker.elastic.co/logstash/logstash:6.4.2](http://docker.elastic.co/logstash/logstash:6.4.2)

ARG GIT\_SHA  
ENV GIT\_SHA=${GIT\_SHA}

USER 0

WORKDIR /usr/share/logstash  
ENV PATH=/usr/share/logstash/bin:$PATH

# Ensure Logstash has a UTF-8 locale available.

# RUN locale-gen en\_US.UTF-8

# ENV LANG='en\_US.UTF-8' LC\_ALL='en\_US.UTF-8'

ENV LANG=en\_US.UTF-8   
LANGUAGE=en\_US.UTF-8   
LC\_CTYPE=en\_US.UTF-8   
LC\_ALL=en\_US.UTF-8

COPY gssapi-1.3.0.gem gssapi-1.3.0.gem

COPY ffi-1.11.1.gem ffi-1.11.1.gem

RUN cat /etc/resolv.conf &&   
yum clean all &&  
yum update -y && yum install sudo gcc make -y &&   
yum clean all &&  
curl [https://curl.haxx.se/download/curl-7.61.0.tar.gz](https://curl.haxx.se/download/curl-7.61.0.tar.gz) -o curl-7.61.0.tar.gz &&   
tar xvzf curl-7.61.0.tar.gz && cd curl-7.61.0 &&   
export GEM\_HOME=/usr/share/logstash/vendor/bundle/jruby/2.3.0 &&   
/usr/share/logstash/vendor/jruby/bin/jruby /usr/share/logstash/vendor/jruby/bin/gem install -V /usr/share/logstash/ffi-1.11.1.gem &&   
/usr/share/logstash/vendor/jruby/bin/jruby /usr/share/logstash/vendor/jruby/bin/gem install -V /usr/share/logstash/gssapi-1.3.0.gem

RUN cat Gemfile \> Gemfile2

RUN printf 'gem "ffi"\ngem "gssapi"' \>\> Gemfile2

RUN cp Gemfile2 Gemfile

# ./configure --with-darwinssl && make && \

# echo "Running make install! " && make install

RUN echo "logstash ALL=(ALL) NOPASSWD:ALL" \>\> /etc/sudoers

USER logstash

* * *

And now when I run the webhdfs plugin I get the same error as this [post](https://discuss.elastic.co/t/logstash-webhdfs-example-with-kerberos-enabled-hadoop-cluster/51941) (that has an _interesting_ solution that I am trying now. If this does not work I have to punt, and would advise logstash staff to include this thread in their docs (in some different format), because this is not working with defined gssapi directions.

---

<div class="post-metadata">

**Author:** ![preddy75](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@preddy75](https://discuss.elastic.co/u/preddy75)\
**Post date:** [June 13, 2019, 4:30pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049/3 "2019-06-13T16:30:19Z")

</div>

And just to reiterate: current stack trace:

```
[INFO] 2019-06-13 16:21:29.978 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.4.2"}

```

[INFO] 2019-06-13 16:21:37.394 [Converge PipelineAction::Create] pipeline - Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>112, "pipeline.batch.size"=\>500, "pipeline.batch.delay"=\>50}  
[ERROR] 2019-06-13 16:21:38.105 [[main]-pipeline-manager] webhdfs - Webhdfs check request failed. (namenode: , Exception: undefined method `read_uint32' for #<FFI::MemoryPointer address=0x7fee6404c7a0 size=4> Did you mean? read_uint read_int read_array_of_uint32 read_array_of_int32 read_pointer read_ulong read_string read_ushort read_array_of_uint64 read_array_of_uint16 get_uint32) [ERROR] 2019-06-13 16:21:38.110 [[main]-pipeline-manager] pipeline - Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::OutputDelegator:0x725df834>", :error=>"undefined method`read\_uint32' for #\<FFI::MemoryPointer address=0x7fee6404c7a0 size=4\>\nDid you mean? read\_uint\n read\_int\n read\_array\_of\_uint32\n read\_array\_of\_int32\n read\_pointer\n read\_ulong\n read\_string\n read\_ushort\n read\_array\_of\_uint64\n read\_array\_of\_uint16\n get\_uint32", :thread=\>"#\<Thread:0x7f10c5d4 run\>"}  
[ERROR] 2019-06-13 16:21:38.114 [[main]-pipeline-manager] pipeline - Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\<WebHDFS::KerberosError: undefined method `read_uint32' for #<FFI::MemoryPointer address=0x7fee6404c7a0 size=4> Did you mean? read_uint read_int read_array_of_uint32 read_array_of_int32 read_pointer read_ulong read_string read_ushort read_array_of_uint64 read_array_of_uint16 get_uint32>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:323:in`request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client\_v1.rb:275:in `operate_requests'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:138:in`list'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs\_helper.rb:49:in `test_client'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs.rb:155:in`register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:102:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:46:in`register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in`block in register\_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in`register\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in`start\_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in`block in start'"], :thread=\>"#\<Thread:0x7f10c5d4 run\>"}  
[ERROR] 2019-06-13 16:21:38.127 [Converge PipelineAction::Create] agent - Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create, action\_result: false", :backtrace=\>nil}  
[INFO] 2019-06-13 16:21:38.402 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600}

and the current suggested fix -\> [https://github.com/gpaggi/gssapi/commit/69b14cce1966fdae713d274ca19339237eca72b9](https://github.com/gpaggi/gssapi/commit/69b14cce1966fdae713d274ca19339237eca72b9)

which is what I am looking into now

---

<div class="post-metadata">

**Author:** ![preddy75](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@preddy75](https://discuss.elastic.co/u/preddy75)\
**Post date:** [June 13, 2019, 4:41pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049/4 "2019-06-13T16:41:46Z")

</div>

And inside currently built image:

bash-4.2$ grep -i -r "read\_uint32" .  
./vendor/bundle/jruby/2.3.0/gems/gssapi-1.3.0/lib/gssapi/simple.rb: if (ret\_flags.read\_uint32 & LibGSSAPI::GSS\_C\_DELEG\_FLAG) != 0  
./vendor/bundle/jruby/2.3.0/gems/gssapi-1.3.0/lib/gssapi/exceptions.rb: min\_stat = min\_stat.read\_uint32  
./vendor/bundle/jruby/2.3.0/gems/gssapi-1.3.0/lib/gssapi/exceptions.rb: @s += "(with major status #{maj}, minor status #{min.read\_uint32}\n"  
bash-4.2$ pwd  
/usr/share/logstash

So going to put this into docker image build (in some form and will retry):

```auto
grep -rl read_uint32 /usr/share/logstash | xargs sed -i 's/read_uint32/read_uint/g'

```

---

<div class="post-metadata">

**Author:** ![preddy75](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@preddy75](https://discuss.elastic.co/u/preddy75)\
**Post date:** [June 13, 2019, 6:34pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049/5 "2019-06-13T18:34:50Z")

</div>

So that fix above got past that problem and hit a new one -\>

[INFO] 2019-06-13 18:31:37.995 [Converge PipelineAction::Create] pipeline - Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>112, "pipeline.batch.size"=\>500, "pipeline.batch.delay"=\>50}  
[ERROR] 2019-06-13 18:31:38.581 [[main]-pipeline-manager] webhdfs - Webhdfs check request failed. (namenode: , Exception: gss\_init\_sec\_context did not return GSS\_S\_COMPLETE: Unspecified GSS failure. Minor code may provide more information  
No Kerberos credentials available (default cache: KEYRING:persistent:1000)  
)  
[ERROR] 2019-06-13 18:31:38.585 [[main]-pipeline-manager] pipeline - Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#LogStash::OutputDelegator:0x10f3889c", :error=\>"gss\_init\_sec\_context did not return GSS\_S\_COMPLETE: Unspecified GSS failure. Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:1000)\n", :thread=\>"#\<Thread:0x1cf29a7d run\>"}  
[ERROR] 2019-06-13 18:31:38.588 [[main]-pipeline-manager] pipeline - Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\<WebHDFS::KerberosError: gss\_init\_sec\_context did not return GSS\_S\_COMPLETE: Unspecified GSS failure. Minor code may provide more information  
No Kerberos credentials available (default cache: KEYRING:persistent:1000)

> , :backtrace=\>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client\_v1.rb:323:in `request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:275:in `operate\_requests'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client\_v1.rb:138:in `list'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs_helper.rb:49:in `test\_client'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs.rb:155:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:102:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:46:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:in `register\_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:in `maybe\_setup\_out\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=\>"#\<Thread:0x1cf29a7d run\>"}  
> [ERROR] 2019-06-13 18:31:38.595 [Converge PipelineAction::Create] agent - Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create, action\_result: false", :backtrace=\>nil}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 11, 2019, 6:34pm UTC](https://discuss.elastic.co/t/directions-for-installing-gssapi-to-use-logstash-webhdfs-plugin-with-kerberos/185049/6 "2019-07-11T18:34:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.