Dashboard importing fails for metricbeat when security (x-pack) is configured


(Yu Watanabe) #1

Hi !

I would like to ask question regarding to sample dashboard importing for metricbeat.

My environment is

elasticsearch 5.0.0-alpha5
Xpack Security configured
Metricbeat 5.0.0-alpha5
Kibana 5.0.0-alpha5

Problem
When I execute the shell import-dashboard.sh it returns nothing for the dashboard import. index-pattern import is not executed after this.

[root@BS-POC-Test-Env kibana]# ./import_dashboards.sh
Import dashboards,visualizations, searches and index pattern from . to http://localhost:9200 in .kibana
Import dashboard Metricbeat-Apache-HTTPD-server-status:
curl: (52) Empty reply from server

search and visualization was successful before I have configured x-pack security.

Analysis
I have changed the curl command in shell script as below.

if [ -d "${DIR}/dashboard" ]; then
    for file in ${DIR}/dashboard/*.json
    do
        NAME=`basename ${file} .json`
        echo "Import dashboard ${NAME}:"
        ${CURL} -u elastic:elastic -XPUT ${ELASTICSEARCH}/${KIBANA_INDEX}/dashboard/${NAME} -d @${file} || exit 1
        echo
    done
fi

if [ -d "${DIR}/index-pattern" ]; then
    for file in ${DIR}/index-pattern/*.json
    do
        NAME=`awk '$1 == "\"title\":" {gsub(/[",]/, "", $2); print $2}' ${file}`
        echo "Import index pattern ${NAME}:"

        ${CURL} -XPUT ${ELASTICSEARCH}/${KIBANA_INDEX}/index-pattern/${NAME} \
            -d -u elastic:elastic @${file} || exit 1
        echo
    done
fi

Am I missing any setting to use the import shell?


(Yu Watanabe) #2

Also I get below error in elasticsearch.

I get below error in elasticsearch log after the shell is executed.

[2016-09-12 02:05:35,892][WARN ][xpack.security.transport.netty4] [kG9P1-s] caught exception while handling client http traffic, closing connection [id: 0xd8a162b1, L:/0:0:0:0:0:0:0:1:9200 - R:/0:0:0:0:0:0:0:1:34456]
java.lang.UnsupportedOperationException: unsupported message type: DefaultFullHttpResponse (expected: ByteBuf, FileRegion)
        at io.netty.channel.nio.AbstractNioByteChannel.filterOutboundMessage(AbstractNioByteChannel.java:260)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:799)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1291)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:748)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:811)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:824)
        at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:804)
        at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:841)
        at io.netty.handler.codec.MessageAggregator.decode(MessageAggregator.java:222)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:571)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:474)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:428)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:398)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:877)
        at java.lang.Thread.run(Thread.java:745)

(Mark Walkom) #3

Hmm, looks like this was fixed in the latest alpha - https://github.com/elastic/elasticsearch/issues/19893

Maybe one of the beats team can comment a bit more, but it might be an issue with x-pack.


(Tudor Golubenco) #4

@YuWatanabe That Elasticsearch issue exists in alpha5, so you need to edit the script and make sure you pass -H Expect: to the curl commands. See this comment here.


(Yu Watanabe) #5

@tudor

Thanks for the help.

Import works perfectly now.

I believe with xpack user option needs to be specified. Default user option replaces the curl command defined in the beginning . I have modified the curl command specified in the user option as below.

 69     -u | -user )
 70         USER=$2
 71         if [ "$USER" = "" ]; then
 72             echo "Error: Missing username"
 73             print_usage
 74             exit 1
 75         fi
 76         CURL="${CURL} --user ${USER}"
 77         ;;

(system) #6

This topic was automatically closed after 21 days. New replies are no longer allowed.