Filebeat missing authentication

sahere37 · December 26, 2018, 1:40pm

hi all,
kindly, i installed filebeat, elasticsearch and kibana in my VMware; notably i am not using logstash. After that, i installed X-pack for kibana and elasticsearch,when i start elasticsearch and kibana, kibana console is up and seems ok. But, the point is there is no index in visualize part of kibana console.Actually, i did not set any configuration about x-pack and authentication for filebeat, and just the configurations had been done for elasticsearch and kibana. The filebeat log is as following, i have no idea that how can i set the related configurations for filebeat. Any advise would be so appreciated. thanks

 2018-12-26T16:51:01+03:30 INFO retryer: send wait signal to consumer
 2018-12-26T16:51:01+03:30 INFO   done
2018-12-26T16:51:15+03:30 INFO Non-zero metrics in the last 30s: 
beat.info.uptime.ms=30009 beat.memstats.gc_next=4830496 beat.memstats.memory_alloc=4009736 beat.memstats.memory_total=8764760 filebeat.events.active=1440 filebeat.events.added=1444 filebeat.events.done=4 filebeat.harvester.open_files=2 filebeat.harvester.running=2 filebeat.harvester.started=2 libbeat.config.module.running=0 libbeat.config.reloads=1 libbeat.output.read.bytes=1850 libbeat.output.type=elasticsearch libbeat.output.write.bytes=605 libbeat.pipeline.clients=2 libbeat.pipeline.events.active=1434 libbeat.pipeline.events.filtered=10 libbeat.pipeline.events.published=1434 libbeat.pipeline.events.retry=100 libbeat.pipeline.events.total=1444 registrar.states.current=2 registrar.states.update=4 registrar.writes=4
  2018-12-26T16:51:17+03:30 ERR  Failed to connect: 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
 2018-12-26T16:51:17+03:30 INFO retryer: send unwait-signal to consumer
2018-12-26T16:51:17+03:30 INFO   done
2018-12-26T16:51:17+03:30 INFO retryer: send wait signal to consumer
2018-12-26T16:51:17+03:30 INFO   done

sahere37 · December 27, 2018, 7:42am

I do the steps mentioned in following link but still i can see above error in the logs

https://www.elastic.co/guide/en/beats/filebeat/6.2/beats-basic-auth.html

steffens · December 27, 2018, 2:40pm

Given the log format I assume you are running a quite old filebeat version?

Your error:

2018-12-26T16:51:17+03:30 ERR Failed to connect: 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401}

You need to configure the right user and password in Filebeat. Elasticsearch requires you to authenticate.

sahere37 · December 29, 2018, 5:05am

Dear steffens

how can i do that?
notably, version of filebeat is 6.1.1

sahere37 · December 29, 2018, 8:13am

Dear steffens
in order to create role and user, the steps mentioned in following link has been done as bellow:

   POST _xpack/security/role/beat_writer
  {
    "cluster": ["manage_index_templates", "monitor"],
   "indices": [
{
  "names": [ "filebeat-*", "metricbeat-*", "packetbeat-*" ], 
  "privileges": ["read","write","create_index"]
}
 ]
}

,

     POST /_xpack/security/user/beat_user
     {
      "password" : "changeme",
      "roles" : [ "beat_writer"],
     "full_name" : "Internal Beat User"
       }

filebeat.yml:
output.elasticsearch:
hosts: ["localhost:9200"]
username: "beat_user"
password: "changeme"

but still i cannot see any indices in the "index patterns" part of kibana console (i have been logged in using elastic (admin) user).
also i use following command to check the existence of index which still there is no "filebeat" index

 GET /_cat/indices

could you please advise me about this issue?

Also, i checked the logs of elasticsearch which everything seems ok and there is no error

sahere37 · December 29, 2018, 9:54am

I am looking in elasticsearch log, again, and i can see following message:

 [ERROR] [o.e.x.s.a.e.NativeUsersStore] [9sIG28T] failed to retrieve 
  password hash for reserved user [elastic]

it is so weird, because it was not previously
i can login into kibana console using elastic user

sahere37 · December 29, 2018, 10:01am

rest of error message is as followings

 [2018-12-29T12:24:09,727][INFO ][o.e.g.GatewayService     ] [9sIG28T] recovered [14] indices into cluster_state
  [2018-12-29T12:24:09,706][ERROR][o.e.x.s.a.e.NativeUsersStore] 
[9sIG28T] failed to retrieve built in user [elastic] info
   org.elasticsearch.action.NoShardAvailableActionException: No shard available for [get [.security][doc][reserved-user-elastic]: routing [null]]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.perform(TransportSingleShardAction.java:209) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.start(TransportSingleShardAction.java:186) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:95) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:59) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$0(SecurityActionFilter.java:103) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:188) ~[?:?]
at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:182) ~[?:?]
at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:176) ~[?:?]
at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:164) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.authorizeRequest(SecurityActionFilter.java:190) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:166) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:195) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:228) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:239) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:193) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:147) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:116) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:165) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$2(SecurityActionFilter.java:117) ~[?:?]
at org.elasticsearch.xpack.security.SecurityContext.executeAsUser(SecurityContext.java:105) ~[?:?]

sahere37 · December 29, 2018, 10:02am

      at org.elasticsearch.xpack.security.authz.AuthorizationUtils.switchUserBasedOnActionOriginAndExecute(AuthorizationUtils.java:116) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:115) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:497) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:73) ~[?:?]
at org.elasticsearch.xpack.security.authc.esnative.NativeUsersStore.getReservedUserInfo(NativeUsersStore.java:589) ~[?:?]
at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.getUserInfo(ReservedRealm.java:213) ~[?:?]
at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.doAuthenticate(ReservedRealm.java:98) ~[?:?]
at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.doAuthenticateAndCache(CachingUsernamePasswordRealm.java:170) ~[?:?]
at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticateWithCache(CachingUsernamePasswordRealm.java:109) ~[?:?]
at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:94) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$13(AuthenticationService.java:284) ~[?:?]
at org.elasticsearch.xpack.common.IteratingActionListener.run(IteratingActionListener.java:93) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:320) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$9(AuthenticationService.java:257) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:266) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$null$0(AuthenticationService.java:201) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.1.1.jar:6.1.1]

sahere37 · December 29, 2018, 10:02am

 at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:239) ~[x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:197) ~[x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:228) ~[x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:239) [x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:193) [x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:147) [x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:99) [x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:69) [x-pack-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:497) [transport-netty4-6.1.1.jar:6.1.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:80) [transport-netty4-6.1.1.jar:6.1.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-6.1.1.jar:6.1.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.13.Final.jar:4.1.13.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.13.Final.jar:4.1.13.Final]

sahere37 · December 29, 2018, 10:03am

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

sahere37 · December 29, 2018, 10:03am

  [2018-12-29T12:24:09,790][ERROR][o.e.x.s.a.e.ReservedRealm] [9sIG28T] failed to retrieve password hash for reserved user [elastic]
[2018-12-29T12:24:09,791][INFO ][o.e.x.s.a.AuthenticationService] 
 [9sIG28T] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2018-12-29T12:24:09,852][ERROR][o.e.x.s.a.e.NativeUsersStore] [9sIG28T] failed to retrieve built in user [elastic] info
org.elasticsearch.action.NoShardAvailableActionException: No shard available for [get [.security][doc][reserved-user-elastic]: routing [null]]

sahere37 · December 29, 2018, 10:24am

notably, when i execute (curl -u elastic 'http://localhost:9200/_xpack/security/_authentication?pretty'), the output is as following:

            {
            "username" : "elastic",
           "roles" : [
          "superuser"
          ],
           "full_name" : null,
         "email" : null,
         "metadata" : {
       "_reserved" : true
       },
        "enabled" : true
       }

sahere37 · December 29, 2018, 12:34pm

if it helps, i changed the password of elastic previously using following command:

      /bin/x-pack/setup-passwords interactive

i am new in x-pack, i used elastic user and its password in filebeat.yml (elasticsearch.out part) at first, when it doesn't work i changed it according role and user defined in console.
Also, i used elastic and its password in kibana.yml file, recently i figure out that it is not needed to use elastic user for communication between kibana and elasticsearch; so i change it as kibana user and its password.
i think maybe the elastic user and its password cached in filebeat and kibana, so in order to prevent of confusion due to using elastic user and its password in filebeat.yml and kibana.yml, i change the password of elastic user in kibana console. therefore, now the filebeat and kibana are using different users and passwords and also password of elastic user has been changed too

now, situation is as following:

 1- kibana is up, and kibana user is used in kibana.yml (elasticsearch.username and password part)

2- i can login into kibana console using elastic user and its password

3- elastic user is defined in (curl -u elastic 'http://localhost:9200/_xpack/security/_authentication?pretty') but in the elasticsearch logs, it seems that authentication of elastic has been terminated

4- role and user have been defined in kibana console and the username and password of defined user is used in filebeat.yml file, but still i can see (failed to connect: 401 unauthorized error) in filebeat log

sahere37 · December 29, 2018, 2:28pm

in addition, the configuration files are as following:

elasticsearch.yml:

     path.data: /var/lib/elasticsearch
     path.logs: /var/log/elasticsearch
     bootstrap.memory_lock: true
     network.host: 'localhost'
     xpack.security.enabled: true
     xpack.security.transport.ssl.enabled: true

kibana.yml:

  server.host: "localhost"
  elasticsearch.url: "http://localhost:9200"
  elasticsearch.username: "kibana"
  elasticsearch.password: "*******"

filebeat.yml

 - type: log
      enabled: true
      paths:
      - /home/srahimi/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/MS1/logs/MS1.log
      tags: 'MS1Log'

       hosts: ["localhost:9200"]
       index: "filebeat"
       setup.template:
             name: 'filebeat'
             pattern: 'filebeat-*'
            enabled: true
       username: "beat_user"
       password: "changeme"

sahere37 · December 30, 2018, 4:56am

I do the steps again, it seems after starting filebeat (which gives us missing authentication error), elasticsearch log have the (failed to retrieve password hash for reserved user [elastic]) error.
could any one advise me about this problem?

sahere37 · December 30, 2018, 11:46am

version of my elasticsearch, filebeat,x-pack and kibana is 6.1.1. i tried another version of filebeat (6.5) and used following command in filebeat.yml:

   xpack.monitoring:
       enabled: true

by adding this line into filebeat.yml, filebeat (v6.5) can stash logs to elasticsearch(v6.1.1) and i can see the index in the discover part of console, therefore the problem is not elasticsearch side and is filebeat side. but it doesn't work in filebeat v6.1 because it doesn't support xpack.monitoring.
I want to use filebeat version 6.1.1 because my elasticsearch and kibana version is 6.1.1, according all these explanations, how can i handle this issue?

sahere37 · December 30, 2018, 12:44pm

I solved the problem. thanks

ikakavas · January 2, 2019, 4:32pm

Hi Sahare,

Could you maybe briefly describe what the actual problem was after all and what steps did you take to resolve this in case this will be helpful for other members of the community facing similar issues?

Thanks

sahere37 · January 5, 2019, 4:54am

hi loannis,
Elasticseach could not verify the authentication information of filebeat. Actually, the filebeat writer user and its role had been define in Kibana console and the same defined in filebeat.yml, but something seems wrong and these information in filebeat side and elasticsearch side seems different. after analysis different situations, i figured out that the problem is in filebeat side and the there is something wrong in the way of my username and password definition in filebeat.yml. according to above posts, the filebeat.yml was as following:

    output.elasticsearch:
       hosts: ["localhost:9200"]
       index: "filebeat"
    setup.template:
       name: 'filebeat'
       pattern: 'filebeat-*'
       enabled: true
      username: "beat_user"
      password: "changeme"

after searching, it has been cleared that, filebeat is searching for "output.elasticsearch.username" and "output.elasticsearch.password" for authentication information. as above filebeat.yml, there are no "output.elasticsearch.username" and "output.elasticsearch.password" , actually username and password information is under "setup.template:" which should be under " output.elasticsearch:
". To handle this issue, i changed the filebeat.yml as following which solved the problem:

     output.elasticsearch:
           hosts: ["localhost:9200"]
           index: "filebeat"
    setup.template:
         name: 'filebeat'
         pattern: 'filebeat-*'
         enabled: true
         output.elasticsearch.username: "beat_user"
         output.elasticsearch.password: "changeme"

system · February 2, 2019, 4:54am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.