. Response: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}
```
---
### Author:  [@knagasri](https://discuss.elastic.co/u/knagasri)
#### Post date: [December 7, 2020, 5:07pm UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/2 "2020-12-07T17:07:57Z")
Any one answer this?
Elasticsearch health also becomes red
---
### Author:  [@knagasri](https://discuss.elastic.co/u/knagasri)
#### Post date: [December 17, 2020, 3:19am UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/3 "2020-12-17T03:19:19Z")
I am directly using the sample examples that are given from the above link. Can anyone answer Do I need to change anything. Below is my file-beat config
```auto
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
name: filebeat
spec:
type: filebeat
version: 7.9.2
elasticsearchRef:
name: elasticsearch-sample
kibanaRef:
name: kibana-sample
config:
filebeat:
autodiscover:
providers:
- type: kubernetes
node: ${NODE_NAME}
hints:
enabled: true
default_config:
type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
processors:
- add_cloud_metadata: {}
- add_host_metadata: {}
daemonSet:
podTemplate:
spec:
serviceAccountName: elastic-operator
automountServiceAccountToken: true
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true # Allows to provide richer host metadata
containers:
- name: filebeat
securityContext:
runAsUser: 0
# If using Red Hat OpenShift uncomment this:
#privileged: true
volumeMounts:
- name: varlogcontainers
mountPath: /var/log/containers
- name: varlogpods
mountPath: /var/log/pods
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumes:
- name: varlogcontainers
hostPath:
path: /var/log/containers
- name: varlogpods
hostPath:
path: /var/log/pods
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
```
---
### Author:  [@mtojek](https://discuss.elastic.co/u/mtojek)
#### Post date: [December 17, 2020, 8:46am UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/4 "2020-12-17T08:46:55Z")
It looks like HTTP 500 which should be present in logs. Did you check Elasticsearch or Kibana for any more details?
---
### Author:  [@knagasri](https://discuss.elastic.co/u/knagasri)
#### Post date: [December 24, 2020, 6:31am UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/5 "2020-12-24T06:31:57Z")
When I see kibana logs, I found below error:
```auto
{"type":"log","@timestamp":"2020-12-24T06:25:54Z","tags":["info","plugins","security","authentication"],"pid":6,"message":"Authentication attempt failed: [security_exception] unable to authenticate user [es-upgrade-filebeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}
{"type":"log","@timestamp":"2020-12-24T06:25:54Z","tags":["info","plugins","security","authentication"],"pid":6,"message":"Authentication attempt failed: [security_exception] unable to authenticate user [es-upgrade-auditbeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}
{"type":"response","@timestamp":"2020-12-24T06:25:54Z","tags":["api"],"pid":6,"method":"get","statusCode":401,"req":{"url":"/api/status","method":"get","headers":{"host":"kibana-config-kb-http.es-upgrade.svc:5601","user-agent":"Go-http-client/1.1","accept":"application/json","content-type":"application/json","kbn-xsrf":"1","accept-encoding":"gzip"},"remoteAddress":"10.90.0.119","userAgent":"10.90.0.119"},"res":{"statusCode":401,"responseTime":46,"contentLength":9},"message":"GET /api/status 401 46ms - 9.0B"}
{"type":"response","@timestamp":"2020-12-24T06:25:54Z","tags":["api"],"pid":6,"method":"get","statusCode":401,"req":{"url":"/api/status","method":"get","headers":{"host":"kibana-config-kb-http.es-upgrade.svc:5601","user-agent":"Go-http-client/1.1","accept":"application/json","content-type":"application/json","kbn-xsrf":"1","accept-encoding":"gzip"},"remoteAddress":"10.90.0.40","userAgent":"10.90.0.40"},"res":{"statusCode":401,"responseTime":58,"contentLength":9},"message":"GET /api/status 401 58ms - 9.0B"}
{"type":"log","@timestamp":"2020-12-24T06:25:54Z","tags":["info","plugins","security","authentication"],"pid":6,"message":"Authentication attempt failed: [security_exception] unable to authenticate user [es-upgrade-packetbeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}
{"type":"log","@timestamp":"2020-12-24T06:25:54Z","tags":["info","plugins","security","authentication"],"pid":6,"message":"Authentication attempt failed: [security_exception] unable to authenticate user [es-upgrade-auditbeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}
{"type":"response","@timestamp":"2020-12-24T06:25:54Z","tags":["api"],"pid":6,"method":"get","statusCode":401,"req":{"url":"/api/status","method":"get","headers":{"host":"kibana-config-kb-http.es-upgrade.svc:5601","user-agent":"Go-http-client/1.1","accept":"application/json","content-type":"application/json","kbn-xsrf":"1","accept-encoding":"gzip"},"remoteAddress":"10.90.0.104","userAgent":"10.90.0.104"},"res":{"statusCode":401,"responseTime":42,"contentLength":9},"message":"GET /api/status 401 42ms - 9.0B"}
{"type":"response","@timestamp":"2020-12-24T06:25:54Z","tags":["api"],"pid":6,"method":"get","statusCode":401,"req":{"url":"/api/status","method":"get","headers":{"host":"kibana-config-kb-http.es-upgrade.svc:5601","user-agent":"Go-http-client/1.1","accept":"application/json","content-type":"application/json","kbn-xsrf":"1","accept-encoding":"gzip"},"remoteAddress":"10.90.0.134","userAgent":"10.90.0.134"},"res":{"statusCode":401,"responseTime":43,"contentLength":9},"message":"GET /api/status 401 43ms - 9.0B"}
{"type":"log","@timestamp":"2020-12-24T06:25:54Z","tags":["info","plugins","security","authentication"],"pid":6,"message":"Authentication attempt failed: [security_exception] unable to authenticate user [es-upgrade-auditbeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}
{"type":"log","@timestamp":"2020-12-24T06:25:54Z","tags":["info","plugins","security","authentication"],"pid":6,"message":"Authentication attempt failed: [security_exception] unable to authenticate user [es-upgrade-filebeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}
```
---
### Author:  [@knagasri](https://discuss.elastic.co/u/knagasri)
#### Post date: [January 4, 2021, 5:16pm UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/6 "2021-01-04T17:16:38Z")
Has anyone know why beats pods are in 3 or 4 restarts
```auto
kubectl -n es-upgrade2 get po | grep packet
packetbeat-beat-packetbeat-2cvwp 1/1 Running 3 4h4m
packetbeat-beat-packetbeat-466t8 1/1 Running 3 4h4m
packetbeat-beat-packetbeat-4q5nx 1/1 Running 4 4h4m
packetbeat-beat-packetbeat-7x8fv 1/1 Running 3 4h4m
packetbeat-beat-packetbeat-98ck4 1/1 Running 3 4h4m
packetbeat-beat-packetbeat-98pl4 1/1 Running 3 4h4m
packetbeat-beat-packetbeat-992z6 1/1 Running 0 4h4m
```
logs of one of packet beat
kubectl -n es-upgrade2 logs packetbeat-beat-packetbeat-466t8
```auto
2021-01-04T17:13:21.339Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":33940,"time":{"ms":20}},"total":{"ticks":203130,"time":{"ms":100},"value":203130},"user":{"ticks":169190,"time":{"ms":80}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral_id":"fe687c57-5f12-4bea-9a6b-7707292eeaa2","uptime":{"ms":14553100}},"memstats":{"gc_next":118960976,"memory_alloc":84188360,"memory_total":23846088272},"runtime":{"goroutines":19}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":398,"batches":8,"total":398}},"pipeline":{"clients":3,"events":{"active":4117,"published":398,"total":398},"queue":{"acked":398}}},"system":{"load":{"1":5.24,"15":4.89,"5":4.89,"norm":{"1":0.0655,"15":0.0611,"5":0.0611}}}}}}
2021-01-04T17:13:51.338Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":34100,"time":{"ms":158}},"total":{"ticks":203940,"time":{"ms":810},"value":203940},"user":{"ticks":169840,"time":{"ms":652}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral_id":"fe687c57-5f12-4bea-9a6b-7707292eeaa2","uptime":{"ms":14583100}},"memstats":{"gc_next":103774448,"memory_alloc":55803608,"memory_total":23926411312},"runtime":{"goroutines":19}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":350,"batches":7,"total":350}},"pipeline":{"clients":3,"events":{"active":4117,"failed":21,"published":350,"total":371},"queue":{"acked":350}}},"system":{"load":{"1":4.95,"15":4.88,"5":4.85,"norm":{"1":0.0619,"15":0.061,"5":0.0606}}},"tcp":{"dropped_because_of_gaps":4}}}}
```
---
### Author:  [@knagasri](https://discuss.elastic.co/u/knagasri)
#### Post date: [January 4, 2021, 5:22pm UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/7 "2021-01-04T17:22:47Z")
when I try to see the previous logs
```auto
kubectl -n es-upgrade2 logs packetbeat-beat-packetbeat-466t8 --previous
2021-01-04T13:10:10.136Z INFO kibana/client.go:119 Kibana url: https://kibana-config-kb-http.es-upgrade2.svc:5601
2021-01-04T13:10:10.334Z INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":470,"time":{"ms":476}},"total":{"ticks":870,"time":{"ms":881},"value":870},"user":{"ticks":400,"time":{"ms":405}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"9b096216-30d2-4925-a736-8c9d03cba656","uptime":{"ms":1846}},"memstats":{"gc_next":41963280,"memory_alloc":35809304,"memory_total":38786080,"rss":84750336},"runtime":{"goroutines":15}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":3,"events":{"active":0}}},"system":{"cpu":{"cores":80},"load":{"1":5.05,"15":6.01,"5":5.26,"norm":{"1":0.0631,"15":0.0751,"5":0.0658}}}}}}
2021-01-04T13:10:10.334Z INFO [monitoring] log/log.go:154 Uptime: 1.896042773s
2021-01-04T13:10:10.334Z INFO [monitoring] log/log.go:131 Stopping metrics logging.
2021-01-04T13:10:10.334Z INFO instance/beat.go:447 packetbeat stopped.
2021-01-04T13:10:10.334Z ERROR instance/beat.go:951 Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://kibana-config-kb-http.es-upgrade2.svc:5601/api/status fails: . Response: {"statusCode":401,"error":"Unauthorized","message":"[security_exception] unable to authenticate user [es-upgrade2-packetbeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\... (truncated).
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://kibana-config-kb-http.es-upgrade2.svc:5601/api/status fails: . Response: {"statusCode":401,"error":"Unauthorized","message":"[security_exception] unable to authenticate user [es-upgrade2-packetbeat-beat-kb-user] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\... (truncated).
```
I can see the same in audit and remaining beats as well.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 1, 2021, 7:22pm UTC](https://discuss.elastic.co/t/trying-to-setup-beats/257794/8 "2021-02-01T19:22:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.