Kibana error after upgrade to version 8.0 [config validation of [logging].dest]: definition for this key is missing]

Elastic Stack Kibana
1

Hi,

I had successfully upgraded to version 8.0 on ELK , while the rest of the stack works , kibana has failed to start citing

dev-elk-app01 kibana[19018]:  FATAL  Error: [config validation of [logging].dest]: definition for this key is missing

i understand from a bit of googling that logging.dest is no longer supported on 8.0 , however my yml configuration has no entry for logging.dest .

Could someone help out ?

here's the full error:

Feb 22 20:24:35 dev-elk-app01 systemd[1]: Started Kibana.
Feb 22 20:24:37 dev-elk-app01 kibana[19121]: Configuring logger failed: ValidationError: [config validation of [logging].dest]: definition for this key is missing
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at ObjectType.validate (/usr/share/kibana/node_modules/@kbn/config-schema/target_node/types/type.js:95:13)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at ConfigService.validateAtPath (/usr/share/kibana/node_modules/@kbn/config/target_node/config_service.js:228:19)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at MapSubscriber.project (/usr/share/kibana/node_modules/@kbn/config/target_node/config_service.js:236:169)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:49:35)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at DistinctUntilChangedSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at DistinctUntilChangedSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:55:26)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     at ReplaySubject._subscribe (/usr/share/kibana/node_modules/rxjs/internal/ReplaySubject.js:80:28) {
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:   cause: SchemaTypeError: definition for this key is missing
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at ObjectType.onError (/usr/share/kibana/node_modules/@kbn/config-schema/target_node/types/type.js:137:14)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at Object.error (/usr/share/kibana/node_modules/@kbn/config-schema/target_node/types/type.js:79:47)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at Object.internals.finalize (/usr/share/kibana/node_modules/joi/lib/validator.js:472:36)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at Object.exports.validate (/usr/share/kibana/node_modules/joi/lib/validator.js:333:38)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at Object.internals.entry (/usr/share/kibana/node_modules/joi/lib/validator.js:142:28)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at Object.exports.entry (/usr/share/kibana/node_modules/joi/lib/validator.js:27:30)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at internals.Base.validate (/usr/share/kibana/node_modules/joi/lib/base.js:548:26)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at ObjectType.validate (/usr/share/kibana/node_modules/@kbn/config-schema/target_node/types/type.js:89:29)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at ConfigService.validateAtPath (/usr/share/kibana/node_modules/@kbn/config/target_node/config_service.js:228:19)
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:       at MapSubscriber.project (/usr/share/kibana/node_modules/@kbn/config/target_node/config_service.js:236:169) {
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     cause: undefined,
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:     path: [ 'dest' ]
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:   }
Feb 22 20:24:37 dev-elk-app01 kibana[19121]: }
Feb 22 20:24:37 dev-elk-app01 kibana[19121]:  FATAL  Error: [config validation of [logging].dest]: definition for this key is missing
Feb 22 20:24:37 dev-elk-app01 systemd[1]: kibana.service: Main process exited, code=exited, status=1/FAILURE
Feb 22 20:24:37 dev-elk-app01 systemd[1]: kibana.service: Failed with result 'exit-code'.
Feb 22 20:24:40 dev-elk-app01 systemd[1]: kibana.service: Service hold-off time over, scheduling restart.
Feb 22 20:24:40 dev-elk-app01 systemd[1]: kibana.service: Scheduled restart job, restart counter is at 1.
Feb 22 20:24:40 dev-elk-app01 systemd[1]: Stopped Kibana.

Here;s the YML file

root@dev-elk-app01:/etc/kibana# cat kibana.yml| grep -v "#"

server.port: 5601

server.host: "172.26.207.174"


server.publicBaseUrl: "http://kibana-dev.staging.com:5601"



server.name: "kibana-dev01"
elasticsearch.hosts: ["http://172.26.207.179:9200"]
logging.root.level: debug

logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

path.data: /var/lib/kibana/
pid.file: /run/kibana/kibana.pid
data.autocomplete.valueSuggestions.timeout: 100000000
data.autocomplete.valueSuggestions.terminateAfter: 100000000
2 Likes
2

found the reason , it seems my systemd unit was still referring to logging.dest , removing it seemed to have worked and moved on to the next problem

ExecStart=/usr/share/kibana/bin/kibana --logging.dest="/var/log/kibana/kibana.log" --pid.file="/run/kibana/kibana.pid" --deprecation.skip_deprecated_settings[0]="logging.dest"

new error

 Feb 22 20:47:48 dev-elk-app01 kibana[22911]:  FATAL  Error: [config validation of [elasticsearch].username]: value of "elastic" is forbidden. This is a superuser account that cannot write to system indices that Kibana needs to function. Use a service account token instead. Learn more: https://www.elastic.co/guide/en/elasticsearch/reference/8.0/service-accounts.html
Feb 22 20:47:48 dev-elk-app01 systemd[1]: kibana.service: Main process exited, code=exited, status=78/n/a
Feb 22 20:47:48 dev-elk-app01 systemd[1]: kibana.service: Failed with result 'exit-code'.
4 Likes
3

How did you manage to remove it ?

4 Likes
4

Hey,

I had the same issue.

To fix it on Ubuntu/Debian:
Open /etc/systemd/system/kibana.service in your favorite editor.
Find the ExecStart line and remove the logging.dest setting.

Before:
ExecStart=/usr/share/kibana/bin/kibana --logging.dest="/var/log/kibana/kibana.log" --pid.file="/run/kibana/kibana.pid" --deprecation.skip_deprecated_settings[0]="logging.dest"

After:
ExecStart=/usr/share/kibana/bin/kibana --pid.file="/run/kibana/kibana.pid" --deprecation.skip_deprecated_settings[0]="logging.dest"

It looks like the command line argument is deprecated and it's expected that setting will be in the /etc/kibana/kibana.yml.

If you want to have kibana logs in /var/log/kibana/kibana.log, make sure your /etc/kibana/kibana.yml has this in it:

logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:

Dustin

4 Likes
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.