Receiving org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed after 7.17.1 to 8.2.3 upgrade

[newlife007 (Mark Harris)]

Receiving org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed after 7.17.1 to 8.2.3 upgrade. This is causing permission issues in accessing indexes in kibana and API using elastic user stating superuser doesn't have permissions. Cannot delete indexes and DataViews are giving errors.

Seeing this in the elasticsearch log.

org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:729) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:419) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:761) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:513) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:350) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:92) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:48) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:642) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.transport.TransportService$UnregisterChildTransportResponseHandler.handleException(TransportService.java:1599) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1339) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1461) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1436) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:50) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:48) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.action.ActionRunnable.onFailure(ActionRunnable.java:77) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:757) [elasticsearch-8.3.2.jar:?]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) [elasticsearch-8.3.2.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]

And getting this error in Kibana.

Error fetching fields for data view ,.alerts-security.alerts-default,apm-*-transaction*,auditbeat-*,filebeat-*,logs-*,packetbeat-*,traces-apm*,winlogbeat-* (ID: security-solution-default)
[object Object]: security_exception: [security_exception] Reason: action [indices:data/read/field_caps] is unauthorized for user [elastic] with roles [superuser], this action is granted by the index privileges [view_index_metadata,manage,read,all]
Error: [object Object]: security_exception: [security_exception] Reason: action [indices:data/read/field_caps] is unauthorized for user [elastic] with roles [superuser], this action is granted by the index privileges [view_index_metadata,manage,read,all]
    at https://elk.sgoc.local:5601/52239/bundles/plugin/dataViews/kibana/dataViews.plugin.js:1:38334
    at async data_views_service_public_DataViewsServicePublic.refreshFieldSpecMap (https://elk.sgoc.local:5601/52239/bundles/plugin/dataViews/kibana/dataViews.plugin.js:1:23084)
    at async data_views_service_public_DataViewsServicePublic.initFromSavedObject (https://elk.sgoc.local:5601/52239/bundles/plugin/dataViews/kibana/dataViews.plugin.js:1:24761)

Any help is much appreciated. Been searching and working on this issue for over a week.