Hi All,
I am using kibana dashboards to visualize the data. but i am getting "Could not find the data view" error very frequently and when i refresh again, this error doesn't come. Please help in resolving this error.
There is one more error coming related to tsvb Gateway timeout. Please check if both are interlinked.
kibana version is 8.7.0
Can someone please help here?
Hi @Sachchan
when you have the error in the dashboard, if you open the visualization in the editor (Edit Dashboard -> top right click on the visualization panel) do you still see the error?
yes, error is still there.
apart from that we are getting below error in kibana nohup
[2023-05-10T14:51:56.166+05:30][ERROR][plugins.taskManager] Task reports:monitor "reports:monitor" failed: TimeoutError: Request timed out
[2023-05-10T14:52:02.874+05:30][WARN ][plugins.taskManager] Task reports:monitor "reports:monitor" failed in attempt to run: undefined
[2023-05-10T14:52:24.905+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:52:29.068+05:30][INFO ][status] Kibana is now available (was degraded)
[2023-05-10T14:53:01.398+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:53:06.170+05:30][INFO ][status] Kibana is now available (was degraded)
[2023-05-10T14:53:57.077+05:30][ERROR][plugins.taskManager] Failed to poll for work: Error: work has timed out
[2023-05-10T14:53:57.190+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:54:10.832+05:30][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":false,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"inactive":0,"unenrolled":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-05-10T14:54:15.218+05:30][INFO ][status] Kibana is now available (was degraded)
[2023-05-10T14:55:09.842+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:55:39.820+05:30][ERROR][plugins.taskManager] Task reports:monitor "reports:monitor" failed: TimeoutError: Request timed out
[2023-05-10T14:55:42.083+05:30][ERROR][plugins.taskManager] Failed to poll for work: Error: work has timed out
[2023-05-10T14:55:58.627+05:30][WARN ][plugins.taskManager] Task reports:monitor "reports:monitor" failed in attempt to run: undefined
[2023-05-10T14:55:58.628+05:30][WARN ][plugins.usageCollection.usage-collection.usage-counters-service] ResponseError: [usage-counters:ebt_counters.client:10052023:enqueued_enqueued:core-overall_status_changed]: version conflict, required seqNo [74572], primary term [12]. current document has seqNo [74575] and primary term [12]: version_conflict_engine_exception
Root causes:
version_conflict_engine_exception: [usage-counters:ebt_counters.client:10052023:enqueued_enqueued:core-overall_status_changed]: version conflict, required seqNo [74572], primary term [12]. current document has seqNo [74575] and primary term [12]
at KibanaTransport.request (/finapp/kibana-8.7.0/node_modules/@elastic/transport/lib/Transport.js:479:27)
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at KibanaTransport.request (/finapp/kibana-8.7.0/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.UpdateApi [as update] (/finapp/kibana-8.7.0/node_modules/@elastic/elasticsearch/lib/api/api/update.js:50:12)
[2023-05-10T14:56:12.084+05:30][ERROR][plugins.taskManager] Failed to poll for work: Error: work has timed out
[2023-05-10T14:56:19.874+05:30][INFO ][status] Kibana is now available (was degraded)
[2023-05-10T14:57:28.539+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:57:36.662+05:30][INFO ][status] Kibana is now available (was degraded)
[2023-05-10T14:57:46.304+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:58:16.232+05:30][ERROR][plugins.taskManager] Failed to poll for work: Error: work has timed out
[2023-05-10T14:58:16.336+05:30][ERROR][plugins.taskManager] [Task Runner] Task reports:monitor failed to release claim after failure: [object Object]
[2023-05-10T14:58:16.336+05:30][ERROR][plugins.taskManager] Failed to mark Task reports:monitor "reports:monitor" as running: undefined
[2023-05-10T14:58:21.192+05:30][INFO ][status] Kibana is now available (was degraded)
[2023-05-10T14:58:57.232+05:30][INFO ][status] Kibana is now degraded (was available)
[2023-05-10T14:59:41.900+05:30][ERROR][plugins.taskManager] Failed to poll for work: Error: work has timed out
[2023-05-10T14:59:41.972+05:30][ERROR][plugins.taskManager] Task endpoint:user-artifact-packager "endpoint:user-artifact-packager:1.0.0" failed: TimeoutError: Request timed out
[2023-05-10T14:59:53.494+05:30][WARN ][plugins.taskManager] Task endpoint:user-artifact-packager "endpoint:user-artifact-packager:1.0.0" failed in attempt to run: undefined
I see many timeout and exceptions in that log.
What about the editor panel? What error do you see in Kibana UI?
Getting the attached error.
Do you get all those errors/timeout only when opening a dashboard? What if you open the dataView management page?
Yes, sometime opening the dashboard and sometime while refreshing the dashboard. behavior is random. No error comes in data view management page
@Marco_Liberati Is it to do something with the errors in the logs? How can we resolve this? Sometime dashboards load very fast but sometime it is dead slow. Kindly sugggest what can be done.
The error messages seems unrelated to me to the dashboard loading. 
@devon.thomson do you have any hint here?
I'm not exactly sure what's going on, it doesn't seem to be specific to a dashboard to me. @Sachchan, it looks like the issue is happening on a legacy visualize or a TSVB panel. Does this error happen to all panels using the same data view? Or does it happen to all panels of the same type (tsvb, visualize etc), or does it happen to all panels regardless of the data view / panel type?
In my opinion the most likely culprit here is something to do with the performance of your cluster.
@devon.thomson this issue is with all the panels using tsvb with different data views but issue is not consistent. Looks like something is breaking in between. Ideally graphs loading complete in 5-10 seconds but when it takes more than that then possibility of this error coming is high.
I have 2 node cluster. 32 G ram and 64 VCPU machines. On one node elastic,kibana,logstash and 2 instance of filebeat running and on other node 2 instance of filebeat.
This seems to me like it would have something to do with load on the cluster. Let me reach out the the Core team to see if the Kibana logs posted above could explain to them why the Data View saved object can't be found.
Please let me know if you need more logs. @devon.thomson
Welcome to our community! 
What is the output from the _cluster/stats?pretty&human API in Elasticsearch?
{
"_nodes": {
"total": 1,
"successful": 1,
"failed": 0
},
"cluster_name": "elasticsearch",
"cluster_uuid": "teD8_Jk6Tc-u0RDMlSGyWA",
"timestamp": 1683772896686,
"status": "yellow",
"indices": {
"count": 211,
"shards": {
"total": 211,
"primaries": 211,
"replication": 0,
"index": {
"shards": {
"min": 1,
"max": 1,
"avg": 1
},
"primaries": {
"min": 1,
"max": 1,
"avg": 1
},
"replication": {
"min": 0,
"max": 0,
"avg": 0
}
}
},
"docs": {
"count": 2341693506,
"deleted": 77173
},
"store": {
"size": "734.2gb",
"size_in_bytes": 788380153070,
"total_data_set_size": "734.2gb",
"total_data_set_size_in_bytes": 788380153070,
"reserved": "0b",
"reserved_in_bytes": 0
},
"fielddata": {
"memory_size": "125.8kb",
"memory_size_in_bytes": 128888,
"evictions": 0
},
"query_cache": {
"memory_size": "345.5mb",
"memory_size_in_bytes": 362356494,
"total_count": 7362416,
"hit_count": 414859,
"miss_count": 6947557,
"cache_size": 3584,
"cache_count": 5952,
"evictions": 2368
},
"completion": {
"size": "0b",
"size_in_bytes": 0
},
"segments": {
"count": 3742,
"memory": "0b",
"memory_in_bytes": 0,
"terms_memory": "0b",
"terms_memory_in_bytes": 0,
"stored_fields_memory": "0b",
"stored_fields_memory_in_bytes": 0,
"term_vectors_memory": "0b",
"term_vectors_memory_in_bytes": 0,
"norms_memory": "0b",
"norms_memory_in_bytes": 0,
"points_memory": "0b",
"points_memory_in_bytes": 0,
"doc_values_memory": "0b",
"doc_values_memory_in_bytes": 0,
"index_writer_memory": "688.5mb",
"index_writer_memory_in_bytes": 721979672,
"version_map_memory": "20.4mb",
"version_map_memory_in_bytes": 21482237,
"fixed_bit_set": "95.7mb",
"fixed_bit_set_memory_in_bytes": 100351704,
"max_unsafe_auto_id_timestamp": 1683743449826,
"file_sizes": {}
},
"mappings": {
"total_field_count": 25872,
"total_deduplicated_field_count": 12782,
"total_deduplicated_mapping_size": "75.5kb",
"total_deduplicated_mapping_size_in_bytes": 77321,
"field_types": [
{
"name": "alias",
"count": 1777,
"index_count": 17,
"script_count": 0
},
{
"name": "binary",
"count": 3,
"index_count": 3,
"script_count": 0
},
{
"name": "boolean",
"count": 139,
"index_count": 19,
"script_count": 0
},
{
"name": "byte",
"count": 2,
"index_count": 2,
"script_count": 0
},
{
"name": "constant_keyword",
"count": 10,
"index_count": 4,
"script_count": 0
},
{
"name": "date",
"count": 658,
"index_count": 200,
"script_count": 0
},
{
"name": "date_nanos",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "date_range",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "double",
"count": 373,
"index_count": 14,
"script_count": 0
},
{
"name": "double_range",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "flattened",
"count": 12,
"index_count": 1,
"script_count": 0
},
{
"name": "float",
"count": 343,
"index_count": 19,
"script_count": 0
},
{
"name": "float_range",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "geo_point",
"count": 10,
"index_count": 2,
"script_count": 0
},
{
"name": "geo_shape",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "half_float",
"count": 33,
"index_count": 13,
"script_count": 0
},
{
"name": "integer",
"count": 14,
"index_count": 6,
"script_count": 0
},
{
"name": "integer_range",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "ip",
"count": 27,
"index_count": 5,
"script_count": 0
},
{
"name": "ip_range",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "keyword",
"count": 6286,
"index_count": 201,
"script_count": 0
},
{
"name": "long",
"count": 5309,
"index_count": 195,
"script_count": 0
},
{
"name": "long_range",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "match_only_text",
"count": 63,
"index_count": 1,
"script_count": 0
},
{
"name": "nested",
"count": 40,
"index_count": 10,
"script_count": 0
},
{
"name": "object",
"count": 6911,
"index_count": 199,
"script_count": 0
},
{
"name": "scaled_float",
"count": 155,
"index_count": 4,
"script_count": 0
},
{
"name": "shape",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "short",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "text",
"count": 3674,
"index_count": 182,
"script_count": 0
},
{
"name": "version",
"count": 1,
"index_count": 1,
"script_count": 0
},
{
"name": "wildcard",
"count": 22,
"index_count": 5,
"script_count": 0
}
],
"runtime_field_types": []
},
"analysis": {
"char_filter_types": [],
"tokenizer_types": [],
"filter_types": [],
"analyzer_types": [],
"built_in_char_filters": [],
"built_in_tokenizers": [],
"built_in_filters": [],
"built_in_analyzers": []
},
"versions": [
{
"version": "8.7.0",
"index_count": 211,
"primary_shard_count": 211,
"total_primary_size": "734.2gb",
"total_primary_bytes": 788380153070
}
],
"search": {
"total": 279788,
"queries": {
"bool": 279567,
"prefix": 2,
"match": 5387,
"range": 253466,
"nested": 4,
"multi_match": 2,
"wildcard": 2,
"match_phrase": 786,
"terms": 15645,
"match_phrase_prefix": 97,
"exists": 22331,
"term": 255439,
"query_string": 2391,
"simple_query_string": 5125
},
"sections": {
"highlight": 410,
"stored_fields": 3269,
"runtime_mappings": 227873,
"query": 279571,
"script_fields": 3269,
"terminate_after": 27,
"pit": 177,
"_source": 1011,
"fields": 3451,
"collapse": 96,
"aggs": 237646
}
}
},
"nodes": {
"count": {
"total": 1,
"coordinating_only": 0,
"data": 1,
"data_cold": 1,
"data_content": 1,
"data_frozen": 1,
"data_hot": 1,
"data_warm": 1,
"index": 0,
"ingest": 1,
"master": 1,
"ml": 1,
"remote_cluster_client": 1,
"search": 0,
"transform": 1,
"voting_only": 0
},
"versions": [
"8.7.0"
],
"os": {
"available_processors": 64,
"allocated_processors": 64,
"names": [
{
"name": "Linux",
"count": 1
}
],
"pretty_names": [
{
"pretty_name": "CentOS Linux 7 (Core)",
"count": 1
}
],
"architectures": [
{
"arch": "amd64",
"count": 1
}
],
"mem": {
"total": "62.3gb",
"total_in_bytes": 66946830336,
"adjusted_total": "62.3gb",
"adjusted_total_in_bytes": 66946830336,
"free": "3.8gb",
"free_in_bytes": 4187348992,
"used": "58.4gb",
"used_in_bytes": 62759481344,
"free_percent": 6,
"used_percent": 94
}
},
"process": {
"cpu": {
"percent": 3
},
"open_file_descriptors": {
"min": 3362,
"max": 3362,
"avg": 3362
}
},
"jvm": {
"max_uptime": "1.7d",
"max_uptime_in_millis": 150069341,
"versions": [
{
"version": "19.0.2",
"vm_name": "OpenJDK 64-Bit Server VM",
"vm_version": "19.0.2+7-44",
"vm_vendor": "Oracle Corporation",
"bundled_jdk": true,
"using_bundled_jdk": true,
"count": 1
}
],
"mem": {
"heap_used": "4.3gb",
"heap_used_in_bytes": 4638216528,
"heap_max": "8gb",
"heap_max_in_bytes": 8589934592
},
"threads": 458
},
"fs": {
"total": "8tb",
"total_in_bytes": 8796093022208,
"free": "7.1tb",
"free_in_bytes": 7820335710208,
"available": "7.1tb",
"available_in_bytes": 7820334792704
},
"plugins": [],
"network_types": {
"transport_types": {
"security4": 1
},
"http_types": {
"security4": 1
}
},
"discovery_types": {
"multi-node": 1
},
"packaging_types": [
{
"flavor": "default",
"type": "tar",
"count": 1
}
],
"ingest": {
"number_of_pipelines": 21,
"processor_stats": {
"date": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"geoip": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"grok": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"pipeline": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"remove": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"rename": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"script": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"set": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
},
"set_security_user": {
"count": 0,
"failed": 0,
"current": 0,
"time": "0s",
"time_in_millis": 0
}
}
},
"indexing_pressure": {
"memory": {
"current": {
"combined_coordinating_and_primary": "0b",
"combined_coordinating_and_primary_in_bytes": 0,
"coordinating": "0b",
"coordinating_in_bytes": 0,
"primary": "0b",
"primary_in_bytes": 0,
"replica": "0b",
"replica_in_bytes": 0,
"all": "0b",
"all_in_bytes": 0
},
"total": {
"combined_coordinating_and_primary": "0b",
"combined_coordinating_and_primary_in_bytes": 0,
"coordinating": "0b",
"coordinating_in_bytes": 0,
"primary": "0b",
"primary_in_bytes": 0,
"replica": "0b",
"replica_in_bytes": 0,
"all": "0b",
"all_in_bytes": 0,
"coordinating_rejections": 0,
"primary_rejections": 0,
"replica_rejections": 0
},
"limit": "0b",
"limit_in_bytes": 0
}
}
}
}
Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you 
@warkolm formatted the code in the previous comment. Please have a look.
Are you running things other than Elasticsearch on this host?
Yes. Logstash,kibana, filebeat
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.