Kibana page js cannot respond, 304 error appears in the log

The page does not respond to my actions.

I see some 304 appearing in the kibana log, what is wrong?

https://paste.ubuntu.com/p/CMJ6ygY3TG/

Hi!

It looks like you may have a corrupted object here, and I apologize that this is not logging any error message. The 304 status code is normal, it indicates that these files are already cached by the browser and don't need to be re-downloaded. A few questions:

  • Did you import this visualization from a previous version of Kibana?
  • Was this visualization working recently?
  • Could you paste the results of running this command in the Dev Tools Console? the <id> can be grabbed from the URL bar. It's the alphanumeric sequence after /edit/ and before the ?. It's of the form 93d...df2:
GET /.kibana/_doc/visualization:<id>

Did you import this visualization from a previous version of Kibana?

elasticsearch and kibana version: 7.4.2

kibana is deployed in the kubernetes cluster, using hostpath storage.

I have always used this elastic cluster, but I have previously deployed a higher version of elastic in the same kubernetes cluster, but they do not affect each other.

But suppose that if the data is contaminated, how to solve it ?

Was this visualization working recently?

Often used

visualization doc

{
  "_index" : ".kibana_2",
  "_type" : "_doc",
  "_id" : "visualization:af3c6bb0-9a5d-11ea-be48-1f045d3a2df2",
  "_version" : 2,
  "_seq_no" : 556,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "visualization" : {
      "title" : "PSSO 页面信息统计",
      "visState" : """{"title":"PSSO 页面信息统计","type":"table","params":{"perPage":10,"showPartialRows":false,"showMetricsAtAllLevels":false,"sort":{"columnIndex":null,"direction":null},"showTotal":false,"totalFunc":"sum","percentageCol":"","dimensions":{"metrics":[{"accessor":3,"format":{"id":"number"},"params":{},"label":"唯一数","aggType":"cardinality"}],"buckets":[{"accessor":0,"format":{"id":"terms","params":{"id":"string","otherBucketLabel":"Other","missingBucketLabel":"Missing","parsedUrl":{"origin":"http://192.168.10.183:31560","pathname":"/app/kibana","basePath":""}}},"params":{},"label":"用户名","aggType":"terms"},{"accessor":1,"format":{"id":"terms","params":{"id":"string","otherBucketLabel":"Other","missingBucketLabel":"Missing","parsedUrl":{"origin":"http://192.168.10.183:31560","pathname":"/app/kibana","basePath":""}}},"params":{},"label":"页面路径","aggType":"terms"},{"accessor":2,"format":{"id":"terms","params":{"id":"number","otherBucketLabel":"Other","missingBucketLabel":"Missing","parsedUrl":{"origin":"http://192.168.10.183:31560","pathname":"/app/kibana","basePath":""}}},"params":{},"label":"transaction.duration.us: Descending","aggType":"terms"}]}},"aggs":[{"id":"2","enabled":true,"type":"terms","schema":"bucket","params":{"field":"transaction.custom.name","orderBy":"_key","order":"desc","size":30,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing","customLabel":"用户名"}},{"id":"4","enabled":true,"type":"terms","schema":"bucket","params":{"field":"transaction.page.url","orderBy":"5","order":"desc","size":5,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing","customLabel":"页面路径"}},{"id":"5","enabled":true,"type":"cardinality","schema":"metric","params":{"field":"trace.id","customLabel":"唯一数"}},{"id":"6","enabled":true,"type":"terms","schema":"bucket","params":{"field":"transaction.duration.us","orderBy":"5","order":"desc","size":5,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing","customLabel":"请求时间"}}]}""",
      "uiStateJSON" : """{"vis":{"params":{"sort":{"columnIndex":null,"direction":null}}}}""",
      "description" : "",
      "version" : 1,
      "kibanaSavedObjectMeta" : {
        "searchSourceJSON" : """{"query":{"query":"","language":"kuery"},"filter":[{"meta":{"alias":null,"negate":false,"disabled":false,"type":"phrase","key":"service.name","params":{"query":"psso-development"},"indexRefName":"kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index"},"query":{"match_phrase":{"service.name":"psso-development"}},"$state":{"store":"appState"}}],"indexRefName":"kibanaSavedObjectMeta.searchSourceJSON.index"}"""
      }
    },
    "type" : "visualization",
    "references" : [
      {
        "name" : "kibanaSavedObjectMeta.searchSourceJSON.index",
        "type" : "index-pattern",
        "id" : "af123ca0-82de-11ea-a587-0fefe6ee9b2d"
      },
      {
        "name" : "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index",
        "type" : "index-pattern",
        "id" : "af123ca0-82de-11ea-a587-0fefe6ee9b2d"
      }
    ],
    "migrationVersion" : {
      "visualization" : "7.4.2"
    },
    "updated_at" : "2020-05-20T06:37:25.944Z"
  }
}

@majagrubic This looks similar to other cases you've encountered. How could we confirm this is the same issue and/or fix the visualization?

This problem could be related to a missing index-pattern. Unfortunately, we don't handle the error here very well and just swallow the exception, so there's not a lot of information into what's going around.
Could you please fetch all your index patterns and cross-reference them with the index patterns in the references part of your exported visualization? Basically, you're looking for just one index pattern: af123ca0-82de-11ea-a587-0fefe6ee9b2d

You can get all your index patterns by querying .kibana index from the dev console:

GET /.kibana/_search?q=type:"index-pattern"
{
"_source": false
}

I executed the query

GET /.kibana/_search?q=type:"index-pattern"
{
"_source": false
}
{
  "took" : 15,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 3.9512439,
    "hits" : [
      {
        "_index" : ".kibana_2",
        "_type" : "_doc",
        "_id" : "index-pattern:apm_static_index_pattern_id",
        "_score" : 3.9512439
      },
      {
        "_index" : ".kibana_2",
        "_type" : "_doc",
        "_id" : "index-pattern:9c49f420-90c9-11ea-be48-1f045d3a2df2",
        "_score" : 3.9512439
      }
    ]
  }
}

In the list of the index patterns, you have two index patterns:
apm_static_index_pattern_id and 9c49f420-90c9-11ea-be48-1f045d3a2df2.
Your visualization references the index pattern with the id: af123ca0-82de-11ea-a587-0fefe6ee9b2d, which is not present in your index patterns. I suspect you deleted that index pattern in the meantime. Unfortunately, in visualization references we only store index pattern ids, so I can't tell you what that index pattern was called.
The mitigation here is that in the saved objects UI you modify your visualization by deleting the content of the filter array from `searchSourceJSON. You can do that as follows:

  1. go to the management console
  2. select Saved Objects from the menu
  3. find your visualization in the list
  4. from the actions selector, select Inspect

5.Find kibanaSavedObjectMeta.searchSourceJSON
6. Delete all the content of the filter array, but leave filter keyword and an empty array


7. Save

This will allow the visualization to load but without filters. You will need to recreate your filters.

In the meantime, I'm investigating with my colleagues if there was a migration gone wrong that caused this index pattern to be deleted and if we can prevent this from happening in the future.

@majagrubic
Thank you for your help.

I removed the content in the filter according to your method, but when I open the page again, it still shows that it is empty, which is a bit strange. And I noticed that the id of this index pattern is also different from the one in the picture above. Does this matter?


You are right, I did delete an index pattern before. Before this, I also have a question, can the visualize objects already designed change its source index?

On a second glance, I see that both your visualization AND your filter reference a missing index pattern. I don't think there's anything you can do here, except recreating a deleted index pattern.

And I noticed that the id of this index pattern is also different from the one in the picture above

I am not sure I understand what you mean here. It's the same missing index pattern id in both places.

can the visualize objects already designed change its source index?

It is only possible by modifying the index pattern id in the saved objects inspector directly, in a similar way you modified the filter array. I'm afraid we don't have a UI-friendly way of doing this. We have an open issue for this, so you can track progress there: https://github.com/elastic/kibana/issues/17542

@majagrubic Thanks for your help, now I think the fastest solution is to delete visualization directly and redesign one