Kibana page js cannot respond, 304 error appears in the log

wajika · May 21, 2020, 7:16am

The page does not respond to my actions.

I see some 304 appearing in the kibana log, what is wrong?

https://paste.ubuntu.com/p/CMJ6ygY3TG/

joshdover · May 21, 2020, 7:38pm

Hi!

It looks like you may have a corrupted object here, and I apologize that this is not logging any error message. The 304 status code is normal, it indicates that these files are already cached by the browser and don't need to be re-downloaded. A few questions:

Did you import this visualization from a previous version of Kibana?
Was this visualization working recently?
Could you paste the results of running this command in the Dev Tools Console? the <id> can be grabbed from the URL bar. It's the alphanumeric sequence after /edit/ and before the ?. It's of the form 93d...df2:

GET /.kibana/_doc/visualization:<id>

wajika · May 22, 2020, 2:15am

Did you import this visualization from a previous version of Kibana?

elasticsearch and kibana version: 7.4.2

kibana is deployed in the kubernetes cluster, using hostpath storage.

I have always used this elastic cluster, but I have previously deployed a higher version of elastic in the same kubernetes cluster, but they do not affect each other.

But suppose that if the data is contaminated, how to solve it ?

Was this visualization working recently?

Often used

visualization doc

{
  "_index" : ".kibana_2",
  "_type" : "_doc",
  "_id" : "visualization:af3c6bb0-9a5d-11ea-be48-1f045d3a2df2",
  "_version" : 2,
  "_seq_no" : 556,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "visualization" : {
      "title" : "PSSO 页面信息统计",
      "visState" : """{"title":"PSSO 页面信息统计","type":"table","params":{"perPage":10,"showPartialRows":false,"showMetricsAtAllLevels":false,"sort":{"columnIndex":null,"direction":null},"showTotal":false,"totalFunc":"sum","percentageCol":"","dimensions":{"metrics":[{"accessor":3,"format":{"id":"number"},"params":{},"label":"唯一数","aggType":"cardinality"}],"buckets":[{"accessor":0,"format":{"id":"terms","params":{"id":"string","otherBucketLabel":"Other","missingBucketLabel":"Missing","parsedUrl":{"origin":"http://192.168.10.183:31560","pathname":"/app/kibana","basePath":""}}},"params":{},"label":"用户名","aggType":"terms"},{"accessor":1,"format":{"id":"terms","params":{"id":"string","otherBucketLabel":"Other","missingBucketLabel":"Missing","parsedUrl":{"origin":"http://192.168.10.183:31560","pathname":"/app/kibana","basePath":""}}},"params":{},"label":"页面路径","aggType":"terms"},{"accessor":2,"format":{"id":"terms","params":{"id":"number","otherBucketLabel":"Other","missingBucketLabel":"Missing","parsedUrl":{"origin":"http://192.168.10.183:31560","pathname":"/app/kibana","basePath":""}}},"params":{},"label":"transaction.duration.us: Descending","aggType":"terms"}]}},"aggs":[{"id":"2","enabled":true,"type":"terms","schema":"bucket","params":{"field":"transaction.custom.name","orderBy":"_key","order":"desc","size":30,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing","customLabel":"用户名"}},{"id":"4","enabled":true,"type":"terms","schema":"bucket","params":{"field":"transaction.page.url","orderBy":"5","order":"desc","size":5,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing","customLabel":"页面路径"}},{"id":"5","enabled":true,"type":"cardinality","schema":"metric","params":{"field":"trace.id","customLabel":"唯一数"}},{"id":"6","enabled":true,"type":"terms","schema":"bucket","params":{"field":"transaction.duration.us","orderBy":"5","order":"desc","size":5,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing","customLabel":"请求时间"}}]}""",
      "uiStateJSON" : """{"vis":{"params":{"sort":{"columnIndex":null,"direction":null}}}}""",
      "description" : "",
      "version" : 1,
      "kibanaSavedObjectMeta" : {
        "searchSourceJSON" : """{"query":{"query":"","language":"kuery"},"filter":[{"meta":{"alias":null,"negate":false,"disabled":false,"type":"phrase","key":"service.name","params":{"query":"psso-development"},"indexRefName":"kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index"},"query":{"match_phrase":{"service.name":"psso-development"}},"$state":{"store":"appState"}}],"indexRefName":"kibanaSavedObjectMeta.searchSourceJSON.index"}"""
      }
    },
    "type" : "visualization",
    "references" : [
      {
        "name" : "kibanaSavedObjectMeta.searchSourceJSON.index",
        "type" : "index-pattern",
        "id" : "af123ca0-82de-11ea-a587-0fefe6ee9b2d"
      },
      {
        "name" : "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index",
        "type" : "index-pattern",
        "id" : "af123ca0-82de-11ea-a587-0fefe6ee9b2d"
      }
    ],
    "migrationVersion" : {
      "visualization" : "7.4.2"
    },
    "updated_at" : "2020-05-20T06:37:25.944Z"
  }
}

joshdover · May 26, 2020, 6:11pm

@majagrubic This looks similar to other cases you've encountered. How could we confirm this is the same issue and/or fix the visualization?

majagrubic · May 26, 2020, 7:06pm

This problem could be related to a missing index-pattern. Unfortunately, we don't handle the error here very well and just swallow the exception, so there's not a lot of information into what's going around.
Could you please fetch all your index patterns and cross-reference them with the index patterns in the references part of your exported visualization? Basically, you're looking for just one index pattern: af123ca0-82de-11ea-a587-0fefe6ee9b2d

You can get all your index patterns by querying .kibana index from the dev console:

GET /.kibana/_search?q=type:"index-pattern"
{
"_source": false
}

wajika · May 27, 2020, 2:11am

I executed the query

GET /.kibana/_search?q=type:"index-pattern"
{
"_source": false
}

{
  "took" : 15,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 3.9512439,
    "hits" : [
      {
        "_index" : ".kibana_2",
        "_type" : "_doc",
        "_id" : "index-pattern:apm_static_index_pattern_id",
        "_score" : 3.9512439
      },
      {
        "_index" : ".kibana_2",
        "_type" : "_doc",
        "_id" : "index-pattern:9c49f420-90c9-11ea-be48-1f045d3a2df2",
        "_score" : 3.9512439
      }
    ]
  }
}

majagrubic · May 27, 2020, 9:35am

In the list of the index patterns, you have two index patterns:
apm_static_index_pattern_id and 9c49f420-90c9-11ea-be48-1f045d3a2df2.
Your visualization references the index pattern with the id: af123ca0-82de-11ea-a587-0fefe6ee9b2d, which is not present in your index patterns. I suspect you deleted that index pattern in the meantime. Unfortunately, in visualization references we only store index pattern ids, so I can't tell you what that index pattern was called.
The mitigation here is that in the saved objects UI you modify your visualization by deleting the content of the filter array from `searchSourceJSON. You can do that as follows:

go to the management console
select Saved Objects from the menu
find your visualization in the list
from the actions selector, select Inspect

Screenshot 2020-05-27 at 10.32.381542×320 32.3 KB

5.Find kibanaSavedObjectMeta.searchSourceJSON
6. Delete all the content of the filter array, but leave filter keyword and an empty array

7. Save

This will allow the visualization to load but without filters. You will need to recreate your filters.

In the meantime, I'm investigating with my colleagues if there was a migration gone wrong that caused this index pattern to be deleted and if we can prevent this from happening in the future.

wajika · May 28, 2020, 12:58am

@majagrubic
Thank you for your help.

I removed the content in the filter according to your method, but when I open the page again, it still shows that it is empty, which is a bit strange. And I noticed that the id of this index pattern is also different from the one in the picture above. Does this matter?

You are right, I did delete an index pattern before. Before this, I also have a question, can the visualize objects already designed change its source index?

majagrubic · May 28, 2020, 10:59am

On a second glance, I see that both your visualization AND your filter reference a missing index pattern. I don't think there's anything you can do here, except recreating a deleted index pattern.

And I noticed that the id of this index pattern is also different from the one in the picture above

I am not sure I understand what you mean here. It's the same missing index pattern id in both places.

can the visualize objects already designed change its source index?

It is only possible by modifying the index pattern id in the saved objects inspector directly, in a similar way you modified the filter array. I'm afraid we don't have a UI-friendly way of doing this. We have an open issue for this, so you can track progress there: Ability to change the index pattern on a visualization · Issue #17542 · elastic/kibana · GitHub

wajika · June 1, 2020, 12:32am

@majagrubic Thanks for your help, now I think the fastest solution is to delete visualization directly and redesign one

wajika · June 3, 2020, 1:54am

@majagrubic
Every Visualization has an index-pattern, but if the index-pattern is lost, all Visualization can't be used, which is very bad. (And it cannot be fixed, for example, change to the new index-pattern id)

#In another case, two elastic clusters have the same data but different index-pattern names. If I want to import the'Visualization Objects' of cluster A into cluster B, an error will occur. That's troublesome when migrating data(I remember encountering this kind of error in version 5.X)

majagrubic · June 5, 2020, 10:00am

Hm, I'm not sure how it would work if the visualization would just switch to a new index pattern. The data in the visualization is tied to your index pattern.

wajika · June 6, 2020, 12:07am

So does the elastic team plan to enhance the migration function of kibana Save Objects?

system · July 4, 2020, 12:07am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.