Kibana URL template with relative path

I have a question regarding Kibana URL templates for fields in the index.

  1. I have used relative path for the below URL template.
    1
  2. On local environment it is replaced with http://localhost:5601/s/reporting/app/kibana#/..... Which is expected.
  3. On my DEV and QA environment also it is replaced with http://localhost:5601 instead of http://dev.somedomain.com:5601 which is not expected.
  4. I have my DEV and QA environment Kibana on some domain (DNS name) and everything is working fine except this URL template.

Kibana Setup:
Version : 7.6.0

Kibana.yml

server.port: 5601
server.host: "dev.somedomain.com"

Please let me know what I am doing wrong, or point me in right direction.

Thanks

Hello @surajbarde

I've tested this on 7.6.0 and it works fine.

I've added a dummy relative path (e.g. /some/pathname/asset.png, with no prefix) and I get a link which points to https://mykibana.instance.com:5601/some/pathname/asset.png.

Did you export the index pattern from your DEV environment and you've imported it in your DEV/QA?

If it is the case, then the reason is the fact the URL scheme, hostname and port are stored in the index pattern.

Can you please try the following?

  1. Go to the Index Pattern in Kibana which contains your URL field (in Management / Index Patterns
  2. Click on the Index Pattern ID from the URL in the browser address bar (e.g.http://dev.somedomain.com:5601/app/kibana#/management/kibana/index_patterns/ff959d40-b880-11e8-a6d9-e546fe2bba5f?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-90d,to:now))&_a=(tab:indexedFields) ).
    The ID in my case is ff959d40-b880-11e8-a6d9-e546fe2bba5f
  3. Go to the Kibana Developer tool and execute
    GET .kibana/_doc/index-pattern:<ID>?filter_path=_source.index-pattern.fieldFormatMap
    
    Where <ID> is the ID we grabbed from the address.
    E.g. in my case is:
    GET .kibana/_doc/index-pattern:ff959d40-b880-11e8-a6d9-e546fe2bba5f?filter_path=_source.index-pattern.fieldFormatMap
    
  4. Share the response with us.

I've found out a problem with the URL formatter in Kibana with relative links, which has been fixed on 7.6.0. This is not exactly your problem though.

Your case might deserve a bug or an enhancement as it seems it has never been reported.

Thank you for your quick reply @Luca_Belluccini .

Did you export the index pattern from your DEV environment and you've imported it in your DEV/QA?

Yes I have developed in local and use the snapshot and restore to promote it to DEV and QA environment.

I run below in Kibana Dev Tools for my DEV/QA/local environment, for all index patterns which have URL field

GET .kibana_1/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc?filter_path=_source.index-pattern.fieldFormatMap

My all responses were empty :

{ }

Please let me know if there is anything else I can check.

Can you try the following?

GET .kibana_1/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc
GET .kibana/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc
GET .kibana*/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc

@Luca_Belluccini I have tried below queries they both give same results.

GET .kibana_1/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc
GET .kibana/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc
{ "_index" : ".kibana_1",
  "_type" : "_doc",
  "_id" : "index-pattern:448fb160-752c-11ea-874a-89c9b90911fc",
  "found" : false}

Also below gave me exception :There are no external requests known to support wildcards that don't support replacing their indices

  GET .kibana*/_doc/index-pattern:448fb160-752c-11ea-874a-89c9b90911fc

So I tried this in my DEV machine, as I have a space called reporting :

 GET .kibana_1/_doc/reporting:index-pattern:448fb160-752c-11ea-874a-89c9b90911fc?filter_path=_source.index-pattern.fieldFormatMap

Result is showing that origin value is coming multiple times as per the URL template use as localhost even if its DEV:

{  "_source" : {    "index-pattern" : {
"fieldFormatMap" : """{"clientinfoid":{"id":"number","params":
{"parsedUrl":{"origin":"http://localhost:5601"
    ,"pathname":"/s/reporting/app/kibana","basePath":"/s/reporting"},"pattern":"00.[000]"}},
    "corporationcode":{"id":"url","params":{"parsedUrl":
{"origin":"http://localhost:5601","pathname":"/s/reporting/app/kibana","basePath":"/s/reporting"},
        "urlTemplate":"/s/reporting/app/kibana#/dashboard/d4d630d0-6fa7-11ea-9103-d5c86fe87988?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-1y,to:now))&_a=(description:'',filters:!(),fullScreenMode:!f,options:(hidePanelTitles:!f,useMargins:!t),panels:!((embeddableConfig:(title:'SFTP%20Time%20Series'),gridData:(h:10,i:'9d28ce5e-0aa4-45f1-9879-1a8533c896eb',w:21,x:0,y:0),id:'052bbc90-6fa9-11ea-9103-d5c86fe87988',panelIndex:'9d28ce5e-0aa4-45f1-9879-1a8533c896eb',title:'SFTP%20Time%20Series',type:visualization,version:'7.6.0'),(embeddableConfig:(title:SFTP-Corporation),gridData:(h:10,i:'71ac2ec6-682f-4d0e-934c-467ad50cc38f',w:15,x:21,y:0),id:f18060a0-6fa9-11ea-9103-d5c86fe87988,panelIndex:'71ac2ec6-682f-4d0e-934c-467ad50cc38f',title:SFTP-Corporation,type:visualization,version:'7.6.0'),(embeddableConfig:(title:''),gridData:(h:19,i:'551865a8-2bce-428b-bedc-c6f5c831a7c1',w:48,x:0,y:10),id:'32771cf0-6fa7-11ea-9103-d5c86fe87988',panelIndex:'551865a8-2bce-428b-bedc-c6f5c831a7c1',type:search,version:'7.6.0')),query:(language:kuery,query:'corporationcode%20:%20{{value}}'),timeRestore:!t,title:'IMPORT%20Summary%20Dashboard',viewMode:view)","labelTemplate":"{{value}}","openLinkInCurrentTab":true}}}"""}  }}

So is there any way that this Origin value can be taken from Kibana.yml file, so that in each environment we don't have to change it manually.

Also I have noticed, that in DEV environment after promotion using snapshot and restore, if I manually change the URL template to give full path with http://dev.somedomain.com:5601 , the links in dashboards works as expected, but the query in Dev Tools still returns the same origin value as localhost.

Hello,
The problem is the parsedUrl":{"origin" is stored when you define the url field format in the index pattern in the original Kibana instance.

If DEV is the environment in which you restored the index pattern snapshotted from the local environment, then this is the reason you have {"origin":"http://localhost:5601".

I think one way to solve it would be to change the type of corporationcode to string in the index pattern via the user interface, save the index pattern.
Then set it back to URL with the relative path and save again.

Never edit the documents in .kibana indices manually.

1 Like

@Luca_Belluccini, The solution you gave worked.

I have changed the corporationcode type to string and saved it and then set it back to URL with the relative path and saved again -- and it worked :slight_smile:

This also changed parsedUrl":{"origin" properly.

But this is still manual step.

Is there any way I can use any script/curl command/api to change the parsedUrl":{"origin" property?
So that I can just run that through some script. This way whenever I do promotion to any environment using snapshot and restore, I can just run the script as a part of deployment to solve this issue.

Please let me know any way to automate it.

Thanks.

I see 2 options:

  • Exporting the index pattern, patching the file and importing it again
  • Performing a partial update via api

I am on mobile now, but the best is to open a bug/improvement issue in the Kibana repository.
I can do it if you wish.

I'll also suggest you how to patch the index pattern document.

1 Like

Thank you @Luca_Belluccini for quick response.

Can you open a bug/improvement issue in the Kibana repository, as I am not aware of how to do it.

Also The second option you gave :

Performing a partial update via api

Can you give me any sample for it, or any reference, like which api to use.

Exporting the index pattern, patching the file and importing it again

For the first option, If I use then do I have to do below steps for promotion:

  1. Use Snapshot and restore.
  2. Then import all indexes which have this URL field(after patching these files).
  3. Do I need to this for every environment or just from local to DEV, as the promotion sequence we have is local-->DEV-->QA-->UAT-->IST-->PRODUCTION.

Second option of a partial update via api looks easy for my condition.

Please let me know.

Thanks

One possible workaround might be:

  1. Get the index-pattern using its title or the ID
# Search by index-pattern title
GET .kibana/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "type": {
              "value": "index-pattern"
            }
          }
        },
        {
          "term": {
            "index-pattern.title": {
              "value": "kibana_sample_data_logs"
            }
          }
        }
      ]
    }
  }
}
# Or if you know the ID
GET .kibana/_doc/index-pattern:90943e30-9a47-11e8-b64d-95841ca0b247
  1. Get the content of the field fieldFormatMap.
    E.g.

    "fieldFormatMap": """{"url.keyword":{"id":"url","params":{"parsedUrl":{"origin":"https://localhost:9243","pathname":"/app/kibana","basePath":""},"urlTemplate":"/some/pathname/asset.png"}},"url":{"id":"url","params":{"parsedUrl":{"origin":"https://localhost:9243","pathname":"/app/kibana","basePath":""},"urlTemplate":"/some/pathname/asset.png"}}}"""
    
  2. Get the _id of the index-pattern.
    E.g. index-pattern:90943e30-9a47-11e8-b64d-95841ca0b247

  3. Modify the content of the fieldFormatMap to replace the URL of your local environment with the correct one

  4. Execute a partial update.
    E.g.

POST .kibana/_update/index-pattern:90943e30-9a47-11e8-b64d-95841ca0b247
{
  "doc": {
    "index-pattern": {
      "fieldFormatMap": """{"url.keyword":{"id":"url","params":{"parsedUrl":{"origin":"https://yourdevenv:5601","pathname":"/app/kibana","basePath":""},"urlTemplate":"/some/pathname/asset.png"}},"url":{"id":"url","params":{"parsedUrl":{"origin":"yourdevenv:5601","pathname":"/app/kibana","basePath":""},"urlTemplate":"/some/pathname/asset.png"}}}"""
    }
  }
}

All those requests can be done using bash, curl and sed and automated in a script.

I've opened the following for reference:

2 Likes

Thank you @Luca_Belluccini

This solution worked for me.
Also thank you for opening the issue for me, it will be great to have this feature by default in Kibana.

Regards,
Suraj

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.