Datatypes are not updated on Kibana


(shyamala) #1

Hi team,

I am new to Elasticsearch. I am sending the Purelytics data from Dynatrace to ES and can see the data on Kibana. we have observed that data.startTime and data.endTime of JSON documents are mapped to Number data type. We are not able to modify the existing index data types. So, I have created the new index and modified the mappings of data.startTime and endTime to Date format, able to see that change in ES but not reflected on Kibana webpage. So could you please suggest what would be the better option to do so.

Thanks
Shyamala


(Matt Bargar) #2

Kibana caches field info from your index pattern. You can manually refresh it by clicking the refresh button on the index pattern management page:


(shyamala) #3

Thanks for the Response Bargs!!!

even after clicking on refresh button also , having the same issue


(Matt Bargar) #4

Hmm, so you deleted the old indices with the incorrect mappings and the index pattern is pointing at the new indices with the correct mappings? What version of Kibana and ES are you using?


(shyamala) #5

I didn't deleted the old version but reindexed the data to new index.I am using 5.4.1 of both Kibana and ES


(Matt Bargar) #6

Does the existing kibana index pattern match both the old indices and new indices you reindexed into? It sounds like you might need to create a new index pattern that only matches the new indices.


(shyamala) #7

I have created new index with the same mapping as old one but the difference is i have changed the data.starttime and data.endtime datatypes to date


(Matt Bargar) #8

Right. Does your index-pattern in Kibana match the names of the old indices, the new indices, or both? If the index-pattern in Kibana still matches the names of the old indices, it's still going to pick up field info from their mappings.


(shyamala) #9

yes names of the both indices is same... so can i delete the old one ?Here i am directly pulling the data from Dynatrace server


(shyamala) #10

Bargs,

Is there any way to update the data types while re indexing the data?


(Matt Bargar) #11

Yes, if you no longer need the old indices it would probably be best to delete them.

Is there any way to update the data types while re indexing the data?

Generally what you'll want to do is define an index template that will match the new index name. The index template will allow you to define the correct mappings ahead of time.


(shyamala) #12

Thanks for Reply Bargs!!!!!

I am pulling the data directly from Dynatrace server, backend itself only creating the mapping..
so how can i edit the edit for this scenario.

Thanks
Shyamala


(Matt Bargar) #13

I'm sorry, I don't think I understand your question. Can you elaborate?


(shyamala) #14

Sure Bargs,

I am taking the data from another server so whenever i am creating the index it automatically creating the mapping backend itself.So i am asking how can i edit such mapping?

Thanks
Shyamala


(Matt Bargar) #15

Do your indices follow a naming convention? If you create an index template before the concrete indices are created, it will match new index names based on the pattern you provide and apply its settings to the new index. So for instance if I defined an index template with the pattern logstash-* and then created a new index called logstash-2017-07-18 the template would match and apply its settings to this new index. Templates only apply at index creation time and mappings cannot be edited after index creation, so for existing indices you would need to re-index after creating this index template.


(shyamala) #16

HI Bargs,

I have deleted all existing indices and created index template and one new index but the data is mapping to old index.how can i delete that old index permanently.


(shyamala) #17

how can i delete index permanently on kibana?


(Matt Bargar) #18

Deleting the index via the ES API should be sufficient https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html

If the old mappings are still applied to the new indices, perhaps there's an old template still around. You can list all of the existing index templates https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html

Finally, if you mean that the old mappings are still showing up in Kibana, you'll either need to refresh the field list or recreate the index pattern in Kibana after you've got the mappings sorted out in Elasticsearch.


(shyamala) #19

Thanks Bargs!!!! Issue got resolved and fields are upadated


(system) #20

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.