# What datatype to select

**URL:** <https://discuss.elastic.co/t/what-datatype-to-select/321847>\
**Category:** Elasticsearch\
**Created:** [December 22, 2022, 2:45pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847 "2022-12-22T14:45:02Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hasan](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@Hasan](https://discuss.elastic.co/u/Hasan)\
**Post date:** [December 22, 2022, 2:45pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/1 "2022-12-22T14:45:02Z")

</div>

Hello, while importing my dataset I have this column which consist of **numerical** and **string** values. When setting the datatype in `mappings` to `keyword` its not showing the numerical values and by changing it to `double` its showing only numerical values. Please help figure out what `datatype` to choose to query all the values of a column. My kibana version is 8.5. Please help me resolve the issue.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 26, 2022, 6:59pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/2 "2022-12-26T18:59:45Z")

</div>

Hi @Hasan

We need more details...

How are you ingesting the data?

Please show sample documents/ data samples.

> [@Hasan](#):
>
> its not showing the numerical values and by changing it

What exactly do you mean "its not showing"... Not showing where / how?

`keyword` will accept text characters or numerical

---

<div class="post-metadata">

**Author:** ![Hasan](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@Hasan](https://discuss.elastic.co/u/Hasan)\
**Post date:** [December 27, 2022, 7:26am UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/3 "2022-12-27T07:26:20Z")

</div>

While adding the `control` to my dashboard that particular `control` is not showing all the values of that field.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/0/601c3c035fcfa76f99616e930a568e88c3477630.png)

Here its only showing **66** available items. However, it consist of around **2000 values**.  
I have kept the type `keyword` in the` mappings` as it contains both numerical and character type values.

Also, I'm uploading the data through csv file and have changed the type of this column to text already.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 27, 2022, 4:47pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/4 "2022-12-27T16:47:45Z")

</div>

Hi @Hasan

I need more details.. .there are a number of reasons...

> [@Hasan](#):
>
> I have kept the type `keyword` in the` mappings` as it contains both numerical and character type values.

Can you show me the mapping in Kibana Dev Tools

`GET /my-index-name`

Also Can you show me a couple sample lines of the input CSV a couple with the number a couple with the text .

Is there a `@timestamp` field? If so the control will only show options that are available during the time shown in the Time Picker

---

<div class="post-metadata">

**Author:** ![Hasan](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@Hasan](https://discuss.elastic.co/u/Hasan)\
**Post date:** [December 28, 2022, 8:14am UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/5 "2022-12-28T08:14:03Z")

</div>

Here is the` mapping` for my current index.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e31f178c6700e5ad363e9c97ab24626182c28ca.png)

Yes `@timestamp` field is required.

CSV file column with the number and text values.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e55f1b30467216bc103665dba9b5e97e43f94fc6.png)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 28, 2022, 4:35pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/6 "2022-12-28T16:35:28Z")

</div>

Hi @Hasan Thanks for the information.

Next time please do not post images of text. They can be hard to read and impossible for us to help/ debug. For instance, I could not cut and paste and try that mapping myself. Next Time please use text and format it.

I forgot to ask, What version of the stack are you using?

> [@stephenb](#):
>
> Is there a `@timestamp` field? If so the control will only show options that are available during the time shown in the Time Picker

I notice the selections that you show do not look like the selections in the CSV are you sure you selected the correct field?

You did not answer the question about the time picker? Or is there any other filter applied to the dashboard that's limiting the selection? KQL Filter, Filter Panel etc...

 ![Screen Shot 2022-12-28 at 8.34.19 AM](https://us1.discourse-cdn.com/elastic/original/3X/c/2/c283f50b8c422a40f033dfe957a7c398009e53ff.png)  
What happens if you tried to type in a value that you know into the control box that you do not see in the list? Does it show up?

Can you run this? It will show the number of distinct values

```auto
GET my-index/_search
{
  "size": 0, 
  "aggs": {
    "options": {
      "cardinality": {
        "field": "ITEM_CODE"
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Hasan](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@Hasan](https://discuss.elastic.co/u/Hasan)\
**Post date:** [January 1, 2023, 9:54am UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/7 "2023-01-01T09:54:00Z")

</div>

Thanks @stephenb for your timely response. Yes I am using the `@timestamp` field and my records are being filtered according to the filtered time slot.

However, The issue got resolved to much extent when I changed the format of the file while uploading. Earlier I was using CSV UTF-8 format and changing it to only CSV Format made it successfully read most of the records in `controls.`

The only problem now is that I want to make my controls search case sensitive and I have achieved it using this

{  
"settings": {  
"index": {  
"number\_of\_shards": 1,  
"number\_of\_replicas": 1,  
"analysis": {  
"normalizer": {  
"normalizer\_case\_insensitive": {  
"type": "custom",  
"filter": ["lowercase"]  
}  
}  
}  
}  
},

But now its only showing the lower case values not upper case ones. What if I remove `"filter": ["lowercase"]` will solve my problem?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [January 1, 2023, 3:54pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/8 "2023-01-01T15:54:24Z")

</div>

Yes get rid of that, type `keyword` is case sensitive by default.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 29, 2023, 3:55pm UTC](https://discuss.elastic.co/t/what-datatype-to-select/321847/9 "2023-01-29T15:55:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
