# Why my query does not work correctly after create new filed in logstash

**URL:** <https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504>\
**Category:** Logstash\
**Tags:** elastic-stack-monitoring\
**Created:** [November 20, 2023, 7:09am UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504 "2023-11-20T07:09:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 20, 2023, 7:09am UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/1 "2023-11-20T07:09:05Z")

</div>

This is my sample data:

```auto
134.255.248.30 - - [20/Nov/2023:09:04:57 +0330] "GET /serve/finnotech/validateDest?key=3f94393b5eaab29a167e5edc8a99860cba121550053bd113d291d71f146a7fa0&parameters=%7B%22dest%22:%22IR520190000000208813572000%22%7D HTTP/1.1" 200 73 "-" "axios/0.19.2" 86

```

and this is my Grok Pattern :

```auto
%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)

```

and du simulate it is showing follow structured data :

```auto
{
  "request": "/serve/finnotech/validateDest?key=3f94393b5eaab29a167e5edc8a99860cba121550053bd113d291d71f146a7fa0&parameters=%7B%22dest%22:%22IR520190000000208813572000%22%7D",
  "auth": "-",
  "ident": "-",
  "response": "200",
  "bytes": "73",
  "clientip": "134.255.248.30",
  "verb": "GET",
  "httpversion": "1.1",
  "timestamp": "20/Nov/2023:09:04:57 +0330"
}

```

So I have written follow pipeline

```auto
input{
   beats {

    port => 5071

}

}

filter{

  grok{
    match => { "message" => '%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)' }

}

}

output{

  stdout{}
  elasticsearch {
   index => newo
   hosts => ["https://IP:9200"]
   cacert => '/etc/logstash/certs/http_ca.crt'
   user => "elastic"
   password => "password"

}

}

```

now it has created index with the name newo and after I created dataview it is showing all fields but when I filter bytes \> 553

It is showing also less than 300

 ![digit](https://us1.discourse-cdn.com/elastic/original/3X/9/8/9887432522a23deceb3d464dba26467f43c91cc7.jpeg)

 ![digit2](https://us1.discourse-cdn.com/elastic/original/3X/7/0/703a015ff669a7e8178965b60b0fa417e9c1371d.jpeg)

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 20, 2023, 8:53am UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/2 "2023-11-20T08:53:36Z")

</div>

Please see attached pic it is showing type of bytes field is text . Although it is defines number in the pattern

 ![digit3](https://us1.discourse-cdn.com/elastic/original/3X/1/e/1ec829756b265230ebc8186743c1cf1bf531e7e6.jpeg)

---

<div class="post-metadata">

**Author:** ![sholzhauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sholzhauer/32/110282_2.png) [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)\
**Post date:** [November 20, 2023, 10:36am UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/3 "2023-11-20T10:36:53Z")

</div>

Hi, logstash does not influence how the mappings are done on elasticsearch side. That is where your solution lies.

To make your field work correctly as a number you need to update the mapping on your indices and optionally update the dataview to recoginize the number as bytes.

## Index mapping

You need to either update or create your index template and add a mapping field to it, based on your examples it should look like (addition):

```json
{
  "properties": {
    "bytes": {
      "type": "float"
    }
}
}

```

## Dataview

Once you have an index rollover (new index as mappings are only applied upon index creation) you can update the format in the dataview. When editing the dataview find your field and update the format to a byte:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/4/f4da79672001c873e0e0a7423c3c987f154e2a47.png)

## Summary

Currently the field is behaving as a text which means the `>` and `<` operators will not function as expected. Updating the mapping and optionally the format will allow you to correctly use your field

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 20, 2023, 11:31am UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/4 "2023-11-20T11:31:40Z")

</div>

> [@sholzhauer](#):
>
> ```auto
> {
> "properties": {
> "bytes": {
> "type": "float"
> }
> }
> }
> 
> ```

@sholzhauer

So thanks .

follow is part of my mapping for that index . My index name is neo

```auto
      },
      "bytes": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256

```

So Does it your mean have to update type of bytes manually ? How can do it ?

---

<div class="post-metadata">

**Author:** ![sholzhauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sholzhauer/32/110282_2.png) [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)\
**Post date:** [November 20, 2023, 11:34am UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/5 "2023-11-20T11:34:19Z")

</div>

You have to modify the `"type": "text"` into the `float` part

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 20, 2023, 12:16pm UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/6 "2023-11-20T12:16:23Z")

</div>

@sholzhauer

Would you please say how can modify it ? I could not find any options in the mapping to do it

Does it correct ?

```auto
PUT neo/_mapping 
{
  "properties": {
    "bytes": {
      "type": "float"
    }
}
}

```

I got this error :

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "mapper [bytes] cannot be changed from type [text] to [float]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "mapper [bytes] cannot be changed from type [text] to [float]"
  },
  "status": 400
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 18, 2023, 12:17pm UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504/7 "2023-12-18T12:17:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.