Weird Bug, Field name is blocked, cant use the same name of field it in other pipelines

[ElasticLiver (Dense Man)]

Hi, I have a pipeline that stores the fields memory_memused_per and memory_swapused_per in an index , in another pipeline that stores data in another index I have tried to use the same name but nothing is indexed. after hours of debugging I just realize if I change the name to mem_memused_per and mem_swapused_per it works without problems, this is the ruby code:

        ruby {
          code => '
            log = event.get("status")
            mem_memused_per = log.scan(/Physical Memory Used\s*%=\s*(-?[\d.]+)/).flatten.first
            mem_swapused_per = log.scan(/Memoria Swap\s*%=\s*(-?[\d.]+)/).flatten.first

            event.set("memory_memused_per", mem_memused_per)
            event.set("memory_swapused_per", mem_swapused_per)

          '
        }

        mutate {
            convert => { "memory_memused_per"  => "float" }
            convert => { "memory_swapused_per" => "float" }
      }

so if i just change the name of the field, and the mutate convert

            event.set("mem_memused_per", mem_memused_per)
            event.set("mem_swapused_per", mem_swapused_per)

the documents are indexed.

Any known bugs related to this?

Logstash 7.5

[stephenb (Stephen Brown)]

Have you looked at the mappings?

Perhaps your original names have an incorrect mapping so the fields can not be stored

And when you changed the name you got a correct mapping and thus the can be stored

[ElasticLiver (Dense Man)]

Hi, thanks for you reply, so i create a new index with no mapping and add these lines to ruby so I can index fields that start with memory an the fields that start with mem, both from the same source:

           memory_memused_per = log.scan(/Physical Memory Used\s*%=\s*(-?[\d.]+)/).flatten.first
           memory_swapused_per = log.scan(/Memoria Swap\s*%=\s*(-?[\d.]+)/).flatten.first


            event.set("memory_memused_per", memory_memused_per)
            event.set("memory_swapused_per", memory_swapused_per)

            event.set("mem_memused_per", memory_memused_per)
            event.set("mem_swapused_per", memory_swapused_per)

the mapping response in devtools:

And then in Discover only the field mem_memused_per appears with data, both fields have the same source, and mapping

[stephenb (Stephen Brown)]

Look at the source json in discover....

Is elasticsearch 7.5 as well?... could be a bug... but seems awfully "weird" as you say
7.5 is pretty old... any chance run a quick new elasticsearch?

[ElasticLiver (Dense Man)]

Oh...I feel very dumb right now, In the same pipeline was another ruby filter using the same field and parsing another type of log , thanks for your anwers and sorry to waste your time brother.

[stephenb (Stephen Brown)]

No worries... We've been running into a lot of these things. It's good to just have someone help you find them.

No waste of time. You're a good member of the community!