Rename Field with []

Elastic Stack Logstash
1

Hello I want to rename some field without affect the other data. I tried with.

input {
    elasticsearch {
           hosts => "localhost:9200"
    index => "services"
    size => 1
    docinfo => true
    }
}
filter {
    mutate {
	rename => { "[Europe][Login[1]]" => "[Europe][Login]" }
    }
}
output {
    elasticsearch {
    hosts => "localhost:9200"
    index => "services"
    }
}

But I have problems because Logstash showme that the "Login[1]" has a Invalid FieldReference so I thing is for the part of the "[1]".
How can I set this kind of field on logstash?

2

What does that field name look like if you go to the JSON tab after expanding an event in the Kibana Discover pane?

3

Try with: [Europe][Login][1]

Also you can rename field with reindex

4

it looks like this:

 "Europe: {
      "Login[1]": {
        "status": "passed"
      },```
5

I tried but I get the same error, that the name is wrong

6

I cannot test it since every method I have tried to create a field with that name results in an invalid field reference :smiley: You could try

ruby {
    code => '
         login = event.remove("[Europe][Login[1]]")
         if login
             event.set("[Europe][login]", login)
         end
    '
}
7

The same result I got a Invalid Reference

8

Then it may not be possible to do it in logstash itself. However, ingest pipelines have a rename processor that might be able to do it. I do not run Elasticsearch myself, so I cannot test it. Perhaps @stephenb can speak to that.You can set the pipeline option on an Elasticsearch output to say which ingest pipeline should be used.

9

I will take a look....

This works you would set this up (with your own pipeline name) then as @Badger indicated you can set the pipeline => "my-pipeline" setting in the logstash Elasticsearch output section

PUT _ingest/pipeline/discuss-test
{
  "processors": [
    {
      "rename": {
        "field": "test.field[1]",
        "target_field": "test.field"
      }
    }
  ]
}
  

POST _ingest/pipeline/discuss-test/_simulate
{
  "docs": [
    {
      "_source": {
        "test": {
          "field[1]": "test value"
        }
      }
    }
  ]
}

result

  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_id" : "_id",
        "_source" : {
          "test" : {
            "field" : "test value"
          }
        },
        "_ingest" : {
          "timestamp" : "2022-05-20T21:46:17.753381622Z"
        }
      }
    }
  ]
}
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.