Rename Field with []

[cris]

Hello I want to rename some field without affect the other data. I tried with.

input {
    elasticsearch {
           hosts => "localhost:9200"
    index => "services"
    size => 1
    docinfo => true
    }
}
filter {
    mutate {
	rename => { "[Europe][Login[1]]" => "[Europe][Login]" }
    }
}
output {
    elasticsearch {
    hosts => "localhost:9200"
    index => "services"
    }
}

But I have problems because Logstash showme that the "Login[1]" has a Invalid FieldReference so I thing is for the part of the "[1]".
How can I set this kind of field on logstash?

[Badger]

What does that field name look like if you go to the JSON tab after expanding an event in the Kibana Discover pane?

[Rios]

@cris

[Europe][Login[1]]

Try with: [Europe][Login][1]

Also you can rename field with reindex

[cris]

it looks like this:

 "Europe: {
      "Login[1]": {
        "status": "passed"
      },```

[cris]

I tried but I get the same error, that the name is wrong

[Badger]

I cannot test it since every method I have tried to create a field with that name results in an invalid field reference You could try

ruby {
    code => '
         login = event.remove("[Europe][Login[1]]")
         if login
             event.set("[Europe][login]", login)
         end
    '
}

[cris]

The same result I got a Invalid Reference

[Badger]

Then it may not be possible to do it in logstash itself. However, ingest pipelines have a rename processor that might be able to do it. I do not run Elasticsearch myself, so I cannot test it. Perhaps @stephenb can speak to that.You can set the pipeline option on an Elasticsearch output to say which ingest pipeline should be used.

[stephenb (Stephen Brown)]

@cris

[Europe][Login[1]]

@Badger

@stephenb can speak to that.You can set the pipeline option on an Elasticsearch output to say which ingest pipeline should be used.

I will take a look....

This works you would set this up (with your own pipeline name) then as @Badger indicated you can set the pipeline => "my-pipeline" setting in the logstash Elasticsearch output section

PUT _ingest/pipeline/discuss-test
{
  "processors": [
    {
      "rename": {
        "field": "test.field[1]",
        "target_field": "test.field"
      }
    }
  ]
}
  

POST _ingest/pipeline/discuss-test/_simulate
{
  "docs": [
    {
      "_source": {
        "test": {
          "field[1]": "test value"
        }
      }
    }
  ]
}

result

  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_id" : "_id",
        "_source" : {
          "test" : {
            "field" : "test value"
          }
        },
        "_ingest" : {
          "timestamp" : "2022-05-20T21:46:17.753381622Z"
        }
      }
    }
  ]
}