How to rename json parsed field

Elastic Stack Logstash
1

Hi,

I am parsing json log file in Logstash. There is a field named @person.name. I tried to rename this field name before sending it to elasticsearch. I also tried to remove the field but I couldn't remove or delete that field because of that my data not getting indexed in Elasticsearch.

Error recorded in elasticsearch

> MapperParsingException[Field name [@person.name] cannot contain '.']
> 	at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:276)
> 	at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:221)
> 	at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parse(ObjectMapper.java:196)
> 	at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:308)
> 	at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:221)
> 	at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:138)
> 	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:119)
> 	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:100)
> 	at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:435)
> 	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:257)
> 	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230)
> 	at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:458)
> 	at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:762)
> 	at

My Logstash config

  input {
       beats {
    		port => 11153
    	}
    }
    filter 
    {
    	if [type] == "person_get" {
    	##Parsing JSON input to JSON Filter..
    		json {    		
    			source => "message"
    		}
    		mutate{
    		rename => { "@person.name" => "@person-name" }
    		remove_field => [ "@person.name"]
    		}
    	
    		fingerprint {
                                   source => ["ResponseTimestamp"]
                                   target => "fingerprint"
                                   key => "78787878"
                                   method => "SHA1"
                                   concatenate_sources => true
                    }
    	
    }
    }
    	
    output{
    if [type] == "person_get" { 
    	elasticsearch {		  
    		  index => "logstash-person_v1"
    		  hosts => ["xxx.xxx.xx:9200"]
    		  document_id => "%{fingerprint}" # !!! prevent duplication		  
    		}
    			stdout {
    				codec => rubydebug
    			}
    			}

    }
2

This should work. Have you tried using the de_dot filter?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.