How to replacement - with 0 by gsub processor in ingest pipeline?

Elastic Stack Elasticsearch
1

Some of the value for field price is "-",so I want to replace it with 0 by gsub processor ,then I write the following code, however it doesn't work, how to write code?

{
"gsub": {
"field": "price",
"pattern": "-",
"replacement": 0
}

BTW, the ES version is 5.5, and I mapped the type of price to double.

The following error message got from the filebeat log file"{"type":"mapper_parsing_exception","reason":"failed to parse [price]","caused_by":{"type":"number_format_exception","reason":"For input string: "-""}}"

2

hey,

have you tried replacement: "0.0" plus a convert processor to change your fieldname?

Also please change your full pipeline, or at least provide a full reproducible example using the simulate pipeline API, this makes it a hundred times easier for others to check out your example.

--Alex

3

The full pipeline is

-- pipeline
PUT _ingest/pipeline/log-nginx-accesslog
{
"description": "Pipeline for parsing the Nginx access logs",
"processors": [{
"grok": {
"field": "message",
"patterns": [
"%{GREEDYDATA:x_forword} %{DATA:price} "%{URIPATHPARAM:uri}""
],
"ignore_missing": true
}
},{
"gsub": {
"field": "price",
"pattern": "-",
"replacement": 0
}
}, {
"remove": {
"field": "nginx.error.time"
}
}],
"on_failure" : [{
"set" : {
"field" : "error",
"value" : "{{ _ingest.on_failure_message }}"
}
}]
}

--template

PUT _template/template_log-platform
{
"template": "nginxlog*",
"settings": {
"number_of_shards": 6,
"number_of_replicas": 1
},
"mappings": {
"doc": {
"_source": {
"enabled": true
},
"properties": {
"bytes": {
"type": "integer"
},

     "price": {
      "type": "double"
    }
    ,
     "request_time": {
      "type": "double"
    }
    
  }
}

}
}

4

I tried your method, however it still doesn't work.

5

A full reproducible example would be great. Maybe you can use the Ingest Simulate API, which can contain the pipeline and sample documents in a single API call. That would help a lot!

--Alex

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.