Hi,
Im having a issue while indexing via filebeat httpjson to elastic
I have this configuration
filebeat.inputs:
- type: httpjson
enabled: true
config_version: 2
fields_under_root: true
json.keys_under_root: true
json.overwrite_keys: true
json.add_error_key: true
processors:
- decode_json_fields:
fields: ["message"]
interval: 5m
request.url: https://endpoint/slowQueries
request.proxy_url: http://proxy:8080
request.transforms:
- set:
target: header.api-key
value: 'key'
output.elasticsearch:
enabled: true
hosts: ["elastic:9200"]
index: "mongoatlas-%{+yyyy.MM.dd}"
username: "elastic"
password: "xxxxxx"
logging.level: info
logging.selectors: ["*"]
setup.template.name: "mongoatlas"
setup.template.pattern: "mongoatlas-*"
The result from the endpoint is a valid JSON
{
"process": "atlas-k2w8aa-shard-00-02.l48fp.mongodb.net:27017",
"queries": [ ]
}
This is the error that Im getting at ES
2022-03-08T17:06:47.236Z WARN [Elasticsearch] Elasticsearch/client.go:405 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xc08201298c05b3cd, ext:2576639853, loc:(*time.Location)(0x55f47600d320)}, Meta:null, Fields:{"agent":{"ephemeral_id":"ab754087-f546-47df-b0bc-f4747179be3e","hostname":"filebeat-mongodb-7c8dc6b4bc-gpgkj","id":"af2be906-c358-4bad-ab25-e18208150fbe","name":"filebeat-mongodb-7c8dc6b4bc-gpgkj","type":"filebeat","version":"7.14.1"},"ecs":{"version":"1.10.0"},"event":{"created":"2022-03-08T17:06:46.201Z"},"host":{"name":"filebeat-mongodb-7c8dc6b4bc-gpgkj"},"input":{"type":"httpjson"},"message":{"process":"atlas-k2w8aa-shard-00-02.l48fp.mongodb.net:27017","queries":}}, Private:interface {}(nil), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [message] of type [text] in document with id '-qt_an8BK1CtvcLQYWlD'. Preview of field's value: '{process:27017=atlas-k2w8aa-shard-00-02.l48fp.mongodb.net, queries=}'","caused_by":{"type":"illegal_state_exception","reason":"Can't get text on a START_OBJECT at 1:361"}}
Help would be appreciated