How to index newly parsed fields to be searchable - My Dynamic Template

Hello, I'm trying to index custom fields that are parsed from s3 json log, it's not seeming to work, I have these processors:

filebeat.inputs:
- type: aws-s3
  queue_url: https://sqs.XXX>XXXX.com/XXXX/XXXXX
  access_key_id: 'XXXX-XXX'
  secret_access_key: 'XXXXXXX-XXXXXXX'
  file_selectors:
  - regex: '^AWSLogs/\d+/CloudTrail/'
    expand_event_list_from_field: 'Records'
  processors:
    - decode_json_fields:
        fields: ["message"]
        process_array: true
        max_depth: 1
        target: "s3.access"
        overwrite_keys: false

And so end up with ~15 fields all prepended with "s3.access.*" but when I check in Kibana discover they're not index and I can't search based on them, how do I get this working? I already tried a dynamic template:

  {
    "s3.access.*": {
      "path_match": "s3.access.*",
      "mapping": {
        "type": "text"
      },
      "match_mapping_type": "*"
    }
  }

but that returned the error - "Can't get text on a START_OBJECT at 1:995"
I think I'm close but any help on this would be great, thank you.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.