Custom Filestream Integration - Adding mappings

Elastic Stack Elastic Agent
5

Hi @erikg

Here is how it works.... and agree it is inconsistent between the different packages... the azure input sets up template scaffolding the filestream does not I am no sure why.

That said currently Custom File Stream Input does not generate a template specific to the data_stream.dataset that you define and thus just uses the default logs-*

You can fix / address this in about 5 minutes .... but if you want to do it really quick here it is I keep this around handy.

Assumes your datastream is logs-mycustom.filestream-*

  • Put this in
  • Rollover the data stream
  • It will show up in the UI
  • It will use all the logs-* goodness and add your own.
  • Its basically a clone then edit of the logs template
PUT _index_template/logs-mycustom.filestream
{
  "version": 1,
  "priority": 200,  << MUST BE GREATER THAN 100 so it takes precendence 
  "template": {
    "mappings": {
      "_source": {
        "mode": "synthetic" 
      }
    },
    "settings": {
      "index": {
        "mode": "standard" <<< I set to logsdb if you want 
      }
    }
  },
  "index_patterns": [
    "logs-mycustom.filestream-*" <<< Your datastream name 
  ],
  "data_stream": {
    "hidden": false,
    "allow_custom_routing": false
  },
  "composed_of": [
    "logs@mappings",
    "logs@settings",
    "ecs@mappings",
    "logs-mycustom.filestream@custom",  << ADD THIS
    "logs@custom"
  ],
  "ignore_missing_component_templates": [
    "logs-mycustom.filestream@custom", << ADD THIS
    "logs@custom"
  ],
  "allow_auto_create": true,
  "_meta": {
    "description": "custom filestream logs template",
    "managed": false <<< TURN OF MANAGED 
  },
  "deprecated": false
}

You can then add your custom mappings and settings to
logs-mycustom.filestream@custom or
liga@custom

1 Like