# Custom filebeat template for JSON log lines

**URL:** https://discuss.elastic.co/t/custom-filebeat-template-for-json-log-lines/114761
**Category:** Beats
**Tags:** filebeat
**Created:** [January 9, 2018, 10:04pm UTC](https://discuss.elastic.co/t/custom-filebeat-template-for-json-log-lines/114761 "2018-01-09T22:04:15Z")
**Posts on this page:** 1
**Showing post:** 5

<div class="post-metadata">

### Author: ![vopsec](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vopsec](https://discuss.elastic.co/u/vopsec)
#### Post date: [January 11, 2018, 5:46pm UTC](https://discuss.elastic.co/t/custom-filebeat-template-for-json-log-lines/114761/5 "2018-01-11T17:46:16Z")

</div>

It's funny: I did try to export the template using `filebeat export template > filebeat.template.json`. I changed the appropriate field types then I imported it to a new index and loaded a copy of my JSON log data. The data was still being written as a "text" type instead of an "ip" type. I wasn't sure why.

In the end, I found a way to make it work. Here is my procedure in case someone else runs into this issue before there is any official documentation on the subject:

1. For filebeat.yml, set up the JSON fields but do not put the JSON event under root:

```auto
filebeat.prospectors:

- type: log

  enabled: true

  paths:
    - /var/log/elk-dev/*
  json.add_error_key: true

{...}

setup.template.name: "elk-dev"
setup.template.pattern: "elk-dev-*"

output.elasticsearch:
  - index: "elk-dev"

```

1. Create a standard JSON template. You can base it off of `filebeat export template`. Here's an example of what it would look like:

```auto
{  
   "index_patterns":[  
      "elk-dev*"
   ],
   "mappings":{  
      "doc":{  
         "properties":{  
            "@timestamp":{ ... },
            "beat":{ ... },
            # BELOW IS WHERE YOU WILL DEFINE YOUR JSON FIELDS AND TYPES
            "json":{ 
                 "properties":{
                       "source-ip":{
                             "type":"ip",
                             "keyword":{
                                   "type": "keyword",
                                   "ignore_above": 256
                             }
                         },
                       ...
                       ...etc...
                       ...
             },
            "error":{ ... },
            "prospector":{ ... }
         }
      }
   }
}

```

1. Use the curl API to put this JSON template into a new index:  
`curl -H "Content-Type: application/json" -PUT 'http://$ELASTICSEARCH_IP:9200/_template/elk-dev?pretty' -d@filebeat.template.json`

2. Make sure that your filebeat.yml is outputting to elasticsearch and that you've set the appropriate index (in this example, it's "elk-dev" as shown in the last line in step 1)

3. You can now ship data to this new index by doing `sudo service filebeat restart`.

4. In my scenario, I had a timestamp in my JSON data which is different from `@timestamp` which filebeat adds automatically. When creating my index, I chose to use my own timestamp instead of filebeat's. _I'm not sure this is required in some way to make the types work_.

* * *

For me, the methods and suggestions I've found online did not work or I could not get them to work.  
I could not find a way to modify fields.yml in a way that preserved custom field types although this seems like the most direct way of doing this.  
Until then, this is my janky fix.

---

_[View the full topic](https://discuss.elastic.co/t/custom-filebeat-template-for-json-log-lines/114761)._