# Custom data format not working in elasticsearch

**URL:** <https://discuss.elastic.co/t/custom-data-format-not-working-in-elasticsearch/260530>\
**Category:** Elasticsearch\
**Created:** [January 8, 2021, 10:40am UTC](https://discuss.elastic.co/t/custom-data-format-not-working-in-elasticsearch/260530 "2021-01-08T10:40:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![gavin-orange](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gavin-orange/32/75287_2.png) [@gavin-orange](https://discuss.elastic.co/u/gavin-orange)\
**Post date:** [January 8, 2021, 10:40am UTC](https://discuss.elastic.co/t/custom-data-format-not-working-in-elasticsearch/260530/1 "2021-01-08T10:40:52Z")

</div>

I am using the filebeat to transfer nginx access log to elasticsearch with json format .  
access.log

```auto
{ "@timestamp": "05/Jan/2021:13:38:28 +0800", "pid": "52336", "connection": "217062444", "connection_requests": "18", "http_x_forwarded_for": "-", "upstream_addr": "10.7.4.146:8082", "upstream_connect_time": "0.036", "upstream_response_length": "242","upstream_response_time": "0.077","remote_addr": "10.8.4.253","remote_port": "52918","request": "POST /njdf_ajax?returnType=json&service=P1005000&encoderType=none HTTP/1.1", "request_length": "1117", "content_length": "492", "request_time": "0.077", "status": "200", "bytes_sent": "422", "body_bytes_sent": "253","http_user_agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36" }

```

The filebeat.yml as below:

```auto
    filebeat.inputs:
    - type: log
      paths:
    # - /usr/local/CCS-20201207161336.txt
    # - /usr/local/testlog4.log
    # - /usr/local/access.log
      - /root/json/json.log
      json.keys_under_root: true
      json.overwrite_keys: true
      json.add_error_keys: true
    #output.logstash:
    # hosts: ["0.0.0.0:5044"]

    setup.kibana:
      hosts: "10.239.113.77:5601"

    setup.ilm.enabled: false
    setup.template.overwrite: true
    setup.template.name: "nginx"
    setup.template.pattern: "nginx-*"

    output.elasticsearch:
      hosts: ["10.239.113.77:9200"]
      index: "nginx-sz-%{+yyyy-MM}"

    processors: 
      - add_locale: ~

```

But the elasticsearch can't parse @timestamp correctly , does anyone have similar issue ?  
I tried to use custom data format in template mappings , but still can't fix it .

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 11, 2021, 12:20am UTC](https://discuss.elastic.co/t/custom-data-format-not-working-in-elasticsearch/260530/2 "2021-01-11T00:20:57Z")

</div>

> [@gavin-orange](#):
>
> But the elasticsearch can't parse @timestamp correctly , does anyone have similar issue ?

Can you post the error that you are seeing?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2021, 12:21am UTC](https://discuss.elastic.co/t/custom-data-format-not-working-in-elasticsearch/260530/3 "2021-02-08T00:21:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.