# Logstash with cloudfront
**URL:** https://discuss.elastic.co/t/logstash-with-cloudfront/197729
**Category:** Logstash
**Created:** [September 2, 2019, 6:54pm UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729 "2019-09-02T18:54:31Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@M\_H](https://discuss.elastic.co/u/M_H)
#### Post date: [September 2, 2019, 6:54pm UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729/1 "2019-09-02T18:54:31Z")
Hi,
i am facing issue while i am using following code in my logstash config file.
my other logs perfectly land into elasticsearch but somehow this filter didn't enter into elasticsearch.
If i used following code separately it works perfectly but when i merge that code with existing code it didn't work
someone please help me to sort out this issue it make my life hell.
Other code have some ELB entries if you want i will share it with masking entry.
//
if [type] == "yayacloudfront" {
grok {
match =\> { "message" =\> "%{DATE:date}\t%{TIME:time}\t%{DATA:x\_edge\_location}\t(?:%{NUMBER:sc\_bytes\_int}|-)\t%{IPORHOST:ipactual}\t%{WORD:cs\_method}\t%{HOSTNAME:cs\_host}\t%{NOTSPACE:cs\_uri\_stem}\t%{NUMBER:elb-status\_code}\t%{GREEDYDATA:referrer}\t%{GREEDYDATA:httpuseragent}\t%{GREEDYDATA:cs\_uri\_query}\t%{GREEDYDATA:cookies}\t%{WORD:x\_edge\_result\_type}\t%{NOTSPACE:x\_edge\_request\_id}\t%{HOSTNAME:domain\_name}\t%{URIPROTO:httpprotocol}\t%{INT:cs\_bytes\_int}\t%{NUMBER:time\_taken\_float}\t%{GREEDYDATA:x\_forwarded\_for}\t%{GREEDYDATA:ssl\_protocol}\t%{GREEDYDATA:ssl\_cipher}\t%{GREEDYDATA:x\_edge\_response\_result\_type}\t%{GREEDYDATA:cs\_protocol\_version}\t%{GREEDYDATA:fle\_status}\t%{GREEDYDATA:fle\_encrypted\_fields}" }
}
mutate {
add\_field =\> { "timestampnew" =\> "%{date} %{time}" }
}
date {
match =\> ["timestampnew", "YYYY-MM-dd'T'HH:mm:ssZ"]
locale =\> "en"
}
geoip { source =\> "ipactual"
target =\> "geoip"
}
useragent { source =\> "httpuseragent" }
mutate {
rename =\> { "name" =\> "agentname" }
rename =\> { "device" =\> "agentdevice" }
rename =\> { "os\_name" =\> "agentosname" }
}
mutate {
remove\_field =\> ["date", "time"]
}
}
output {
elasticsearch {
user =\> "elastic"
hosts =\> localhost
index =\> "all.com-%{+YYYY.MM.dd}"
template\_name =\> "[yaya.com](http://yaya.com)"
template =\> "/etc/logstash/conf.d/elb.json"
manage\_template =\> true
template\_overwrite =\> true
}
}
//
[WARN] 2019-09-02 23:40:38.031 [[main]\>worker0] elasticsearch - Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"yaya.com-2019.09.02", :\_type=\>"doc", :routing=\>nil}, #LogStash::Event:0x5d56ac68], :response=\>{"index"=\>{"\_index"=\>"yaya.com-2019.09.02", "\_type"=\>"doc", "\_id"=\>"xxxxxxxxxx", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field [timestampnew] of type [date]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "19-08-29 09:14:00" is malformed at "-08-29 09:14:00""}}}}}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 2, 2019, 7:09pm UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729/2 "2019-09-02T19:09:08Z")
> [@M\_H](#):
>
> failed to parse field [timestampnew] of type [date]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "19-08-29 09:14:00" is malformed at "-08-29 09:14:00"
If your index template does not specify a [format](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html) for the timestampnew field, then the default is
```
"strict_date_optional_time||epoch_millis"
```
A strict\_date would start with a 4 digit year, and epoch millis would typically be a 13-digit number, so either way it fails when it hits the first hyphen.
Specify a format in your template.
---
### Author:  [@M\_H](https://discuss.elastic.co/u/M_H)
#### Post date: [September 5, 2019, 7:07am UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729/3 "2019-09-05T07:07:36Z")
Thanks @Badger
I have tried to remove
date {
match =\> ["timestampnew", "YYYY-MM-dd'T'HH:mm:ssZ"]
locale =\> "en"
}
and its all things work perfectly and shown timestampnew field in kibana, but when i add date module it again stop parsing and didn't add into elasticsearch i tried to use rubydebug but it didn't map my new timestampnew date into @timestamp
i have tried all stuff but didn't work
#date {
# match =\> ["timestampnew" , "ISO8601", "yyyy-MM-dd HH:mm:ss.SSS"]
# target =\> "@timestamp"
#}
#date {
#match =\> ["timestampnew", "MMM dd HH:mm:ss", "MMM d HH:mm:ss"]
##0019-09-04T08:25:14.000Z
# match =\> ["timestampnew", "yyyy-MM-dd'T'HH:mm:ss'.'SSSZ"]
# target =\> "@timestamp"
#
# }
but nothing works.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 5, 2019, 11:52am UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729/4 "2019-09-05T11:52:37Z")
You need to change your elasticsearch configuration (the index template) not your logstash configuration.
---
### Author:  [@M\_H](https://discuss.elastic.co/u/M_H)
#### Post date: [September 5, 2019, 4:09pm UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729/5 "2019-09-05T16:09:21Z")
Thanks @Badger,
I have sort out this issue using this value
date {
match =\> ["timestampnew", "yy-MM-dd'T'HH:mm:ss"]
}
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 3, 2019, 4:09pm UTC](https://discuss.elastic.co/t/logstash-with-cloudfront/197729/6 "2019-10-03T16:09:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.