Format for timestamps

[georgemichael]

I am working on some logging software and was wondering what format kibana indexes timestamps. I was thinking of using something similar to log.go for formatting log header to buffer as seen here:

func (l *Logger) formatHeader(buf *[]byte, t time.Time, file string, line int) {
	if l.flag&Lmsgprefix == 0 {
		*buf = append(*buf, l.prefix...)
	}
	if l.flag&(Ldate|Ltime|Lmicroseconds) != 0 {
		if l.flag&LUTC != 0 {
			t = t.UTC()
		}
		if l.flag&Ldate != 0 {
			year, month, day := t.Date()
			itoa(buf, year, 4)
			*buf = append(*buf, '/')
			itoa(buf, int(month), 2)
			*buf = append(*buf, '/')
			itoa(buf, day, 2)
			*buf = append(*buf, ' ')
		}
		if l.flag&(Ltime|Lmicroseconds) != 0 {
			hour, min, sec := t.Clock()
			itoa(buf, hour, 2)
			*buf = append(*buf, ':')
			itoa(buf, min, 2)
			*buf = append(*buf, ':')
			itoa(buf, sec, 2)
			if l.flag&Lmicroseconds != 0 {
				*buf = append(*buf, '.')
				itoa(buf, t.Nanosecond()/1e3, 6)
			}
			*buf = append(*buf, ' ')
		}
	}

Any info would be appreciated!

[lukeelmers]

There are two separate concepts to be aware of here: The first is how Kibana displays dates, and the second is how Elasticsearch indexes them.

Kibana currently uses Moment JS for parsing dates that are displayed. Moment recognizes ISO 8601 and RFC 2822 date formats, with a fallback to the native JavaScript Date.

Elasticsearch date fields are internally converted to UTC and stored as a long representing milliseconds-since-epoch, but can be retrieved with a number of built-in formats. For example, the Discover app in Kibana will try to request any date fields in date_time format to get back the ISO 8601 date string, which Moment JS then converts to whatever dateFormat is specified in Kibana's advanced settings.

While there are no hard and fast rules here, in general I would recommend following the Elastic Common Schema, which suggests a @timestamp field that is an ISO 8601 date string.