Data isn't indexed by elasticsearch with custom template

Elastic Stack Elasticsearch
1

So i use CURL to send my template to ElasticSearch.

When I then run Logstash (to reindex my data) and view my index there is no data for each record (40,000+) other than the default 4 fields. _id, _index, _type and _score.

If i view the index data to see the mappings, i can see all of the fields that Logstash sends and the fields from my template.

When i delete the template & index and re-run Logstash my data is displayed perfectly fine but the field types that i set in my template are wrong - as expected.

Logstash version is 5.2.4 and ES version is 5.2.0.

Here is the template i am using.

Thanks in advance.

curl - XPUT 'localhost:9200/_template/template?pretty' - H 'Content-Type: application/json' - d '{
	"template": "logstash*",
	"settings": {
		"number_of_shards": 5
	},
	"mappings": {
		"apachelog": {
			"_source": {
				"enabled": false
			},
			"properties": {
				"geoip": {
					"type": "nested",
					"dynamic": true,
					"properties": {
						"ip": {
							"type": "ip"
						},
						"location": {
							"type": "geo_point"
						},
						"latitude": {
							"type": "half_float"
						},
						"longitude": {
							"type": "half_float"
						}
					}
				}
			}
		}
	}
}
'
2

Is this not expected as you have disabled source?

3

I thought (from the documentation) that source is referring to the 'source' of the data. I.e. the file path to the log file.
Is this not correct?

4

No, it refers to the JSON document you are indexing. It indexes the fields, so you can still aggregate across it, but does not store the JSON, which prevents you from viewing the data.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.