Understanding index template behavior

Elastic Stack Elasticsearch
1

Hi

I would like to define a define an index template to cast an array (e.g. [-86.832375, 35.913284]) as geo_point. Defining a template and posting a record via Kibana dev tool works beautifully. However, doing the same steps but posting through HTTP route leads to parse_exception error.

Here is a reproducible example:

Step 1 Define the template within Kibana dev tool:

PUT _template/meetup_template
{
	"settings": {
		"number_of_shards": "1",
		"number_of_replicas": "1"
	},
	"index_patterns": "meetup*",
	"aliases": {},
	"mappings": {
		"properties": {
			"group_coordinate": {
				"type": "geo_point"
			},
			"venue_coordinate": {
				"type": "geo_point"
			}
		}
	}
}

Step 2 within Kibana dev tool add the first record and create and index

POST /meetupprocessedindex01/_doc
{"timestamp":"2019-08-18T19:54:59.953-07:00","rsvp_id":"1802216455","member_id":"281884161","member_name":"Jon Palmer","photo":"https://secure.meetupstatic.com/photos/member/4/9/f/6/thumb_288618934.jpeg","response":"no","mtime":"51600-05-16T05:36:27.000-07:00","lat":35.913284,"lon":-86.832375,"venue_id":"24413059","venue_name":"Outer Limits","guests":0,"group_city":"Nashville","group_country":"us","group_id":"25471951","group_lat":36.17,"group_lon":-86.72,"group_name":"Tennessee PFS and SFS","group_topics":[{"topic_name":"Roleplaying Games (RPGs)","urlkey":"roleplaying-games-rpgs"},{"topic_name":"Tabletop Role Playing and Board Games","urlkey":"tabletop-role-playing-and-board-games"},{"topic_name":"Pathfinder Roleplaying Game","urlkey":"pathfinder-roleplaying-game"},{"topic_name":"Pathfinder Society","urlkey":"pathfinder-society"},{"topic_name":"Starfinder Society","urlkey":"starfinder-society"},{"topic_name":"Pathfinder 2 Society","urlkey":"pathfinder-2-society"}],"group_urlname":"TN-PFS-SFS","group_coordinate":[-86.72,36.17],"venue_coordinate":[-86.832375,35.913284]}

All works well here!

Step 3 within Kibana dev tool delete the newly created index

DELETE meetupprocessedindex01

Step 4 open terminal and try to post the same record

curl -H "Content-Type: application/json" -X POST "http://localhost:9200/meetupprocessedindex01/default/" -d '{"timestamp":"2019-08-18T19:54:59.953-07:00","rsvp_id":"1802216455","member_id":"281884161","member_name":"Jon Palmer","photo":"https://secure.meetupstatic.com/photos/member/4/9/f/6/thumb_288618934.jpeg","response":"no","mtime":"51600-05-16T05:36:27.000-07:00","lat":35.913284,"lon":-86.832375,"venue_id":"24413059","venue_name":"Outer Limits","guests":0,"group_city":"Nashville","group_country":"us","group_id":"25471951","group_lat":36.17,"group_lon":-86.72,"group_name":"Tennessee PFS and SFS","group_topics":[{"topic_name":"Roleplaying Games (RPGs)","urlkey":"roleplaying-games-rpgs"},{"topic_name":"Tabletop Role Playing and Board Games","urlkey":"tabletop-role-playing-and-board-games"},{"topic_name":"Pathfinder Roleplaying Game","urlkey":"pathfinder-roleplaying-game"},{"topic_name":"Pathfinder Society","urlkey":"pathfinder-society"},{"topic_name":"Starfinder Society","urlkey":"starfinder-society"},{"topic_name":"Pathfinder 2 Society","urlkey":"pathfinder-2-society"}],"group_urlname":"TN-PFS-SFS","group_coordinate":[-86.72,36.17],"venue_coordinate":[-86.832375,35.913284]}'

Get the error

{"error":{"root_cause":[{"type":"parse_exception","reason":"geo_point expected"}],"type":"mapper_parsing_exception","reason":"failed to parse field [group_coordinate] of type [geo_point]","caused_by":{"type":"parse_exception","reason":"geo_point expected"}},"status":400}

Question: why steps 3 and 4 lead to error while 1 and 2 work? How should I be posting using the http rout? Any help will be much appreciated!

Thank you!

2

I think the error you're getting is a red herring. In your step 4, you are using the default document type. You should use the _doc endpoint.

3

Thank you very much for the response! I will replace "default" with " _doc" and report back.

4

That was it!!! Thank you very much Abdon!!

5

For context, I am using Apache nifi 1.9.2. and puthttpelasticsearchrecord processor. In the topic field I had chosen default which was working fine unless I had defined the array of [lat,lon] as geo_point. In that case, it was giving an error. I first tried to reproduce the http post (above) which recreated the same error I was seeing in nifi logs. The solution that Abdon suggested for the http post via curl worked beautifully with nifi too. Specifically, I changed the topic to _doc and then ingestion after applying the index template still worked well.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.