I created a gist here
It could be something as trivial as the way I crafted the mappings. I've been using ES 6 for years, but migrating to 8+ brings on new adventures.
The gist gives the json object I am passing in. It's a generic test to uncover the mysteries of the new ES platform.
Side note, the version of mappings I pasted in is missing a closing curly bracket at the end.
I failed to show the code:
Reader r = new StringReader(node.toJSONString());
try {
IndexRequest<JsonData> request =
IndexRequest.of(b -> b
.id(id)
.index(index)
.withJson(r));
IndexResponse indexResponse = client.index(request);
result.setResultObject(indexResponse.result().toString());
} catch (Exception e) {
where node is a JSONObject - we grab its JSON string value, pass that to a reader, then into the IndexRequest
Like I suggested in the other thread, you should try to go to Kibana Dev tools post the mapping and then try to post one of your documents and then you will get the errors or not.
It looks like your field is not a simple field that it is some sub-object with an equal sign.
That doesn't look like a normal field value to me.. like your JSON is invalid.
{en=Funky label}
That would be precisely what I want to do but, for the moment, Kibana is misbehaving
[2022-06-26T07:16:06.545-07:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. self signed certificate in certificate chain
It stops there on boot; when I go to the page it says Kibana is not ready yet.
Busy trying to figure that one out.
Note: this is Kibana 8.2.2 running against ES 8.2.3. In the past minor differences didn't affect the performance.
How did You install?
If you used the enrollment token that should be set..
BUT I think you reinstalled elasticsearch from 8.2.2 to 8.2.3 ... I think you did a fresh install, so you'll need to re-enroll Kibana.
to do this.
Stop Kibana
Clean out the autogenerate lines at the bottom all these.
# This section was automatically generated during setup.
elasticsearch.hosts: ['https://192.168.2.107:9200']
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE2NTYyNTY4MDkzMTU6UGFMckRlQzFSRkNfNkVuaHM3VkxLdw
elasticsearch.ssl.certificateAuthorities: [/Users/sbrown/workspace/elastic-install/8.2.3/kibana-8.2.3/data/ca_1656256809504.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://192.168.2.107:9200'], ca_trusted_fingerprint: ec6a75c66e0952b695782dbea399b26cdea2714172e0655b48ec4805321c1d45}]
go to elasticsearch directory and create a new kibana enrollment token
./bin/elasticsearch-create-enrollment-token -s kibana
warning: ignoring JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-11.0.13.jdk/Contents/Home; using bundled JDK
eyJ2ZXIiOiI4LjIuMyIsImFkciI6WyIxOTIuMTY4LjIuMTA3OjkyMDAiXSwiZmdyIjoiZWM2YTc1YzY2ZTA5NTJiNjk1NzgyZGJlYTM5OWIyNmNkZWEyNzE0MTcyZTA2NTViNDhlYzQ4MDUzMjFjMWQ0NSIsImtleSI6Il9vV1lvSUVCN1dqdWN6aHlpNTExOnhWbGVGTlNyUVkyRzFtNnJtdUJmVXcifQ==
hyperion:elasticsearch-8.2.3 sbrown$
Start Kibana
Click on the link it provides in the console
Paste in new enrollment token into the browser and you should be good.
Thanks!
While I am exploring this issue with kabana, the suggestion is that either my mappings are wrong, or that this JSON is bad
{
"lox": "1656282032356",
"details": {
"en": "Funky label"
},
"label": {
"en": "Funky label"
}
}
Frankly, there's nothing wrong with that JSON; my game has been to design mappings which can handle it. I've had that JSON running on ES 6 series, but everything has changed.
May I ask if some one could kindly suggest what the mappings should look like for structures like that?
The mappings I have in play are these:
{
"mappings": {
"properties": {
"lox": {
"type": "keyword",
"store": true
},
"label": {
"type": "text",
"analyzer": "standard",
"fields": {
"en": {
"type": "text",
"analyzer": "english"
},
"fr": {
"type": "text",
"analyzer": "french"
},
"de": {
"type": "text",
"analyzer": "german"
}
}
},
"details": {
"type": "text",
"analyzer": "standard",
"fields": {
"en": {
"type": "text",
"analyzer": "english",
"store": true
},
"fr": {
"type": "text",
"analyzer": "french",
"store": true
},
"de": {
"type": "text",
"analyzer": "german",
"store": true
}
}
}
}
}
}
If your intention is to have a documenst that looks like these
POST discuss/_doc
{
"lox": "1656282032356",
"details": {
"en": "Long Sleeve Cotton",
"fr": "Coton à manches longues"
},
"label": {
"en": "Large Red Shirt",
"fr": "Grande chemise rouge"
}
}
# Or
POST discuss/_doc
{
"lox": "1656282032356",
"details": {
"en": "Long Sleeve Cotton"
},
"label": {
"en": "Large Red Shirt"
}
}
# Or
POST discuss/_doc
{
"lox": "1656282032356",
"details": {
"fr": "Coton à manches longues"
},
"label": {
"fr": "Grande chemise rouge"
}
}
Then your mapping would look like
PUT discuss/
{
"mappings": {
"properties": {
"lox": {
"type": "keyword",
"store": true
},
"label": {
"properties": {
"en": {
"type": "text",
"analyzer": "english"
},
"fr": {
"type": "text",
"analyzer": "french"
},
"de": {
"type": "text",
"analyzer": "german"
}
}
},
"details": {
"properties": {
"en": {
"type": "text",
"analyzer": "english",
"store": true
},
"fr": {
"type": "text",
"analyzer": "french",
"store": true
},
"de": {
"type": "text",
"analyzer": "german",
"store": true
}
}
}
}
}
}
If you want a single fields with the same value that is analyzed several ways then it would look like this.. but I am not sure that make sense. In this case english would be the default.
PUT discuss/
{
"mappings": {
"properties": {
"lox": {
"type": "keyword",
"store": true
},
"label": {
"type": "text",
"analyzer": "english",
"fields": {
"fr": {
"type": "text",
"analyzer": "french"
},
"de": {
"type": "text",
"analyzer": "german"
}
}
},
"details": {
"type": "text",
"analyzer": "english",
"store": true,
"fields": {
"fr": {
"type": "text",
"analyzer": "french",
"store": true
},
"de": {
"type": "text",
"analyzer": "german",
"store": true
}
}
}
}
}
}
Then you are just analyzing the same text 3 different ways... maybe that is what you want.
POST discuss/_doc
{
"lox": "1656282032356",
"details": "Long Sleeve Cotton",
"label": "Large Red Shirt"
}
GET /discuss/_search
{
"fields": [
"*"
]
}
# Result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "discuss",
"_id" : "NEw1o4EBUD-rWNEOQAdl",
"_score" : 1.0,
"_source" : {
"lox" : "1656282032356",
"details" : "Long Sleeve Cotton",
"label" : "Large Red Shirt"
},
"fields" : {
"details.fr" : [
"Long Sleeve Cotton"
],
"label.fr" : [
"Large Red Shirt"
],
"lox" : [
"1656282032356"
],
"details.de" : [
"Long Sleeve Cotton"
],
"details" : [
"Long Sleeve Cotton"
],
"label" : [
"Large Red Shirt"
],
"label.de" : [
"Large Red Shirt"
]
}
}
]
}
}
New lesson: if you go up and post new mappings, you get an error that mappings already exist.
So, you google deleting mappings and learn that post ES 7 you don't delete mappings. Instead, you migrate them according to some largely tangled instructions. The instructions speak to "types" but, um, where is the "type" in
PUT discuss/
{
"mappings": {
discuss is the index name. Maybe that's the type?
So if I use that interpretation, I should do all my practice on index names I don't mind throwing away for production?
Confused...
Many thanks for the suggestions. They feel right, but can't test them until I resolve the issue.
An obvious interim solution would be to delete the data and start over (i've done that before).
Indeed, by deleting ES data and starting over, including new passwords and kibana tokens, I blew right past that issue, which takes me now to the "final frontier" of making my queries work. I'll now be able to debug those.
Many thanks again.
my conclusion for going forward: do your development work on throw away index definitions and switch to production once everything is working.
There are no more "types" in indices...they have been removed see here
Think of a mapping as the schema of an individual table in an RDBMS.
To add a to the confusion 
That table could have many fields that are sparsely populated and a field that indicated a type if you like.
discuss Is the index name.
If the schema of your data is wildly different, you would probably want to put it in two different indices / mappings.
If they're close / similar and you just want to indicate that there is some field that tells you which type it is so you could sort or filter that's fine too.
What has been removed is setting
_type to anything other than _doc
With respect to development on prod and production to me, it's like any other datastore ... You develop in development, tested out and staging or non-prod and then you promote thank you production.
Let us know when you have some questions on queries. Open a new thread.
Glad you're moving forward
Yes, I'm moving forward. Thanks for all your help.
For now, finding decent documentation on complex structures like the one I used for testing is really difficult, so I backed off and am just working with simple structures to develop the queries for those.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.