My index has a nested definition that includes a property of type geo-point. I can see it in Discover but it is not accessible from Documents layer in Maps. Is this the expected behavior or does my mapping need to change? I am using ES 7.6 currently.
If I have the fields mapped separated - not as part of nested definition, then I can access the geo-point field in Maps. This is fine for an object that has a single entry - I can flatten.
Problem is when I have a field that is a list of objects, which is why I am treating as nested object so that I can retain relationships within each object.
I know also, that for the "list of objects", I can populate a separate field to capture all the geo-points - just trying to avoid that, if possible. It is an accessible property in Maps.
Can I get some support for writing an ingest pipeline that will walk the array of nested objects an pull out the "coordinates" field into a separate field in the index that Maps can use?
I can provide details.
I tried using an ingest pipeline with a "foreach" to walk the list and then an "append" within the "foreach" to copy each "coordinates" object to a new field (a list of "coordinates") for the index BUT "coordinates" is getting converted to a string and lat/long are no longer recognized as a "geo_point". think I I understand why that is happening in the append processor.
Can you suggest a better way to access the "coordinates" objects from the nested list?
I'd suggest you post your last question as a new topic, this time in the Elastic Stack -> Elasticsearch forum using the stack-ingest-management tag so you reach the correct audience for your issue.
Nevermind, I think I got it. Instead of a foreach processor I used a painless script. You have below the full set of instructions from the dev console but the main part is the script itself that simply loops over the nested field and adds the coordinates to a new array that is later assigned to the nationsTravelledCoordsList field.
Mind that this probably would benefit from some type checking.
int records = ctx.nationsTravelled.length;
List result = [];
for (int i = 0; i < records; i++){
result.add(ctx.nationsTravelled[i].coordinates)
}
ctx.nationsTravelledCoordsList = result ;
Let us know if this was helpful.
Script on DevTools
# Remove the index if it exists
DELETE test_index_nested
# Create the index with the nested field and the outer list to receive the data
PUT test_index_nested
{
"mappings": {
"properties": {
"nationsTravelled": {
"type": "nested",
"properties": {
"coordinates": {"type": "geo_point"},
"name": {"type": "keyword"}
}
},
"nationsTravelledCoordsList": {"type":"geo_point"}
}
}
}
# Create a pipeline with a script processor
PUT _ingest/pipeline/nested_test
{
"processors": [
{
"script": {
"source": """
int records = ctx.nationsTravelled.length;
List result = [];
for (int i = 0; i < records; i++){
result.add(ctx.nationsTravelled[i].coordinates)
}
ctx.nationsTravelledCoordsList = result ;
"""
}
}
]
}
# Test the pipeline, mind that in string notation the order is lat/lon (!)
POST _ingest/pipeline/nested_test/_simulate
{
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"nationsTravelled": [
{
"coordinates": "40.797177,-73.960648",
"name": "New York"
},
{
"coordinates": "19.425154,-99.113159",
"name": "Mexico City"
},
{
"coordinates": "-33.449777,-70.648956",
"name": "Santiago de Chile"
},
{
"coordinates": "-33.961586,18.462524",
"name": "Cape Town"
}
]
}
}
]
}
# Insert the document
POST test_index_nested/_doc?pipeline=nested_test
{
"nationsTravelled": [
{
"coordinates": "40.797177,-73.960648",
"name": "New York"
},
{
"coordinates": "19.425154,-99.113159",
"name": "Mexico City"
},
{
"coordinates": "-33.449777,-70.648956",
"name": "Santiago de Chile"
},
{
"coordinates": "-33.961586,18.462524",
"name": "Cape Town"
}
]
}
# Check the result
GET test_index_nested/_search
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.