Script based transform during index

Elastic Stack Elasticsearch
1

Hi Group
First apologies if this is not the right way to ask the below question but
this is my first time.

I have some documents with source IP and destination IP address. I want to
enhance these documents with geo info with script transform when they
arrive. So I created a script in python and I resolve geo info for every
destination ip and store in _source (from what I understand)

The template for the index is as below. Everything works fine however I
have two issues.

  1. The field (which does not exist and I create it namely "location") is
    not shown in a search unless explicitly asked.
  2. Kibana 3 does not show this field or it shows as empty.

The field location is there if I explicitly ask for it. Can you please let
me know how I can have these added fields prior to index available as
normal fields ?

Thanks in advance !

P.S inside the python script I update the below
ctx['location'] = ip2geo(dest_ip)
ctx['_source']['location'] = ip2geo(dest_ip)

POST /geotest/gdoc/_search
{
"query": {
"match_all": {}
},
"fields": [
"src_ip",
"dst_ip",
"location" <-- This is the new field which I add via
ctx['_source']['location'] = ip2geo(dest_ip)
]
}

My template

PUT /_template/geo

{
"template": "geo*",
"mappings": {
"gdoc": {
"transform": {
"lang": "python",
"script": "python_ip2geo"
},
"_source": {
"enabled": "true"
},
"properties": {
"src_ip": {
"type": "ip",
"index": "not_analyzed"
},
"dst_ip": {
"type": "ip",
"index": "not_analyzed"
},
"location": {
"type": "geo_point",
"index": "analyzed", <-- does not need to be analyzed
really
"store": "true",
"doc_values": "true",
"null_value": ""
}
}
}
}
}

Can someone explain a bit more on how transform fields are stored and how
they can be indexed ?

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3e552ca9-f110-47f0-aae5-63ea8f2a89d8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.