Python Elasticsearch-DSL UpdateByQuery Syntax

[_Joe]

ElasticSearch 7.5 Indices are firewall log data. I am experiencing too many failed GeoIP lookups using GeoLite2-City.mmdb. To fix these issues, I am working on a Python script that will do lookups for documents with missing location data to ipgeolocation.io and then update the documents.

Everything works great up to where the documents need to be updated. I am new to DSL and Painless scripting, and would appreciate some help.

The Painless script is...

script = {
    "script": {
      "lang": "painless",
      "source": """
          ctx._source.geoip.city_name = params.city_name;
          ctx._source.geoip.continent_code = params.continent_code;
          ctx._source.geoip.country_name = params.country_name;
          ctx._source.geoip.country_code2 = params.country_code2;
          ctx._source.geoip.country_code3 = params.country_code3;
          ctx._source.geoip.dma_code = params.dma_code;
          ctx._source.geoip.ip = params.ip;
          ctx._source.geoip.latitude = params.latitude;
          ctx._source.geoip.longitude = params.longitude;
          ctx._source.geoip.postal_code = params.postal_code;
          ctx._source.geoip.region_code = params.region_code;
          ctx._source.geoip.region_name = params.region_name;
          ctx._source.geoip.timezone = params.timezone;
          ctx._source.tags = params.newTags
      """,
      "params": {
        "city_name": city_name,
        "continent_code": continent_code,
        "country_name": country_name,
        "country_code2": country_code2,
        "country_code3": country_code3,
        "dma_code": dma_code,
        "ip": ip,
        "latitude": str(latitude),
        "longitude": str(longitude),
        "postal_code": postal_code,
        "region_code": region_code,
        "region_name": region_name,
        "timezone": timezone,
        "newTags": newTagsList
      }
    }
  }

The script string prints correctly...

Python code to execute the UpdateByQuery is...

docsUBQ = UpdateByQuery(using=client, index=index).filter("term",tags=tagFailed).query("match", src_ip=src_ip)

docsUBQ = docsUBQ.script(source=script)

The above query works fine if UpdateByQuery() is replaced with Search(). It returns all documents with failed GeoIP lookups for the specified src_ip.

No errors are being returned; however, no updates are being made...

What am I missing, other than a brain?

[_Joe]

SOLUTION...

tagFailed = "_geoip_lookup_failure"
src_ip = "194.26.29.117"

city_name = "Saint Petersburg"
continent_code = "EU"
country_name = "Russia"
country_code2 = "RU"
country_code3 = "RUS"
ip = "194.26.29.117"
latitude = 59.88881
longitude = 30.31671
location = {"lat": 59.88881, "lon": 30.31671}
postal_code = "196084"
region_code = ""
region_name = "Northwestern Federal District"
timezone = "Europe/Moscow"
newTags = str(["PFSense", "firewall", "DstPortService", "_geoip_lookup_fixed"])

source = "ctx._source.geoip.city_name = params.city_name; ctx._source.geoip.continent_code = params.continent_code; ctx._source.geoip.country_name = params.country_name; ctx._source.geoip.country_code2 = params.country_code2; ctx._source.geoip.country_code3 = params.country_code3; ctx._source.geoip.ip = params.ip; ctx._source.geoip.latitude = params.latitude; ctx._source.geoip.longitude = params.longitude; ctx._source.geoip.location = params.location; ctx._source.geoip.region_code = params.region_code; ctx._source.geoip.region_name = params.region_name; ctx._source.geoip.timezone = params.timezone; ctx._source.tags = params.newTags"

params = {'city_name': city_name, 'continent_code': continent_code, 'country_name': country_name, 'country_code2': country_code2, 'country_code3': country_code3, 'ip': ip, 'latitude': latitude, 'longitude': longitude, 'location': location, 'region_code': region_code, 'region_name': region_name, 'timezone': timezone, 'newTags': newTags}

docsUBQ = UpdateByQuery().using(client).index(index).query("match", src_ip=src_ip).filter("term", tags=tagFailed).script(source=source, params=params)
response = docsUBQ.execute()