Curl query from json

Elastic Stack Elasticsearch
1

Hi guys,
I have some query(witch works fine in kibana debugger), my goal is to make this query work from curl command line(windows). i have other queries with no issues.
I think the problem is with the """ in the "reduce_script" part. it breaks the json structure.
The working query:

{
   "id":"index_compare",
   "source":{
      "index":[
         "mitre",
         "winlogbeat-7.14.0-2022.01.06-000001"
      ],
      "query":{
         "match_all":{
            
         }
      }
   },
   "dest":{
      "index":"compare"
   },
   "pivot":{
      "group_by":{
         "unique-id":{
            "terms":{
               "field":"winlog.event_id"
            }
         }
      },
      "aggregations":{
         "compare":{
            "scripted_metric":{
               "map_script":"state.doc = new HashMap(params['_source'])",
               "combine_script":"return state",
               "reduce_script":""" 
            if (states.size() != 2) {
return ""count_mismatch""
            }
return ""match""
            """
            }
         }
      }
   }
}

What i have tried:
command :

curl -XPOST --header "Content-Type: application/json" "http://localhost:9200/_transform/_preview?pretty" -d @2.json

json file:

{
  "id": "index_compare",
  "source": {
    "index": [
      "mitre",
      "winlogbeat-7.14.0-2022.01.06-000001"
    ],
    "query": {
      "match_all": {}
    }
  },
  "dest": {
    "index": "compare"
  },
  "pivot": {
    "group_by": {
      "unique-id": {
        "terms": {
          "field": "winlog.event_id"
        }
      }
    },
    "aggregations": {
      "compare": {
        "scripted_metric": {
          "map_script": "state.doc = new HashMap(params['_source'])",
          "combine_script": "return state",
"reduce_script": """ 
            if (states.size() != 2) {
return "count_mismatch"
            }
return "match"
            """
        }
      }
    }
  }
}

Any ideas?

2

If you are already using Kibana Dev Tools, you can click on the wrench icon for the request and toggle through the 'Auto indent' or use the 'Copy as cURL' option and it will format it as valid JSON for you to use. The triple double quotes are only usable in Dev Tools to help make things easily readable with CR/LF characters, etc.

You should end up with something like the following:
"reduce_script":" \n if (states.size() != 2) {\nreturn \"\"count_mismatch\"\"\n }\nreturn \"\"match\"\"\n "

You could also clean that up even more if you wanted.

3

Hi angelo,
Thanks for your response !
I have just tried 2 things:

  1. 'Copy as cURL from the wrench button - did not work - "Unrecognized character escape '''
  2. replaced the reduce_script with the string you suggested - also did not work. json parse exception.

Thanks again

4

Can you post your updated version of your JSON file and any additional error details - ie: listing failing line or column number?

5

I got it working by only using the "reduce_script" part from the wrench.
Something is wrong with the escaping output from the wrench.
The working request:

{
  "id": "index_compare",
  "source": {
    "index": [
      "mitre",
      "winlogbeat-7.14.0-2022.01.06-000001"
    ],
    "query": {
      "match_all": {}
    }
  },
  "dest": {
    "index": "compare"
  },
  "pivot": {
    "group_by": {
      "unique-id": {
        "terms": {
          "field": "winlog.event_id"
        }
      }
    },
    "aggregations": {
      "compare": {
        "scripted_metric": {
          "map_script": "state.doc = new HashMap(params['_source'])",
          "combine_script": "return state",
"reduce_script": " \r\n            if (states.size() != 2) {\r\nreturn \"count_mismatch\"\r\n            }\r\nreturn \"match\"\r\n            "
        }
      }
    }
  }
}'

Thanks for your solution angelo !!

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.