Problem creating transforms

[JulGor (Julio)]

Hi,

I upgraded by cluster to version 7.5 and I'm trying the Transforms from Kibana but I'm having some problem that I cannot identify.

The big picture is that when I create a continuous transform with more than 4-5 aggregations, the transform is created but not the index and it can't be started. When it happens and I click on the "Start" button, the only message I receive is:

"An error occurred starting the transform XXX"

with no further detail...

I've tried creating exactly the same transform (same configuration) and changing the number of aggregations: 1, 2, 3... My final goal is creating one with 13 aggregations but I could never create any transform with more than 4 aggregations.

I also checked that is not a problem with the fields I'm aggregating, because for example, if I create 1 transform with:

• avg(field1)
• sum(field1)
• avg(field2)
• sum(field2)
• avg(field3)
• sum(field3)
• avg(field4)
• sum(field4)

It always fails, but if I do 2 transforms with the same aggregations splitted into fields1-2 and fields3-4, then both transforms works normally.

Is there some limitations in the number of transforms supported?

Note: I'm using Elastic Service

Thanks in advance!!!

[Hendrik_Muhs]

Hi,

there is no limitation in the number of aggregations. Therefore it seems your transform fails for other reasons.

The destination index is created on start, not as part of creating the transform. On start transform checks whether the destination index exists and creates it if not. It would be good to know if the destination index gets created or not.

To further debug your issue:

Can you check the message tab of your job, you can find it in the jobs list after expanding the view, for example:

In addition it would be useful if you can query for transform stats in the dev console like this:

GET _transform/{your_failing_transform}/_stats

The output of this command should contain the failure reason.

Please let me know about your findings.

[JulGor (Julio)]

Hi Hendrik,

Thanks for your reply.

The index never is created and I double checked that there is no index with that name as you can see in the following images:

So, I guess the problem is at index creation, but I can't figure out the reason because there is no error message:

Is there anything else I can check to find the error?

Thanks!

[Hendrik_Muhs]

Hi,

the error happens when transforms creates the index (the audit message is send right before the index creation).

Can you start the transform via the dev console and send me the output:

POST _transform/orders_by_cutomers_min/_start 

The full error should be in the response.

[JulGor (Julio)]

Hi Hendrik,

This is the error from the Dev console:

I checked the mapping of that field and it is a float field:

In fact, during my tests, I created other continuous transforms with the same aggregations for that field and they work perfectly:

Further, the Preview of the transform that is failling does also work:

Any idea?

Many thanks!

[Hendrik_Muhs]

Good morning!

We are getting closer, thanks for your patience.

The error means in a nutshell that transform tries to create a destination index with a field subtotal_price of some simple type and a nested field subtotal_price.something, so basically an object with another field in it.

I tried to reproduce your problem: for me the UI errors for such a mapping conflict on preview. I would like to find out, why for you it does not, seems like a special case.

Can you share your configuration, e.g. for the preview, you find a copy method on the upper right corner of your preview in the UI:

FWIW: As a workaround you can rename all your nested fields, e.g subtotal_price.avg to subtotal_price_avg.

[JulGor (Julio)]

Hi Hendrik,

Thank you for your patient and your time.

Here is the configuration os the "source index preview":

GET orders/_search
{
  "query": {
    "bool": {
      "must_not": {
        "bool": {
          "should": [
            {
              "exists": {
                "field": "cancelled_at"
              }
            }
          ],
          "minimum_should_match": 1
        }
      }
    }
  }
}

And, this is the full transform:

POST _transform/_preview
{
  "source": {
    "index": [
      "orders"
    ],
    "query": {
      "bool": {
        "must_not": {
          "bool": {
            "should": [
              {
                "exists": {
                  "field": "cancelled_at"
                }
              }
            ],
            "minimum_should_match": 1
          }
        }
      }
    }
  },
  "pivot": {
    "group_by": {
      "customer.email.keyword": {
        "terms": {
          "field": "customer.email.keyword"
        }
      }
    },
    "aggregations": {
      "subtotal_price.avg": {
        "avg": {
          "field": "subtotal_price"
        }
      },
      "subtotal_price.sum": {
        "sum": {
          "field": "subtotal_price"
        }
      },
      "total_discounts.avg": {
        "avg": {
          "field": "total_discounts"
        }
      },
      "total_discounts.sum": {
        "sum": {
          "field": "total_discounts"
        }
      },
      "total_price.avg": {
        "avg": {
          "field": "total_price"
        }
      },
      "total_price.sum": {
        "sum": {
          "field": "total_price"
        }
      },
      "customer.orders_count.max": {
        "max": {
          "field": "customer.orders_count"
        }
      }
    }
  }
}

Cheers

[Hendrik_Muhs]

Hi,

can you send me the output of the preview call, after you executed it in the dev console? It returns the mappings it would use to create the destination index.

I see nothing wrong with your config. This is a transform that fails on start for you?

[JulGor (Julio)]

Sure, here it is:

{
  "preview" : [
    {
      "total_discounts" : {
        "avg" : 0.0,
        "sum" : 0.0
      },
      "subtotal_price" : {
        "avg" : 323.0,
        "sum" : 323.0
      },
      "total_price" : {
        "avg" : 323.0,
        "sum" : 323.0
      },
      "customer" : {
        "orders_count" : {
          "max" : 1.0
        },
        "email" : {
          "keyword" : "XXXXX"
        }
      }
    },
    {...rest of the results...}
  ],
  "mappings" : {
    "properties" : {
      "total_discounts.sum" : {
        "type" : "double"
      },
      "subtotal_price.sum" : {
        "type" : "double"
      },
      "total_price.sum" : {
        "type" : "double"
      },
      "total_price.avg" : {
        "type" : "double"
      },
      "customer.email.keyword" : {
        "type" : "keyword"
      },
      "total_discounts.avg" : {
        "type" : "double"
      },
      "customer.orders_count.max" : {
        "type" : "long"
      },
      "subtotal_price.avg" : {
        "type" : "double"
      }
    }
  }
}

Many thanks!!

[Hendrik_Muhs]

Hi,

the mappings look fine, however there might be another source for conflicting mappings: index templates.

To find out, you can try to create your problematic transform with a destination index not starting with the prefix orders, for example transform_orders_by_customers_min?

If that still fails and/or to check whether the suspicion is correct, can you get the index templates with:

GET /_template

and look for templates that start with orders or a template that would match orders, e.g. a template for *? The pattern is defined in the field index_patterns of the individual templates.

[JulGor (Julio)]

Your are great, Hendrik!! That was the problem.

There is one template that is affecting the final mapping, although I don't know exactly why.

I checked all my templates and find the only one that matches the index name and review the mapping and it looks OK to me (I keep just the fields that I'm using for grouping and aggregating):

[...]
    "index_patterns" : [
      "orders*"
    ],
[...]
"mappings" : {
  "properties" : {
    [...]
    "subtotal_price" : {
      "type" : "float"
    },
    "total_discounts" : {
      "type" : "float"
    },
    "total_price" : {
      "type" : "float"
    },
    "customer" : {
      "properties" : {
        "orders_count" : {
          "type" : "long"
        },
        "email" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "ignore_above" : 256,
              "type" : "keyword"
            }
          }
        },
        [...]
    },
    [...]

If you know the reason, please share it with me.

In any way, thanks a lot for helping me with this!!!

[Hendrik_Muhs]

The problem is that the template defines:

{
    "subtotal_price" : {
        "type" : "float"
    },
}

which in json would be for example:

{
    "subtotal_price": 55
}

but in the transform you create:

{
    "subtotal_price.sum" : {
        "type" : "double"
    },
    "subtotal_price.avg" : {
        "type" : "double"
    }
}

which in json looks like this:

{
    "subtotal_price": {
        "sum": 100,
        "avg": 23
    }
}

As you can see sum and avg are nested objects of subtotal_price. You can not have something that is a number and an object at the same time.

I admit the error message is quite cryptic and it would be nice if request validation would have found this problem before _start.

Several usability issues have been found thanks to you. I will follow up and put them on the backlog for upcoming releases.

[JulGor (Julio)]

Now I got it.

Thanks again for your help and your time!