X-Pack is not supported and Machine Learning is not available for [linux-s390x]


(Chakravarthy G) #1

According to this, ml wont be suppored if we use 32 bit JVM ,but I am using 64 bit machine and JVM (OpenJDK 64-Bit Server VM (build 10.0.2-adoptopenjdk+13, mixed mode)) , but still getting the same exception while starting Elastic search-6.4.0 and Test case is getting failed with below message.

> ERROR   2.03s | MachineLearningLicensingTests.testMachineLearningPutJobActionRestricted <<< FAILURES!
   > Throwable #1: org.elasticsearch.common.inject.CreationException: Guice creation errors:
   > 1) Error injecting constructor, ElasticsearchException[X-Pack is not supported and Machine Learning is not available for [linux-s390x]; you can use the other X-Pack features (unsupported) by setting xpack.ml.enabled: false in elasticsearch.yml]
   >   at org.elasticsearch.xpack.ml.MachineLearningFeatureSet.<init>(Unknown Source)
   >   while locating org.elasticsearch.xpack.ml.MachineLearningFeatureSet
   > Caused by: ElasticsearchException[X-Pack is not supported and Machine Learning is not available for [linux-s390x]; you can use the other X-Pack features (unsupported) by setting xpack.ml.enabled: false in elasticsearch.yml]

So is it specific to architecture or JVM ? If we can't enable ml support, does it impact functionality of Elastic search ?


(Christian Dahlqvist) #2

Machine learning is architecture specific as it is implemented in C++, and it looks like you are running on a unsupported platform.


(Chakravarthy G) #3

Thanks Chrisitain ,
Executable files required for few architectures have been maintained as zip file @ http://prelert-artifacts.s3.amazonaws.com/maven/org/elasticsearch/ml/ml-cpp/6.4.0/ml-cpp-6.4.0.zip and this is getting downloaded while building elastic search.
Is there any way to build x-pack for other architectures also !?


(David Roberts) #4

There isn't an easy way to build the ML C++ for a new platform.

The steps to do it would be:

  1. Work out how to set up a suitable build machine - the process would be similar to the instructions in https://github.com/elastic/ml-cpp/tree/master/build-setup but with changes to account for the different platform
  2. Create appropriate make rules for the new platform in https://github.com/elastic/ml-cpp/tree/master/mk
  3. Try building, see what fails to compile, make code changes to account for platform differences, repeat until it compiles

There are currently no plans to do this for Z series mainframes.

If you want to take your chances developing Elasticsearch on a Z series mainframe your best bet is probably to write an alias for invoking gradlew with a number of -X arguments that exclude all the test tasks that fail because the ML C++ doesn't exist. Probably also edit the code to set the default for xpack.ml.enabled to false so that non-ML test suites get past that check.


(Christian Dahlqvist) #5

No part of the Elastic stack is supported on Z series mainframe, so if you are looking to utilise the stack and avoid potentially exotic problems I would recommend you switch to one of the supported platforms.


(Chakravarthy G) #6

Thanks @droberts195 & @Christian_Dahlqvist for the response,
i am from IBM, we are trying to add s390x support for Elasticsearch . As we are facing issues with x-pack and ml which are not supported for s390x, checking if we can port these as suggested by @droberts195.


(David Roberts) #7

Just wanted to check you are aware of how the Elastic stack is licensed. Some parts are free open source under the Apache License v2. Other parts, including all of X-Pack (and the ML C++ code is part of X-Pack), are under the commercial Elastic license. Section 2 of the Elastic license says this:

2. COMMERCIAL SOFTWARE SOURCE CODE

  2.1 Limited License. Subject to the terms and conditions of Section 2.2 of
  this Agreement, Elastic hereby grants to You, AT NO CHARGE and for so long as
  you are not in breach of any provision of this Agreement, a limited,
  non-exclusive, non-transferable, fully paid up royalty free right and license
  to the Commercial Software in Source Code format, without the right to grant
  or authorize sublicenses, to prepare Derivative Works of the Commercial
  Software, provided You (i) do not hack the licensing mechanism, or otherwise
  circumvent the intended limitations on the use of Elastic Software to enable
  features other than Basic Features and Functions or those features You are
  entitled to as part of a Subscription, and (ii) use the resulting object code
  only for reasonable testing purposes.

  2.2 Restrictions. Nothing in Section 2.1 grants You the right to (i) use the
  Commercial Software Source Code other than in accordance with Section 2.1
  above, (ii) use a Derivative Work of the Commercial Software outside of a
  Non-production Environment, in any production capacity, on a temporary or
  permanent basis, or (iii) transfer, sell, rent, lease, distribute, sublicense,
  loan or otherwise make available the Commercial Software Source Code, in whole
  or in part, to any third party. Notwithstanding the foregoing, You may
  maintain a copy of the repository in which the Source Code of the Commercial
  Software resides and that copy may be publicly accessible, provided that you
  include this Agreement with Your copy of the repository.

So the license does not prohibit you from having a go at porting Elasticsearch including the X-Pack bits to s390x. I'm sure you'll succeed if you try hard enough as it's just Linux running on a different hardware architecture. However, the Elastic license would not permit you to go beyond this test/development environment and run the X-Pack portions of Elasticsearch on s390x in production. If you want to go into production with your ported code without agreeing some sort of special deal with Elastic then you'll need to limit your port to just the Apache 2 licensed parts (and for this see my earlier reply about the -X argument of gradlew).

Even if you submit PRs that make Elasticsearch including the ML C++ work on s390x, without a broader partnership agreement Elastic will not add simply add s390x to the support matrix because of the complications with ongoing testing to ensure we don't accidentally break something on s390x.

Is your team at IBM already in contact with an Elastic account manager or someone in our partners team about this project? If you have not done so already I think you need to think about the licensing aspects of this project before putting too much work into it.


(George) #8

@chakravarthy_G

any luck with this? we could really use ML support under s390x. Have you guys tried @droberts195 instructions?

Thanks for your help!


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.