I am able to install v7.2 with the statefulset config that is listed below, but for a service I'm running, it hasn't been updated to support v7 yet. Currently, the service only supports v6.3. The following is the error I get when trying to boot up the statefulset.
Statefulset config
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-cluster
namespace: logging
spec:
serviceName: elasticsearch
replicas: 3
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
spec:
containers:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:6.3.1
resources:
limits:
cpu: 1000m
requests:
cpu: 100m
ports:
- name: http
containerPort: 9200
protocol: TCP
- name: transport
containerPort: 9300
protocol: TCP
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
env:
- name: cluster.name
value: k8s-logs
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: discovery.seed_hosts
value: "es-cluster-0.elasticsearch,es-cluster-1.elasticsearch,es-cluster-2.elasticsearch"
- name: cluster.initial_master_nodes
value: "es-cluster-0,es-cluster-1,es-cluster-2"
- name: ES_JAVA_OPTS
value: "-Xms512m -Xmx512m"
initContainers:
- name: fix-permissions
image: busybox
command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
securityContext:
privileged: true
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
- name: increase-vm-max-map
image: busybox
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
- name: increase-fd-ulimit
image: busybox
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
volumeClaimTemplates:
- metadata:
name: data
labels:
app: elasticsearch
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: gp2
resources:
requests:
storage: 100Gi
Error log. Too long to include it in OP.
[es-cluster-0] initializing ...
[es-cluster-0] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchException[java.io.IOException: failed to read [id:4, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/node-4.st]]; nested: IOException[failed to read [id:4, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/node-4.st]]; nested: IllegalArgumentException[[node_meta_data] unknown field [node_version], parser not found];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.3.1.jar:6.3.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.3.1.jar:6.3.1]
Caused by: org.elasticsearch.ElasticsearchException: java.io.IOException: failed to read [id:4, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/node-4.st]
at org.elasticsearch.ExceptionsHelper.maybeThrowRuntimeAndSuppress(ExceptionsHelper.java:199) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.gateway.MetaDataStateFormat.loadLatestState(MetaDataStateFormat.java:331) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeEnvironment.loadOrCreateNodeMetaData(NodeEnvironment.java:357) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:245) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.node.Node.(Node.java:270) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:213) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.3.1.jar:6.3.1]
... 6 more
Caused by: java.io.IOException: failed to read [id:4, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/node-4.st]
at org.elasticsearch.gateway.MetaDataStateFormat.loadLatestState(MetaDataStateFormat.java:325) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeEnvironment.loadOrCreateNodeMetaData(NodeEnvironment.java:357) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:245) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.node.Node.(Node.java:270) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:213) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.3.1.jar:6.3.1]
... 6 more
Caused by: java.lang.IllegalArgumentException: [node_meta_data] unknown field [node_version], parser not found
at org.elasticsearch.common.xcontent.ObjectParser.getParser(ObjectParser.java:347) ~[elasticsearch-x-content-6.3.1.jar:6.3.1]
at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:158) ~[elasticsearch-x-content-6.3.1.jar:6.3.1]
at org.elasticsearch.common.xcontent.ObjectParser.apply(ObjectParser.java:182) ~[elasticsearch-x-content-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeMetaData$1.fromXContent(NodeMetaData.java:110) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeMetaData$1.fromXContent(NodeMetaData.java:94) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.gateway.MetaDataStateFormat.read(MetaDataStateFormat.java:199) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.gateway.MetaDataStateFormat.loadLatestState(MetaDataStateFormat.java:320) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeEnvironment.loadOrCreateNodeMetaData(NodeEnvironment.java:357) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:245) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.node.Node.(Node.java:270) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:213) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.3.1.jar:6.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.3.1.jar:6.3.1]
You are attempting to downgrade a node in place. The data on disk was written by a 7.2.0 node so cannot be read by a node of an earlier version.
If this is a new installation then your best bet is to wipe the data path and start again. If you need to preserve the data then you will need to do a reindex-from-remote to get the data back into the old cluster.
Ah of course; I don't know why I didn't think of that. When upgrading from 6.3 to 7.2+, will the new installation be able to utilize the database, or will I need to re-index? It's not mentioned in the upgrade docs.
Upgrading in place is fine, indeed it's normally the best thing to do. It's just downgrading that isn't supported.