Elasticsearch shut down (with Jupyter notebooks)

Elastic Stack Elasticsearch
1

I am trying to use Elasticsearch directly from Jupyter notebooks. The code that is launching it is

import os
from subprocess import Popen, PIPE, STDOUT
# Run Elasticsearch as a background process 
!chown -R daemon:daemon elasticsearch-9.0.3
es_server = Popen( # start subprocess
    args=['elasticsearch-9.0.3/bin/elasticsearch'], 
    #stdout=PIPE, stderr=STDOUT, # outcomment this entire code line in order to see log directly on Jupyter cell output
    preexec_fn=lambda: os.setuid( os.getuid() ) # ID of subprocess
)
# Wait until Elasticsearch has started:
!sleep 30 # 30 seconds

The problem is that, when I restart the script, then Elasticsearch will give me errors that it has not exited normally. As a result, I am forced to delete the elasticsearch-9.0.3 folder and download it again.
Currently I am working with the workaround of launching Elasticsearch with the Linux terminal command

./bin/elasticsearch

The terminal allows me to stop Elasticsearch without any problems, when I enter Ctlr+C. However, I would like to know, if I can do it automatically in Jupyter notebooks without having to reinstall Elasticsearch each time I want to restart the script.
In addition, the program is overwriting my elasticsearch.yml file (deleting everything except the lines that are commented in the yml).

I am new to using Elasticsearch, so I appreciate every explanation for why these weird problems are happening.

2

Btw. the Python code is from the book " Natural Language Processing with Transformers: Building Language Applications With Hugging Face". I am trying to follow it, but the code is a bit outdated, because it uses elasticsearch-7.9.2.
Restarting elasticsearch-7.9.2 with Python scripts causes no problems. It looks like Elasticsearch 9 is a bit trickier.

3

Maybe share the actual errors?

Also, did you consider to use the same elasticsearch version as in the book you are following? As a beginner, you are (please take no offence!) less likely to be able to know/spot what you need to change or adapt between elasticsearch versions that are 2 major releases (and almost 5 years) apart?

4

The whole log is too long, but you can see the important part at the bottom.

chown: changing ownership of 'elasticsearch-9.0.3/LICENSE.txt': Operation not permitted
chown: changing ownership of 'elasticsearch-9.0.3': Operation not permitted
[2025-07-11T19:56:59,825][INFO ][o.e.b.Elasticsearch      ] [MyPC] version[9.0.3], pid[16558], build[tar/cc7302afc8499e83262ba2ceaa96451681f0609d/2025-06-18T22:09:56.772581489Z], OS[Linux/6.11.0-29-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/24/24+36-3646]
[2025-07-11T19:56:59,852][INFO ][o.e.b.Elasticsearch      ] [MyPC] JVM home [/home/myname/Projects/Playground/elasticsearch-9.0.3/jdk], using bundled JDK [true]
[2025-07-11T19:56:59,858][INFO ][o.e.b.Elasticsearch      ] [MyPC] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=CLDR, -Dorg.apache.lucene.vectorization.upperJavaFeatureVersion=24, -Des.distribution.type=tar, -Des.java.type=bundled JDK, --enable-native-access=org.elasticsearch.nativeaccess,org.apache.lucene.core, --enable-native-access=ALL-UNNAMED, --illegal-native-access=deny, -XX:ReplayDataFile=logs/replay_pid%p.log, -Des.entitlements.enabled=true, -XX:+EnableDynamicAgentLoading, -Djdk.attach.allowAttachSelf=true, --patch-module=java.base=lib/entitlement-bridge/elasticsearch-entitlement-bridge-9.0.3.jar, --add-exports=java.base/org.elasticsearch.entitlement.bridge=org.elasticsearch.entitlement,java.logging,java.net.http,java.naming,jdk.net, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-9171026149478436671, --add-modules=jdk.incubator.vector, -Dorg.apache.lucene.store.defaultReadAdvice=normal, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms16015m, -Xmx16015m, -XX:MaxDirectMemorySize=8396996608, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=25, --module-path=/home/myname/Projects/Playground/elasticsearch-9.0.3/lib, --add-modules=jdk.net, --add-modules=jdk.management.agent, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
[2025-07-11T19:56:59,862][INFO ][o.e.b.Elasticsearch      ] [MyPC] Default Locale [en_US]
[2025-07-11T19:57:00,094][INFO ][o.e.n.j.JdkVectorLibrary ] [MyPC] vec_caps=1
[2025-07-11T19:57:00,101][INFO ][o.e.n.NativeAccess       ] [MyPC] Using native vector library; to disable start with -Dorg.elasticsearch.nativeaccess.enableVectorLibrary=false
[2025-07-11T19:57:00,124][INFO ][o.e.n.NativeAccess       ] [MyPC] Using [jdk] native provider and native methods for [Linux]
[2025-07-11T19:57:00,233][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [MyPC] Java vector incubator API enabled; uses preferredBitSize=256; FMA enabled
[2025-07-11T19:57:00,294][INFO ][o.e.b.Elasticsearch      ] [MyPC] Bootstrapping Entitlements
[2025-07-11T19:57:04,781][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [repository-url]
[2025-07-11T19:57:04,781][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [rest-root]
[2025-07-11T19:57:04,782][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-core]
[2025-07-11T19:57:04,782][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-redact]
[2025-07-11T19:57:04,782][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [ingest-user-agent]
[2025-07-11T19:57:04,783][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-async-search]
[2025-07-11T19:57:04,783][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-monitoring]
[2025-07-11T19:57:04,783][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [repository-s3]
[2025-07-11T19:57:04,783][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-analytics]
[2025-07-11T19:57:04,784][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-esql-core]
[2025-07-11T19:57:04,784][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-ent-search]
[2025-07-11T19:57:04,784][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-autoscaling]
[2025-07-11T19:57:04,785][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [lang-painless]
[2025-07-11T19:57:04,785][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-ml]
[2025-07-11T19:57:04,785][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [legacy-geo]
[2025-07-11T19:57:04,785][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [lang-mustache]
[2025-07-11T19:57:04,786][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [logsdb]
[2025-07-11T19:57:04,786][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-ql]
[2025-07-11T19:57:04,787][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [rank-rrf]
[2025-07-11T19:57:04,787][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [analysis-common]
[2025-07-11T19:57:04,787][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [health-shards-availability]
[2025-07-11T19:57:04,787][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [transport-netty4]
[2025-07-11T19:57:04,788][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [aggregations]
[2025-07-11T19:57:04,788][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [ingest-common]
[2025-07-11T19:57:04,788][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [frozen-indices]
[2025-07-11T19:57:04,789][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-identity-provider]
[2025-07-11T19:57:04,789][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-shutdown]
[2025-07-11T19:57:04,789][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-text-structure]
[2025-07-11T19:57:04,790][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [snapshot-repo-test-kit]
[2025-07-11T19:57:04,790][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [ml-package-loader]
[2025-07-11T19:57:04,790][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [kibana]
[2025-07-11T19:57:04,791][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [constant-keyword]
[2025-07-11T19:57:04,791][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-logstash]
[2025-07-11T19:57:04,791][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-graph]
[2025-07-11T19:57:04,791][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-ccr]
[2025-07-11T19:57:04,792][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [rank-vectors]
[2025-07-11T19:57:04,792][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-esql]
[2025-07-11T19:57:04,792][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [parent-join]
[2025-07-11T19:57:04,793][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [counted-keyword]
[2025-07-11T19:57:04,793][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-enrich]
[2025-07-11T19:57:04,794][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [repositories-metering-api]
[2025-07-11T19:57:04,794][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [transform]
[2025-07-11T19:57:04,794][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [repository-azure]
[2025-07-11T19:57:04,795][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [dot-prefix-validation]
[2025-07-11T19:57:04,795][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [repository-gcs]
[2025-07-11T19:57:04,795][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [spatial]
[2025-07-11T19:57:04,795][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-otel-data]
[2025-07-11T19:57:04,796][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [mapper-extras]
[2025-07-11T19:57:04,796][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [apm]
[2025-07-11T19:57:04,796][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [mapper-version]
[2025-07-11T19:57:04,796][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-rollup]
[2025-07-11T19:57:04,796][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [percolator]
[2025-07-11T19:57:04,797][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-migrate]
[2025-07-11T19:57:04,797][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-stack]
[2025-07-11T19:57:04,797][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [data-streams]
[2025-07-11T19:57:04,797][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [reindex]
[2025-07-11T19:57:04,797][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [rank-eval]
[2025-07-11T19:57:04,798][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-security]
[2025-07-11T19:57:04,798][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [blob-cache]
[2025-07-11T19:57:04,798][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [searchable-snapshots]
[2025-07-11T19:57:04,798][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-slm]
[2025-07-11T19:57:04,799][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-geoip-enterprise-downloader]
[2025-07-11T19:57:04,799][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [snapshot-based-recoveries]
[2025-07-11T19:57:04,799][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-watcher]
[2025-07-11T19:57:04,799][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [old-lucene-versions]
[2025-07-11T19:57:04,799][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-ilm]
[2025-07-11T19:57:04,800][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-voting-only-node]
[2025-07-11T19:57:04,800][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-inference]
[2025-07-11T19:57:04,800][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-deprecation]
[2025-07-11T19:57:04,800][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-fleet]
[2025-07-11T19:57:04,800][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-profiling]
[2025-07-11T19:57:04,801][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-aggregate-metric]
[2025-07-11T19:57:04,801][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-downsample]
[2025-07-11T19:57:04,801][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [ingest-geoip]
[2025-07-11T19:57:04,801][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-write-load-forecaster]
[2025-07-11T19:57:04,802][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [search-business-rules]
[2025-07-11T19:57:04,802][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [ingest-attachment]
[2025-07-11T19:57:04,802][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [wildcard]
[2025-07-11T19:57:04,802][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-apm-data]
[2025-07-11T19:57:04,803][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [unsigned-long]
[2025-07-11T19:57:04,803][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-sql]
[2025-07-11T19:57:04,803][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [runtime-fields-common]
[2025-07-11T19:57:04,803][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-async]
[2025-07-11T19:57:04,803][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [vector-tile]
[2025-07-11T19:57:04,803][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-kql]
[2025-07-11T19:57:04,804][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [lang-expression]
[2025-07-11T19:57:04,804][INFO ][o.e.p.PluginsService     ] [MyPC] loaded module [x-pack-eql]
[2025-07-11T19:57:06,869][INFO ][o.e.e.NodeEnvironment    ] [MyPC] using [1] data paths, mounts [[/ (/dev/sdb2)]], net usable_space [803.2gb], net total_space [914.7gb], types [ext4]
[2025-07-11T19:57:06,869][INFO ][o.e.e.NodeEnvironment    ] [MyPC] heap size [15.6gb], compressed ordinary object pointers [true]
[2025-07-11T19:57:06,954][INFO ][o.e.n.Node               ] [MyPC] node name [MyPC], node ID [JcGbkqqSRI25Ox1C_aM-vg], cluster name [elasticsearch], roles [ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold]
[2025-07-11T19:57:10,070][INFO ][o.e.i.r.RecoverySettings ] [MyPC] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2025-07-11T19:57:10,259][INFO ][o.e.f.FeatureService     ] [MyPC] Registered local node features [ES_V_8, ES_V_9, cluster.reroute.ignores_metric_param, cluster.stats.source_modes, linear_retriever_supported, lucene_10_1_upgrade, lucene_10_upgrade, security.queryable_built_in_roles, simulate.ignored.fields]
[2025-07-11T19:57:10,309][INFO ][o.e.c.m.DataStreamGlobalRetentionSettings] [MyPC] Updated default factory retention to [null]
[2025-07-11T19:57:10,310][INFO ][o.e.c.m.DataStreamGlobalRetentionSettings] [MyPC] Updated max factory retention to [null]
[2025-07-11T19:57:10,488][ERROR][o.e.b.Elasticsearch      ] [MyPC] fatal exception while booting Elasticsearch org.elasticsearch.ElasticsearchSecurityException: invalid configuration for xpack.security.transport.ssl - [xpack.security.transport.ssl.enabled] is not set, but the following settings have been configured in elasticsearch.yml : [xpack.security.transport.ssl.keystore.secure_password,xpack.security.transport.ssl.truststore.secure_password]
	at org.elasticsearch.xcore@9.0.3/org.elasticsearch.xpack.core.ssl.SSLService.validateServerConfiguration(SSLService.java:655)
	at org.elasticsearch.xcore@9.0.3/org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:629)
	at org.elasticsearch.xcore@9.0.3/org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:160)
	at org.elasticsearch.xcore@9.0.3/org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:495)
	at org.elasticsearch.xcore@9.0.3/org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:325)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.node.NodeConstruction.lambda$construct$17(NodeConstruction.java:902)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.plugins.PluginsService.lambda$flatMap$0(PluginsService.java:186)
	at java.base/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:289)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:636)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:656)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:662)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:667)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:924)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:293)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.node.Node.<init>(Node.java:185)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:426)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:426)
	at org.elasticsearch.server@9.0.3/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:102)

ERROR: Elasticsearch did not exit normally - check the logs at /home/myname/Projects/Playground/elasticsearch-9.0.3/logs/elasticsearch.log

ERROR: Elasticsearch died while starting up, with exit code 1
5

I know that

es_server.kill()

helps with the problem, but I cannot be always sure that the process is always stopped properly. I would like to know in general, why Elasticsearch is making such problems.

Btw. it is not an option to follow the book's code blindly, because it uses a very old version of haystack. So will have to adapt to newer versions anyway.

6

I just realized I made a stupid mistake, because my script contains the code

url = """https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.0.3-linux-x86_64.tar.gz"""
!wget -nc -q {url}
!tar -xzf elasticsearch-9.0.3-linux-x86_64.tar.gz

which is of course messing up the files, if I run the script again.
Sorry for the confusions I caused, but maybe this will help other people with a similar problem.

7

This is the actual error causing elasticsearch not to be able to start:

[2025-07-11T19:57:10,488]
[ERROR][o.e.b.Elasticsearch      ]
[MyPC] 
fatal exception while booting Elasticsearch org.elasticsearch.ElasticsearchSecurityException:
invalid configuration for xpack.security.transport.ssl
[xpack.security.transport.ssl.enabled] is not set, but the following settings have been configured in elasticsearch.yml : 
[xpack.security.transport.ssl.keystore.secure_password,
xpack.security.transport.ssl.truststore.secure_password]

What have you done to address the issue highlighted here?

Well, I wish you well, but I think you are likely making the journey harder for yourself. One suggestion is to get elasticsearch up and running, and leave it running, outside of the Jupyter notebooks. The specific python code thats starting/stopping elasticsearch in those notebooks is probably peripheral to the main purpose of the book.