Minor update:
I tried the suggestion to prepend setting the environmental variables to
map $hostname to a variable at the beginning of an elasticsearch config
file and it doesn't work. No error, it just doesn't result in expected
functionality.
So,
Am going to add CentOS and Ubuntu machine to my cluster and see if there is
any diff. since CentOS still uses SystemVinit and Ubuntu uses Upstart for
subsystems management.
In the meantime, for the life of me, if the Elasticsearch coding is
correct, I can't imagine how a Java app shouldn't be able to read
systemwide variables into its own configuration, particularly if the build
(eg rpm or deb) obviously targets a Linux system. So, although I can also
briefly mention that openSUSE (like some other distros like Mandriva) uses
an update-alternatives utility to manage configuration and management of
the Java environment(including easily switching between different Java
installed on the same system), I still would assume that if Elasticsearch
is coded properly that shouldn't make a diff because I doubt that node
names are variables that would be scoped to only Java use.
If anyone can point to the general part of the elasticsearch code that is
supposed to interpret, use and/or dynamically generate the strings that are
used for node identification, I'd be willing to take a brief look to see if
there might be anything obvious that might be implemented differently.
In the meantime, will be testing the other machines I mentioned I'm adding.
If I see the same behavior using the pre-built packages, then I'll likely
try building from source if that's how you're successfully getting this to
work (or are the tar you're using simply jar files?).
Tony
On Tuesday, December 24, 2013 12:28:09 PM UTC-8, Karol Gwaj wrote:
yep, my setup is a little bit different from yours
i actually downloaded elasticsearch as tar.gz file
probably the problem you are experiencing is related more to the fact that
systemctl is not passing environment variables to executed script
i had similar problem with upstart (which is equivalent of systemctl on
ubuntu)
from what i know about systemctl, the way to define environment variables
that will be passed to executed script is through EnvironmentFile
so maybe try to define NAME environment variable in
/etc/sysconfig/elasticsearch (EnvironmentFile )
from your service unit file i see that elasticsearch startup script is
located in: /usr/share/elasticsearch/bin/elasticsearch
if you add definition of your environment variable at the beginning of
this file then you will not have to worry about systemctl not passing
environment variables to your script
so try something like that first:
export NAME=$HOSTNAME
echo $NAME # this should
print your hostname
/usr/share/elasticsearch/bin/elasticsearch
On Tuesday, December 24, 2013 4:24:02 PM UTC, Tony Su wrote:
OK,
I ran each of your tests, but because I'm invoking elasticsearch as a
service and not from the CLI, I interactively ran the command you wanted
prepended, then started the elasticsearch service, then reloaded
elasticsearch-head pointing to the elasticsearch node.
So, as follows, I first disabled the service so it doesn't start
automatically. After each block below I ran elasticsearch-head with the
same results, a random "friendly" node name was created in
elasticsearch-head
echo $NAME
/bin/hostname
$NAME
ELASTICSEAR-1
systemctl stop elasticsearch.service
export NAME=hostname
systemctl start elasticsearch.service
systemtcl stop elasticsearch.service
export NAME=$HOSTNAME
systemctl start elasticsearch.service
systemctl stop elasticsearch.service
export NAME=$hostname
systemctl start elasticsearch.service
For your reference is the contents of the elasticsearch.service Unit file
(aka service configuration file). Although it references many exterior
files (the -Des commands), I doubt in this case anything in them are likely
to be relevant because we seem to be dealing with a variable (if supported)
is a null value.
[Unit]
Description=Starts and stops a single elasticsearch instance on this
system
Documentation=http://www.elasticsearch.org
[Service]
Type=forking
EnvironmentFile=/etc/sysconfig/elasticsearch
User=elasticsearch
Group=elasticsearch
PIDFile=/var/run/elasticsearch/elasticsearch.pid
ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p
/var/run/elasticsearch/elasticsearch.pid -Des.default.config=$CONF_FILE
-Des.default.path.home=$ES_HOME -Des.default.path.logs=$LOG_DIR
-Des.default.path.data=$DATA_DIR -Des.default.path.work=$WORK_DIR
-Des.default.path.conf=$CONF_DIR
See MAX_OPEN_FILES
LimitNOFILE=65535
See MAX_LOCKED_MEMORY, use "infinity" when MAX_LOCKED_MEMORY=unlimited
and using bootstrap.mlockall: true
#LimitMEMLOCK=infinity
[Install]
WantedBy=multi-user.target
A thought, if you're running the same version as me, I wonder if there
might be a difference between the RPM build (which I am using) and yours
which of course is a DEB build unless you built from source. I'm using the
RPM downloaded directly through the Elasticsearch website.
Tony
On Monday, December 23, 2013 6:16:45 PM UTC-8, Karol Gwaj wrote:
Im using elasticsearch version 0.90.7 (which is this same as the one you
mentioned in your question)
Also im running my elasticsearch cluster on ubuntu, so my example was
more suited for this linux distribution
and coming back to your problem, to diagnose it you can:
- add echo $NAME statement at the beginning of bin/elasticsearch
script (if nothing is printed then your environment variable is not
declared correctly)
- add export NAME=
hostname at the beginning of bin/elasticsearch
script
- add export NAME=$HOSTNAME at the beginning of bin/elasticsearch
script
can you try the steps above first (one at the time), before defining
your environment variable in /etc/profile.local
Cheers,
On Monday, December 23, 2013 9:52:40 PM UTC, Tony Su wrote:
After considering this post,
I successfully created an environmental variable by adding to the bash
profile file (actually on the openSUSE I'm running, I created a file
/etc/profile.local which contains system customizations, the original
/etc/profile should not be edited). BTW - on a non-Windows box, "hostname"
must be in lower case, not upper case.
/etc/profile.local
export NAME=/bin/hostname
After running "source /etc/profile.local" to activate the contents of
the file I can successfully test the new variable, it does return the
machine's hostname.
$NAME
But, when I modify the elasticsearch.yml file exactly as described
node.name http://node.name/: ${NAME}
The result is that the elasticsearch service fails to start with the
following error:
ELASTICSEAR-1 systemd[1]: Starting Starts and stops a single
elasticsearch instance on this system...
Dec 23 13:23:18 ELASTICSEAR-1 systemd[1]: PID file
/var/run/elasticsearch/elasticsearch.pid not readable (yet?) after start.
Dec 23 13:23:43 ELASTICSEAR-1 systemd[1]: Started Starts and stops a
single elasticsearch instance on this system.
Dec 23 13:23:43 ELASTICSEAR-1 systemd[1]: elasticsearch.service: main
process exited, code=exited, status=3/NOTIMPLEMENTED
Dec 23 13:23:43 ELASTICSEAR-1 systemd[1]: Unit elasticsearch.service
entered failed state.
I also tried without the curly braces, but then the string is read
literally and not as a variable.
Commenting out the attempt to set the node name to the hostname allows
the elasticsearch service to start again.
From the above error(not implemented), is it possible that the current
stable elasticsearch release does not support your recommendation and I
need to maybe install an unstable version?
Thx,
Tony
On Sunday, December 22, 2013 4:53:48 PM UTC-8, Karol Gwaj wrote:
Take a look at default elasticsearch.yml:
https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml
especially this part:
Any element in the configuration can be replaced with environment variables
by placing them in ${...} notation. For example:
node.rack: ${RACK_ENV_VAR}
so for example to accomplish this using environment variables:
-
edit elasticsearch.yml:
node.name: ${NAME}
-
create custom bash script:
#!/bin/bash
create NAME environment variable
export NAME=$HOSTNAME
start search node
/path/to/elasticsearch/bin/elasticsearch
instead of creating custom bash script for starting elasticsearch you can modify default elasticsearch startup script (bin/elasticsearch) by adding export NAME=$HOSTNAME at the begining of the script
On Sunday, December 22, 2013 9:41:13 PM UTC, Tony Su wrote:
Thx jorg, unfortunately your suggestions only configure actual
strings, like my original example I'm hoping to automatically retrieve and
insert the hostname of the machine. BTW if your suggestion is used in a
config I've determined the "./bin/elasticsearch" should be omitted.
Karol,
That looks closer to what I'm looking for but I'm uncertain if the
variable can be setup at the beginning of the config file or can somehow be
run as part of the ES_JAVA_OPTS entry. Or, should the variable be setup as
a variable when defining java variables, eg in a profile? I'm also
surprised by your use of angle brackets, should they be used as you
describe or do they only represent a general grouping of the statement?
Thx,
Tony
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/898e884b-cd8d-46f7-ae0a-ad6420acab91%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.