# Can't understand the problem from the log

**URL:** https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033
**Category:** Elasticsearch
**Tags:** rally
**Created:** [April 9, 2019, 1:26pm UTC](https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033 "2019-04-09T13:26:52Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Mikhail\_Khludnev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikhail_khludnev/32/59591_2.png) [@Mikhail\_Khludnev](https://discuss.elastic.co/u/Mikhail_Khludnev)
#### Post date: [April 9, 2019, 1:26pm UTC](https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033/1 "2019-04-09T13:26:52Z")

</div>

I'm trying to run

> $ esrally --track-path=track.json --preserve-install true --car="4gheap" --client-options="timeout:6000" --distribution-version=7.0.0-rc2

Here's what I have in log [https://paste.apache.org/0wRo](https://paste.apache.org/0wRo)  
Something happen, but I can't grasp what.  
Elastic log is absent. Then when I'me trying to launch elastic I've got a culprit.

> :~/.rally/benchmarks/races/2019-04-09-12-22-24/rally-node-0/install/elasticsearch-7.0.0-rc2/bin$ ./elasticsearch --verbose  
> OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.  
> ERROR: Temporary file directory [${ES\_TMPDIR}] does not exist or is not accessible

Here's my java

> :~/.rally/benchmarks/races/2019-04-09-12-22-24/rally-node-0/install/elasticsearch-7.0.0-rc2/bin$ $JAVA\_HOME/bin/java --version  
> openjdk 12 2019-03-19  
> OpenJDK Runtime Environment AdoptOpenJDK (build 12+33)  
> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 12+33, mixed mode, sharing)

Checking tmp dir write access.

> :~/.rally/benchmarks/races/2019-04-09-12-22-24/rally-node-0/install/elasticsearch-7.0.0-rc2/bin$ cat \> ${ES\_TMPDIR}/o  
> sss

> :~/.rally/benchmarks/races/2019-04-09-12-22-24/rally-node-0/install/elasticsearch-7.0.0-rc2/bin$ file ${ES\_TMPDIR}/o  
> /tmp/o: ASCII text, with no line terminators

How I can understand what's going on from rally logs? Do you know how to deal with that ES\_TMPDIR failure?

---

<div class="post-metadata">

### Author: ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)
#### Post date: [April 9, 2019, 2:31pm UTC](https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033/2 "2019-04-09T14:31:08Z")

</div>

Hi Mikhail,

Unfortunately you are hitting: [https://github.com/elastic/elasticsearch/issues/40812](https://github.com/elastic/elasticsearch/issues/40812)

Rally picks up the configuration option from the [rally-teams repo](https://github.com/elastic/rally-teams/blob/ad04992ac6bdc57ca644e70c345b13b45da0f26b/cars/v1/vanilla/templates/config/jvm.options#L72) (7 branch) which contains the right content for `-Djava.io.tmpdir` (without double quotes) for the latest 7.0.0. The `7.0.0-rc2` artifact, that Rally downloads, requires double quotes for this setting.

Since you are having a local setup of Rally and Elasticsearch you could also switch to the [from-sources-complete](https://esrally.readthedocs.io/en/stable/pipelines.html?highlight=from%20source%20complete#from-sources-complete) pipeline and pick one of the commits of the [Elasticsearch 7.0 branch](https://github.com/elastic/elasticsearch/commits/7.0) using `--revision=` until 7.0.0 is GA. This way Rally will build Elasticsearch locally (just make sure you have the right [JAVAx\_HOME env vars set](https://esrally.readthedocs.io/en/stable/install.html?highlight=JAVAx_HOME#jdk) see also [here](https://github.com/elastic/elasticsearch/blob/7.0/CONTRIBUTING.md#contributing-to-the-elasticsearch-codebase) -- for Elasticsearch 7 you'll need JAVA12\_HOME to build and if you want to use a different jvm to run Elasticsearch with, set `JAVAx_HOME` accordingly and specify the major version using `--runtime-jdk`).

Regards,  
Dimitris

---

<div class="post-metadata">

### Author: ![Mikhail\_Khludnev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikhail_khludnev/32/59591_2.png) [@Mikhail\_Khludnev](https://discuss.elastic.co/u/Mikhail_Khludnev)
#### Post date: [April 9, 2019, 7:41pm UTC](https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033/3 "2019-04-09T19:41:28Z")

</div>

Ok. Thanks, @dliappis. It clarifies the issue. But how we can make it easier understandable from rally.log? Can we make it log stderr from elastic?

---

<div class="post-metadata">

### Author: ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)
#### Post date: [April 10, 2019, 10:06am UTC](https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033/4 "2019-04-10T10:06:41Z")

</div>

This should be present in rally.log. We reproduced this using:

`esrally --distribution-version=7.0.0-rc2 --test-mode --challenge=append-no-conflicts-index-only`

and could see the following in rally.log

```auto
2019-04-10 10:03:51,772 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO Allowed JDK versions are [12, 11, 8].
2019-04-10 10:03:51,949 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO Detected JDK with major version [12] in [/Library/Java/JavaVirtualMachines/jdk-12.jdk/Contents/Home].
2019-04-10 10:03:51,949 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO Starting node [rally-node-0] based on car [defaults].
2019-04-10 10:03:52,3 ActorAddr-(T|:52550)/PID:41926 esrally.utils.process INFO b'openjdk version "12-ea" 2019-03-19\n'
2019-04-10 10:03:52,4 ActorAddr-(T|:52550)/PID:41926 esrally.utils.process INFO b'OpenJDK Runtime Environment (build 12-ea+24)\n'
2019-04-10 10:03:52,4 ActorAddr-(T|:52550)/PID:41926 esrally.utils.process INFO b'OpenJDK 64-Bit Server VM (build 12-ea+24, mixed mode, sharing)\n'
2019-04-10 10:03:52,8 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO Setting [-XX:+ExitOnOutOfMemoryError] to detect out of memory errors during the benchmark.
2019-04-10 10:03:52,409 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO rally-node-0 (stdout): OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
2019-04-10 10:03:53,488 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO rally-node-0 (stdout): ERROR: Temporary file directory [${ES_TMPDIR}] does not exist or is not accessible
2019-04-10 10:03:53,570 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher INFO rally-node-0 (stdout): No more output. Process has likely terminated.
2019-04-10 10:03:53,675 ActorAddr-(T|:52550)/PID:41926 esrally.mechanic.launcher ERROR Node [rally-node-0] has terminated with exit code [78].
2019-04-10 10:03:53,697 ActorAddr-(T|:52524)/PID:41787 esrally.actor INFO Received a benchmark failure from [ActorAddr-(T|:52537)] and will forward it now.

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 8, 2019, 10:06am UTC](https://discuss.elastic.co/t/cant-understand-the-problem-from-the-log/176033/5 "2019-05-08T10:06:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
