# Esrally run failed for "Cannot find documents-2.json.bz2" **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [June 23, 2018, 7:55am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109 "2018-06-23T07:55:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 23, 2018, 7:55am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/1 "2018-06-23T07:55:32Z")
it report errors as following: * * * \*\*\*\*\*\* Use this pipeline only if you are aware of the tradeoffs. \*\*\*\*\*\* \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* Watch your step! \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* * * * [INFO] Racing on track [geonames], challenge [append-no-conflicts-index-only] and car ['external'] with version [2.3.5]. [WARNING] refresh\_total\_time is 2 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading. [WARNING] flush\_total\_time is 4 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading. [ERROR] Cannot race. Error in track preparator Cannot find /home/.rally/benchmarks/data/geonames/documents-2.json.bz2. Please disable offline mode and retry again. Getting further help: * * * - Check the log files in /root/.rally/logs for errors. - Read the documentation at [https://esrally.readthedocs.io/en/0.11.0/](https://esrally.readthedocs.io/en/0.11.0/) - Ask a question on the forum at [https://discuss.elastic.co/c/elasticsearch/rally](https://discuss.elastic.co/c/elasticsearch/rally) - Raise an issue at [https://github.com/elastic/rally/issues](https://github.com/elastic/rally/issues) and include the log files in /root/.rally/logs. however, there is documents.json.bz2 file: [root@hdh154 /home/.rally/benchmarks/data/geonames]# ll total 193224 -rwxrwxrwx 1 root root 197857614 Jun 23 15:37 documents.json.bz2 could you help to resolve it please? thank you so much ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 23, 2018, 7:56am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/2 "2018-06-23T07:56:53Z")
esrally command: esrally --track=geonames --target-hosts=10.3.68.168:9200,10.3.68.172:9200,10.3.68.174:9200 --challenge=append-no-conflicts-index-only --pipeline=benchmark-only --offline ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 23, 2018, 8:02am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/3 "2018-06-23T08:02:14Z")
If you are using it in offline mode, you need to [download all the data the track uses](https://esrally.readthedocs.io/en/0.11.0/offline.html?highlight=offline#using-tracks) before you run it. ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 25, 2018, 1:03am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/4 "2018-06-25T01:03:19Z")
I have download them, and put them under another path: [root@hdh154 ~/.rally/benchmarks/tracks/default/geonames]# ll total 660 drwxrwxrwx 2 root root 4096 Jun 20 11:27 challenges -rwxrwxrwx 1 root root 44 Jun 20 10:39 files.txt -rw-r--r-- 1 root root 2685 Jun 20 10:39 index.json drwxrwxrwx 2 root root 4096 Jun 20 10:39 operations -rw-r--r-- 1 root root 2061 Jun 20 10:39 README.md -rw-r--r-- 1 root root 642669 Jun 20 10:39 terms.txt -rwxrwxrwx 1 root root 1196 Jun 20 14:13 track.json -rw-r--r-- 1 root root 4192 Jun 20 10:39 track.py Is there some problem? ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 25, 2018, 6:11am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/5 "2018-06-25T06:11:47Z")
Hi, you have: > [@Jessica254](#): > > [root@hdh154 /home/.rally/benchmarks/data/geonames]# ll > total 193224 > -rwxrwxrwx 1 root root 197857614 Jun 23 15:37 documents.json.bz2 However, the error message says: > [@Jessica254](#): > > Cannot find /home/.rally/benchmarks/data/geonames/documents-2.json.bz2. Please disable offline mode and retry again. So there is a file called `documents.json.bz2` but Rally expects `documents-2.json.bz2`. Renaming the file won't help because it seems you downloaded and older version (the expected size is 264698741 bytes but yours has 197857614 bytes). How did you download the file? Did you use the `download.sh` script in [`rally-tracks`](https://github.com/elastic/rally-tracks) as suggested in the [docs](http://esrally.readthedocs.io/en/stable/offline.html#using-tracks)? Daniel ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 25, 2018, 8:07am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/6 "2018-06-25T08:07:23Z")
thank you for your reply. Cause our server can't connect to network, so where can I get the documents-2.json.bz2 manually please? Maybe I can try to replace the the documents.json.bz2 to documents-2.json.bz2, and try again. ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 25, 2018, 8:40am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/7 "2018-06-25T08:40:48Z")
I have tried download the bz2 file by download.sh, it failed: [root@hdh154 /home]# ./download.sh geonames : No such file or directory ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 25, 2018, 10:41am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/8 "2018-06-25T10:41:29Z")
Hi, > [@Jessica254](#): > > Cause our server can't connect to network, so where can I get the documents-2.json.bz2 manually please? That is what the `download.sh` script is for. It will download all track-related files that you need to run in offline mode. > [@Jessica254](#): > > Maybe I can try to replace the the documents.json.bz2 to documents-2.json.bz2, and try again. As I have explained in my first answer, this will not help you. This is the wrong file to begin with. The file length does not match and also it will have the wrong file structure. > [@Jessica254](#): > > I have tried download the bz2 file by download.sh, it failed: > > [root@hdh154 /home]# ./download.sh geonames > : No such file or directory Is there a `download.sh` file in that directory? If not, please follow the [instructions in the docs](http://esrally.readthedocs.io/en/stable/offline.html#using-tracks). Daniel ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 25, 2018, 10:49am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/9 "2018-06-25T10:49:57Z")
I have got the file, however, still some error: [INFO] Racing on track [geonames], challenge [append-no-conflicts-index-only] and car ['external'] with version [2.3.5]. [INFO] Decompressing track data from [/home/.rally/benchmarks/data/geonames/documents-2.json.bz2] to [/home/.rally/benchmarks/data/geonames/documents-2.json] (resulting size: 3.30 GB) ... [ERROR] Cannot race. Error in track preparator Invalid data stream ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 25, 2018, 12:04pm UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/10 "2018-06-25T12:04:58Z")
Hi, this sounds like the downloaded file is corrupted. Can you show the output of `ls -la /home/.rally/benchmarks/data/geonames`? Thanks. Daniel ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 26, 2018, 1:24am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/11 "2018-06-26T01:24:10Z")
It seems all right. [root@hdh154 ~/.rally/benchmarks/tracks/default/geonames]# ls -la /home/.rally/benchmarks/data/geonames total 456028 drwxrwxrwx 2 root root 4096 Jun 25 19:01 . drwxr-xr-x 5 root root 4096 Jun 20 14:03 .. -rw-r--r-- 1 root root 4403200 Jun 25 19:10 documents-2.json -rw-r--r-- 1 root root 264698741 Jun 25 18:49 documents-2.json.bz2 -rwxrwxrwx 1 root root 197857614 Jun 23 15:37 documents.json.bz2 ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 26, 2018, 6:21am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/12 "2018-06-26T06:21:25Z")
Hi, indeed, the file size of `documents-2.json.bz2` seems fine. Can you please delete the partially uncompressed file `documents-2.json` (i.e. `rm -f documents-2.json`) and retry? Daniel ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 26, 2018, 6:37am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/13 "2018-06-26T06:37:22Z")
Also failed: [INFO] Racing on track [geonames], challenge [append-no-conflicts-index-only] and car ['external'] with version [2.3.5]. [INFO] Decompressing track data from [/home/.rally/benchmarks/data/geonames/documents-2.json.bz2] to [/home/.rally/benchmarks/data/geonames/documents-2.json] (resulting size: 3.30 GB) ... [WARNING] refresh\_total\_time is 36 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading. [WARNING] flush\_total\_time is 33 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading. [ERROR] Cannot race. Error in track preparator Invalid data stream ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 26, 2018, 7:36am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/14 "2018-06-26T07:36:25Z")
after run the command, it generated json file again [root@hdh154 ~/.rally/benchmarks/tracks/default/geonames]# ls /home/.rally/benchmarks/data/geonames documents-2.json documents-2.json.bz2 ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 26, 2018, 8:08am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/15 "2018-06-26T08:08:17Z")
Can you please do the following? 1. Run `md5sum documents-2.json.bz2` and paste the result. The fingerprint should be `c6fbf5e7b20c3c46f4cd6ab8385a9cb7`. 2. Remove `documents-2.json` once more with `rm -f documents-2.json` 3. Attempt to decompress it yourself with `bzip2 -dk documents-2.json.bz2`. Note that this requires the `bzip2` tool which you might need to install separately. ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 26, 2018, 8:39am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/16 "2018-06-26T08:39:28Z")
result as following: [root@hdh154 /home/.rally/benchmarks/data/geonames]# md5sum documents-2.json.bz2 67f10be0f09ae6820d314a9c795fc5a7 documents-2.json.bz2 [root@hdh154 /home/.rally/benchmarks/data/geonames]# bzip2 -dk documents-2.json.bz2 bzip2: Data integrity error when decompressing. Input file = documents-2.json.bz2, output file = documents-2.json It is possible that the compressed file(s) have become corrupted. You can use the -tvv option to test integrity of such files. You can use the `bzip2recover' program to attempt to recover data from undamaged sections of corrupted files. bzip2: Deleting output file documents-2.json, if it exists. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 26, 2018, 10:16am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/17 "2018-06-26T10:16:48Z")
It seems that, for whatever reason, `documents-2.json.bz2` is broken. You can redownload it with the following command: ```auto curl -o documents-2.json.bz2 http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents-2.json.bz2 ``` (this is actually what the `download.sh` script does as well...) Please verify integrity by checking the checksum right after downloading: ```auto md5sum documents-2.json.bz2 ``` which should print `c6fbf5e7b20c3c46f4cd6ab8385a9cb7`. ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 27, 2018, 1:05am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/18 "2018-06-27T01:05:38Z")
> [@danielmitterdorfer](#): > > curl -o documents-2.json.bz2 [http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents-2.json.bz2](http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents-2.json.bz2) my server can't recognize the link, so I download the bz2 file manually by browser, is it OK? [root@hdh154 /home/.rally/benchmarks/data/geonames]# curl -o documents-2.json.bz2 [http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents-2.json.bz2](http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents-2.json.bz2) curl: (6) Couldn't resolve host '[benchmarks.elasticsearch.org.s3.amazonaws.com](http://benchmarks.elasticsearch.org.s3.amazonaws.com)' ---
**Author:** ![Jessica254](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jessica254/32/32909_2.png) [@Jessica254](https://discuss.elastic.co/u/Jessica254)\ **Post date:** [June 27, 2018, 3:31am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/19 "2018-06-27T03:31:14Z")
I've downloaded it through web browser, and here's output: [root@hdh154 /home/.rally/benchmarks/data/geonames]# md5sum documents-2.json.bz2 3682050a7eb9dc53ba923379bc4a4ea3 documents-2.json.bz2 it seems still not right. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 27, 2018, 5:10am UTC](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109/20 "2018-06-27T05:10:50Z")
Downloading via a browser is fine as well. I just don't understand why the artefact is corrupted on your machine. I tried that exact command now on several machines with the same (successful) result. Are you behind a proxy? Do you have an opportunity to download the file from a machine that is not behind that proxy? [Next page](https://discuss.elastic.co/t/esrally-run-failed-for-cannot-find-documents-2-json-bz2/137109.md?page=2)