[URGENT] Corpora Definiton , no base-url defined || Indexing Local dump.json

Elastic Stack Elasticsearch
,
1

Hi I am very new to esRally , I tried finding the similar topic but couldn't . Any help would be really appreciated.

So , I have hosted a local es on docker. version : 8.5.2 , single node .

I want to do benchmarking using my products.json , so bulk indexing and challenges like parallel bulk indexing to measure throughput.

here is track.sjon

    "version": 2,
    "description": "Tutorial benchmark for Rally",
    "indices": [
      {
        "name": "customlogs",
        "body": ""
      },
      {
        "name": "customgeo",
        "body": ""
      }
    ],
    "corpora": [
      {
        "name": "rally-tutorial",
        "documents": [
          {
            "source-file": "./geodocs.json",
            "document-count": 100000,
            "target-index": "customgeo"
          },
          {
            "source-file": "./geodocs.json",
            "document-count": 100000,
            "target-index": "customlogs"
          }
        ]
      }
    ],
    "schedule": [
      {
        "operation": {
          "operation-type": "create-index"
        }
      },
      {
        "operation": {
          "operation-type": "cluster-health",
          "request-params": {
            "wait_for_status": "yellow"
          }
        }
      },
      {
        "parallel": {
          "tasks": [
            {
              "name": "bulk1",
              "operation": {
                "operation-type": "bulk",
                "indices": "customgeo",
                "bulk-size": 100
              },
              "warmup-time-period": 0,
              "clients": 1,
              "target-throughput": 800
            },
            {
              "name": "bulk2",
              "operation": {
                "operation-type": "bulk",
                "indices": "customlogs",
                "bulk-size": 100
              },
              "warmup-time-period": 0,
              "clients": 1,
              "target-throughput": 800
            }
          ]
        }
      }
    ]
  }

here is the command i am running

//

esrally race --track=/Users/jarvis/GOSTOR/backend/clean_up/PRODUCT-STORE/GO_LIVE/rally-tracks/local-track --test-mode --target-hosts="127.0.0.1:9200" --kill-running-processes --pipeline=benchmark-only --report-file=~/Desktop/report.csv --report-format=csv

Screenshot 2023-02-02 at 2.20.43 PM
Screenshot 2023-02-02 at 2.20.43 PM1524×860 142 KB

Screenshot 2023-02-02 at 2.19.28 PM
Screenshot 2023-02-02 at 2.19.28 PM2676×1018 308 KB

2

Hello, and thank you for your interest in Rally!

You need to use --track-path instead of --track are you're passing a path and not a track name like geonames.

3

Thanks for the Reply .

Tried Both

esrally race --track-path=/Users/jarvis/GOSTOR/backend/clean_up/PRODUCT-STORE/GO_LIVE/rally-tracks/local-track/track.json --test-mode  --target-hosts="127.0.0.1:9200"   --kill-running-processes  --pipeline=benchmark-only --report-file=~/Desktop/report.csv --report-format=csv ```

esrally race --track-path=/Users/jarvis/GOSTOR/backend/clean_up/PRODUCT-STORE/GO_LIVE/rally-tracks/local-track --test-mode --target-hosts="127.0.0.1:9200" --kill-running-processes --pipeline=benchmark-only --report-file=~/Desktop/report.csv --report-format=csv ```

Why is it exepecting base-url , when i have defined corpora like this

      {
        "name": "rally-tutorial",
        "documents": [
          {
            "source-file": "./geodocs.json",
            "document-count": 100000,
            "target-index": "customgeo"
          },
          {
            "source-file": "./geodocs.json",
            "document-count": 100000,
            "target-index": "customlogs"
          }
        ]
      }
    ]

Still getting the same error :

Screenshot 2023-02-02 at 3.36.55 PM
Screenshot 2023-02-02 at 3.36.55 PM1538×882 149 KB

4

The issue is because you are using the test-mode. Please see Define Custom Workloads: Tracks - Rally 2.10.0 documentation you additionally need to create a -1k.json version of your corpus file.

We could improve our error here.

5

already tracked in test-mode support for source files with fewer than 1k documents · Issue #1354 · elastic/rally · GitHub

6

Thanks @dliappis & @Quentin_Pradet .

I followed How to write own track from scratch. That helped.

There is no good post / videos which explains esrally. I have started my journey will definitely make one which would atleast help the beginners to get the hands on.

Once again . Great work everyone . Would love to keep contributing one way or another.

7

Glad you got the issue sorted and thank you for the feedback @Sriram_Kumar . I agree that more entry-level enablement material is needed; the documentation is exhaustive but is more reference like and we seem to be missing onboarding material, in the form of blog articles or videos.

We welcome all contributions including areas related to documentation and improving the end user experience.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.