Esrally is now report support es7?

Elastic Stack Elasticsearch
1

esrally is now support es7?
when i use esrally report to elasticsearch
esrally test with http_logs can't put mapping
in es7 log

[2019-04-18T15:44:07,457][DEBUG][o.e.a.a.i.t.p.TransportPutIndexTemplateAction] [node-3] failed to put template [rally-metrics]
org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [_default_ : {_source={enabled=true}, dynamic_templates=[{strings={mapping={type=keyword}, match_mapping_type=string, match=*}}], _all={enabled=false}, properties={max={type=float}, relative-time={type=long}, environment={type=keyword}, unit={type=keyword}, @timestamp={format=epoch_millis, type=date}, min={type=float}, task={type=keyword}, median={type=float}, trial-timestamp={format=basic_date_time_no_millis, type=date, fields={raw={type=keyword}}}, car={type=keyword}, trial-id={type=keyword}, mean={type=float}, operation-type={type=keyword}, name={type=keyword}, challenge={type=keyword}, lap={type=short}, track={type=keyword}, job={type=keyword}, value={type=float}, operation={type=keyword}, sample-type={type=keyword}}}]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:394) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:323) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.metadata.MetaDataIndexTemplateService.validateAndAddTemplate(MetaDataIndexTemplateService.java:253) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.metadata.MetaDataIndexTemplateService.access$300(MetaDataIndexTemplateService.java:65) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.metadata.MetaDataIndexTemplateService$2.execute(MetaDataIndexTemplateService.java:176) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:687) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:310) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:210) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:142) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-7.0.0.jar:7.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root mapping definition has unsupported parameters:  [_default_ : {_source={enabled=true}, dynamic_templates=[{strings={mapping={type=keyword}, match_mapping_type=string, match=*}}], _all={enabled=false}, properties={max={type=float}, relative-time={type=long}, environment={type=keyword}, unit={type=keyword}, @timestamp={format=epoch_millis, type=date}, min={type=float}, task={type=keyword}, median={type=float}, trial-timestamp={format=basic_date_time_no_millis, type=date, fields={raw={type=keyword}}}, car={type=keyword}, trial-id={type=keyword}, mean={type=float}, operation-type={type=keyword}, name={type=keyword}, challenge={type=keyword}, lap={type=short}, track={type=keyword}, job={type=keyword}, value={type=float}, operation={type=keyword}, sample-type={type=keyword}}}]
        at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:152) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:140) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:92) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:392) ~[elasticsearch-7.0.0.jar:7.0.0]
2

in esrally log is

2019-04-18 07:44:07,518 -not-actor-/PID:74196 esrally.rally INFO Actor system is still running. Waiting...
2019-04-18 07:44:08,520 -not-actor-/PID:74196 esrally.rally INFO Shutdown completed.
2019-04-18 07:44:08,520 -not-actor-/PID:74196 esrally.rally ERROR Cannot run subcommand [race].
Traceback (most recent call last):
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/rally.py", line 475, in dispatch_sub_command
    race(cfg)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/rally.py", line 404, in race
    with_actor_system(lambda c: racecontrol.run(c), cfg)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/rally.py", line 425, in with_actor_system
    runnable(cfg)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/rally.py", line 404, in <lambda>
    with_actor_system(lambda c: racecontrol.run(c), cfg)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/racecontrol.py", line 417, in run
    raise e
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/racecontrol.py", line 414, in run
    pipeline(cfg)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/racecontrol.py", line 78, in __call__
    self.target(cfg)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/racecontrol.py", line 351, in benchmark_only
    return race(cfg, external=True)
  File "/usr/local/python3/lib/python3.7/site-packages/esrally/racecontrol.py", line 303, in race
    raise exceptions.RallyError(result.message, result.cause)
esrally.exceptions.RallyError: ("Error in race control (('A transport error occurred while running the operation [put_template] against your Elasticsearch metri
cs store on host [node-5] at port [19200].', None))", None)
3

Hey,

what version of esrally are you using? Just run esrally --version

--Alex

4 
[root@xxxxdefault]# esrally --version
esrally 1.0.4
5

need change the mapping.

/usr/local/python3/lib/python3.7/site-packages/esrally/resources/metrics-template.json

I find this file with long time..

6

but the data can't into..

[2019-04-19T10:37:57,061][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [node-3] failed to put mappings on indices [[[rally-metrics-2019-04/L_hDBYxpSzeAI2425W5Y_g]]], type [metrics]
java.lang.IllegalArgumentException: Rejecting mapping update to [rally-metrics-2019-04] as the final mapping would have more than 1 type: [_doc, metrics]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:449) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:398) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:331) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:315) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:238) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:687) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:310) ~[elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:210) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:142) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-7.0.0.jar:7.0.0]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-7.0.0.jar:7.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
7

Hello,

This has been implemented in https://github.com/elastic/rally/pull/654, but as we haven't released 1.1.0 yet, you should either stay tuned for a new release soon or install Rally from source.

Cheers,
Dimitris

8

this esrally 1.0.4 is not support es 5.6.4?
it with offline install.
sorry cant use 1.1.0. wating for you.

9

esrally 1.0.4 should be fine with ES metric store 5.6.x

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.