Elasticsearch/Node.js load tests giving poor results, how can I lower latency on my queries?

Elastic Stack Elasticsearch
1

Hi, I'm new to Elasticsearch and have set up Elasticsearch locally to run some tests. This is more of an experiment to look at the capabilities of ES and is not meant to be a production application. However, I my results after running K6.io came out what I think to be slow (compared to a MySQL database), so wanted to share the results and see if anyone has ideas on how to improve.

It is also possible that this is not a good use case for Elasticsearch, or my single node configuration in my local computer does not have enough computing power to handle the load.

Context
Testing ES as a data store for a Node/Express app. The app is backend-only at this stage (not connected to frontend yet). The app is supposed to be an image gallery for a vacation home website that has listings and each listing has photos. I use ES to store photo objects in the following schema:

{
  index: 'photos',
  body: {
    properties: {
      url_path: { type: 'text' },
      caption: { type: 'text' },
      space_type: { type: 'keyword' },
      order: { type: 'integer' },
      is_main: { type: 'boolean' },
      listing_id: { type: 'integer' },
    }
  }
}

The Express app has endpoints set up so that it can query for photos that belong to a certain listing_id. For example:

GET localhost:9200/photos/_search
query: {
  match: { listing_id: listingId }
}

Currently, I'm only doing GET requests.

K6.io results
In K6, I'm asking 100 virtual users to make requests with a sleep time of 1 second, for 30 seconds. That is 100 requests per second and 3,000 total requests in 30 seconds time.

execution: local
output: -
script: script.js

duration: 30s, iterations: -
vus: 100, max: 100

done [==========================================================] 30s / 30s

✓ status was 200
✗ transaction time OK
 ↳  58% — ✓ 1349 / ✗ 947

checks.....................: 79.37% ✓ 3645  ✗ 947
data_received..............: 5.4 MB 179 kB/s
data_sent..................: 244 kB 8.1 kB/s
http_req_blocked...........: avg=114.08µs min=2µs    med=6µs      max=6.4ms  p(90)=15µs     p(95)=36.25µs
http_req_connecting........: avg=83.2µs   min=0s     med=0s       max=5.46ms p(90)=0s       p(95)=0s
http_req_duration..........: avg=324.78ms min=5.6ms  med=155.34ms max=2.14s  p(90)=832.53ms p(95)=1.69s
http_req_receiving.........: avg=96.09µs  min=34µs   med=85µs     max=1.81ms p(90)=133µs    p(95)=157µs
http_req_sending...........: avg=38.2µs   min=9µs    med=22µs     max=5.65ms p(90)=43µs     p(95)=65µs
http_req_tls_handshaking...: avg=0s       min=0s     med=0s       max=0s     p(90)=0s       p(95)=0s
http_req_waiting...........: avg=324.65ms min=5.46ms med=155.24ms max=2.14s  p(90)=832.36ms p(95)=1.69s
http_reqs..................: 2296   76.532668/s
iteration_duration.........: avg=1.33s    min=1s     med=1.16s    max=3.15s  p(90)=1.84s    p(95)=2.71s
iterations.................: 2196   73.199364/s
vus........................: 100    min=100 max=100
vus_max....................: 100    min=100 max=100

Elasticsearch index settings and cluster status

"photos": {
  "settings": {
    "index": {
      "creation_date": "1582519161348",
      "number_of_shards": "1",
      "number_of_replicas": "1",
      "refresh_interval" : "30s",
      "uuid": "NFIrfHrnQDK8W1Tn3sHMng",
      "version": {
        "created": "7060099"
      },
      "provided_name": "photos"
    }
  }
}

# GET /_cluster/health?wait_for_status=yellow&timeout=50s
{
  "cluster_name" : "elasticsearch_gabrielsong",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 7,
  "active_shards" : 7,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 6,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 53.84615384615385
}

cluster         status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
elasticsearch   yellow          1         1      7   7    0    0        6             0                  -                 53.8%

ip        heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1           27          99  31    3.05                  dilm      *      MacBook-Air.local

System
MacBook Air (Retina, 13-inch, 2019)
1.6 GHz Dual-Core Intel Core i5
8 GB 2133 MHz LPDDR3
128 GB Flash Storage (with 43 GB available)
Intel UHD Graphics 617 1536 MB

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.