Problem fixing index to realize a scroll with Typescript, node and Express

Elastic Stack Elasticsearch
1

Hello,

I'm new to elastic and I have an issue using scroll method.

I've started the development of a small project in Typescript to complete some of Kibana's features and to simplify some actions on the front-end.

I use the Javascript client of Elasticsearch to search my data and I have a problem to use the scroll method.

I can't set the right index but I know I have the right technique because in the DevTools console of Kibana I get the right result.

In DevTools I have the following queries that work (I execute the first request which is a search and then the second request which is a scroll by providing the scrollId returned by the first request):

GET my_index-*/_search?scroll=1m
{
  "query": {
   "bool": {
      "filter": [{
        "match": {
          "field":"data"
        }
        },{
        "range": {
          "@timestamp": {
            "gte": "2020-05-08T10:00:00.100Z",
            "lte": "2022-05-11T10:00:00.200Z",
            "format": "strict_date_optional_time"
          }
        }
      }]
    }
  },
  "_source": {
    "includes": ["some_field1", "some_field2"]
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ], 
  "size": 100
}

GET _search/scroll
{
  "scroll": "1m",
  "scroll_id":"DnF1ZXJ5VG ... NtUHdsQQ=="
}

These two requests return the expected result.

My backend is in Typescript and uses NodeJs with Express.

The first request, the search, works normally and returns me the right result. It is this code :

some import 
...

const client = new Client({
  node: configElasticClient.node
})

export const searchRouter = express.Router();

searchRouter.get('/search', async (req, res) => {
  const response = await client.search<SearchResponse<HitResult>>({
    index: "my_index-*",
    scroll: "1m",
    body: {
      query: {
        "bool": {
           "filter": [{
             "match": {
               "field_to_search":"data_to_search"
             }
             },{
            "range": {
              "@timestamp": {
                "gte": "2020-05-08T10:00:00.100Z",
                "lte": "2022-05-11T10:00:00.200Z",
                "format": "strict_date_optional_time"
             }
             }
           }]
         }
       },
      _source: ["some_field1", "some_field2"],
      sort: [
        {
          "@timestamp": {
            order: "desc"
          }
        }
      ], 
      size: 100
    }
  });
  console.log("result: ", response);
  res.json(response);
});

Then, when I make the scroll I should change my index to: "_search/scroll". But no matter what I try, when I try to access my route using scroll I get a "Cannot GET (the route)" error.

Here is the code of this second route (I put in comment the other way tried):

searchRouter.get('/search/scrollnext/:scrollId', async (req, res) => {
  const response = await client.scroll<SearchResponse<HitResult>>({
    scroll_id: req.params.scrollId,
    scroll: "1m",
    //index: "_search/scroll", wherever I put this field I get an error because the scroll method has no index field defined, of course I try without trying to implement a new index but I get an error when accessing my route on browser
  });

  //I also try this way :
  //const response = await client.scroll<SearchResponse<HitResult>>({
  //  method: 'GET',
  //  body: {
  //    scroll_id: req.params.scrollId,
  //    scroll: "1m",
  //}});

  console.log("result: ", response);
  res.json(response);
});

I get my scroll_id as a parameter from the first request which I fill in my second request in the scroll function.

In my browser I execute the search request then I paste the scrollId given in my route like that:
http://localhost:9000/sessions/search/scrollnext/DnF1 ... NtUHdsQQ==

I also try with quote like that:
http://localhost:9000/sessions/search/scrollnext/"DnF1 ... NtUHdsQQ=="

But I get a message error in every case:
Cannot GET /sessions/search/scrollnext/DnF1ZXJ ... tUHdsQQ==

I think that my problem comes from the scroll index which must be wrong. Does anyone know how to fix the index used by the scroll method?

Any help would be greatly appreciated :slight_smile:

2

Finally find the answer, the syntax was good and it wasn't an index problem, I had to add an async management layer in my request like this:

searchRouter.get('/path/to/route/:scrollId', (asyncHandler(async (req, res) => {
  const response = await client.scroll<YourDataType>({
    scroll_id: req.params.scrollId,
    scroll: "1m",
  });
  res.json(response);
})));

with asyncHandler looking something like this:

export function asyncHandler(fct: some_interface_to_define_request_caracteristic): RequestHandler {
    return (req, res, next) => {
       //Here add a try/catch layer
    }
}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.