Modern Plugin router, specify timeout

Elastic Stack Kibana
1

Hey guys, making a Kibana plugin.

I'm uploading a large-ish (think 100mb) file with HTTP2 multipart/form-data, and streaming the data into a GCP bucket. I was able to do this cleanly with the old plugin, with the following code as a Hapi.js route. I'm not able to do this with the new plugin, because it looks like Kibana is wrapping the Hapi.js a little bit, and some options appear to be different.

Here is the old, pure Hapi.js route definition:

  server.route({
    path: '/api/myPlugin/file/upload',
    method: 'POST',
    config: {
      validate: { payload: null }, // This was null due to memory issues.
      payload: {
        maxBytes: 1073741824,
        output: 'stream',
        parse: true,
      },
    },
    async handler(req: any, h: any): Promise<boolean> {
      try {
        await fileCon.uploadFile(req.payload);
        return true;
      } catch (e) {
        return h.response('Error updating File: ' + e.message).code(400);
      }
    },
  });

This seemed to work just fine. The request would go through, taking 20ish seconds to upload a binary that was about 100MB.

The new plugin framework has Kibana wrapping the Hapi.js router, and I'm not able to use this, and have to rewrite. I've been able to do this with all of my other routes, but I'm having an issue with this one.

 router.post({
    path: '/api/myPlugin/file/upload',
    validate: {
      body: schema.object({
        file: schema.stream(),
      }),
    },
    options: {
      body: {
        output: 'stream',
        parse: true,
        maxBytes: 1073741824,
      },
    },
  },
  async (ctx, req, res) => {
    try {
      const { file } = req.body;
      await fileCon.uploadFile(file);
      return res.ok({
        body: { success: true },
      });
    } catch (e) {
      return res.internalError({ body: e });
    }
  });

My client (we browser) uploads for 20-ish seconds, the Kibana rejects the request with a 408 Timeout. This happens before any of my server-side code is called.

Another sidenote, is that my plugin code would be called before the upload finished, whereas here, my code doesn't appear to be called until after the upload finished. I do not believe the router is properly streaming the content.

Is there a way with the new plugin structure that I can give a raw Hapi.js route definition like the first option to the plugin's router? Can i override timeout settings?

2

Do you need to parse payload before proxying?

Do you have the same problem if you switch to streaming?

router.post({
    path: '/api/myPlugin/file/upload',
    validate: {
      body: schema.stream(),
    },
    options: {
      body: {
        output: 'stream',
        parse: false,
        maxBytes: 1073741824,
      },
    },
  },
  async (ctx, req, res) => {
    try {
      const fileStream = req.body;
      await fileCon.uploadFile(fileStream);
      return res.ok({
        body: { success: true },
      });
    } catch (e) {
      return res.internalError({ body: e });
    }
  });
3

EDIT: For posterity's sake, the requirement on the 'kbn-sytem-request: true' isn't needed, and the corresponding line of code that verifies it exists isn't either.

Thanks for the reply, I think I've gotten it working well, baring some more tests, and this is very much so the direction I took. In the code examples I linked, there are actually 2 other items that are included in the FormData upload that I did need to parse. I move those to a separate call to happen just before this one, and it allowed me to not have to parse this call.

Moving forward, I needed a stream read from the request. If you walk down the fileCon.uploadFile(fileStream) function, it ends up piping a readstream (request form data file stream) into a write stream (gcp storage api). I ended up using the following approach.

router.post({
    path: '/api/myPlugin/file/upload/{id}', //ID is obtained from separating out a call, rather than later in this call.
    validate: {
      params: schema.object({
        id: schema.string(),
      }),
    },
    options: {
      body: {
        maxBytes: 1073741824,
        output: 'stream',
        parse: false, // changing this to false seems to have fixed the timeout, but made the other small form entries inaccessible, prompting me to move those to another api call.
      },
    },
  },
  async (ctx, req, res) => {
    try {
      // Client side fetch(..) is applying the 'kbn-system-request: true' header.
      // This gives me access to the raw 'req' object, as seen a few lines down.
      // I found it around kibana/src/core/public/http/fetch.ts on line 146 on main branch.
      if (!req.headers['kbn-system-request']) throw new Error('Client must include a kbn-system-request header as true.');

      // The raw handlers were stored on the 'req' passed into the handler on a Symbol key. (The only symbol key, I hope.)
      const reqSymbol = Object.getOwnPropertySymbols(req)[0];
      // @ts-ignore
      const { req: rawReq }: any = req[reqSymbol].raw;

      await fileCon.uploadFile(rawReq, req.params.id);

      return res.ok({
        body: { success: true },
      });
    } catch (e) {
      console.log(e.stack);
      return res.internalError({ body: e });
    }
  });

Down the line, in the fileCon.uploadFile(fileStream) I'm using the multiparty` package to parse the multiparm/form-data, which is wants the raw request object as its input.

Do you think using adding the following would remove the asSystemRequest, and multiparty dependency?

    validate: {
      body: schema.stream(),
    },
4

kbn-system-request just tells Kibana that it's not a user request.

validate: {
  body: schema.stream(),
},

Provides you access to request body stream, but not the whole raw request. Is it + Content-Type header sufficient for fileCon.uploadFile ?

IIRC if you specify

      options: {
        body: {
          maxBytes: maxImportPayloadBytes,
          output: 'stream',
          accepts: 'multipart/form-data',
        },
      },

The server will parse the payload

5

Two issues with that format:
1: Larger files (25MB, about 20 seconds to "upload" cause a 408 timeout. I'll also note that my breakpoint never gets hit. Normally when Its properly streaming, it will hit the breakpoint early into the upload, rather than later after the end.
2: When I use a smaller file (800 bytes) it doesn't have a timeout error, but fails the validation. Expected a stream, but recieved an object in the body.

#1 was fixed previously setting parse: false

I will say in testing this I found that you're right, the kbn-system-request: true isn't needed, and can be left out, while still getting the raw request object from the Symbol key in the KibanaRequest object.

6

Did your request have any headers yo enforce parsing? As a last resort, you can workaround payload validation with schema.any().

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.