Use Environment Variables from Kubernetes Pod in Elastic Synthetics

Elastic Observability Synthetics
1

TL;DR

Can I use local environment variables from my execution environment in my synthetics journeys (e.g., from the Kubernetes pod where the test runs), and if so, how?

The Long Version

I am using @elastic/synthetics-1.4.0 to write user journeys and push them to our Elastic Synthetics environment, which runs in Elastic Cloud 8.10.3. I then schedule them to run in private locations, using the elastic-agent-complete:8.10.3 image.

When I use environment variables, using the process.env.VARIABLE format, it bundles the variables from the machine used to push the tests to Elastic Cloud. As far as I can tell, it does this by replacing references in to environment variables in synthetics.config.ts or in individual script files with strings.

The issue lies in using a CI/CD pipeline to push tests to Elastic. Environment variables are not usually defined in the CI/CD pipeline, so I would like to simply be able to define my environment variables as a Kubernetes secret, mount them in the pod, and use them in my scripts, similar to how I'm able to use environment variables in other Elastic Agent integrations.

A second issue with CI/CD deployments when using environment variables is that, as far as I can tell, if the value of the local environment variable changes, it does not appear to recognize that as a change in the script, so it will not push the script with the updated variable value. The workaround is to first delete the remote script, then deploy it again, which requires multiple pull requests and deployments, so it doesn't lend itself well to a CI/CD pipeline.

Another issue, unrelated to CI/CD, is that parameters don't seem to be usable in playwrightOptions, so when it is necessary to set playwrightOptions.httpCredentials, it must be done either using environment variables or by including the secret in synthetics.config.ts. However, Kibana shows playwrightOptions (including the password) as clear text, which is insecure, since anybody who has access to edit the script can then see the credentials.

My question is: Can I use local environment variables from my execution environment (e.g., from the Kubernetes pod where the test runs), and if so, how?

2

Hello @DougR ,

Are you trying to use env vars with browser monitors or with lightweight monitor?

Also have you considered using global params?

Let me see if I can find an example of using process env vars

Regards

3

So instead of using process.env in config file i think you should use process.env directly in the browser script that should work.

    step('Go to https://www.fast.com/', async () => {
      await page.goto('https://www.fast.com/');
      await page.fill('input[name="q"]', process.env.TZ);
    });
4

I'm using them with browser monitors. I would use globals, but anybody with all access to Synthetics can then also view the secrets, and users currently need this access in order to set alerts, so we don't currently use this in order to mitigate risk.

I'll look into this - thanks. But the one place where this doesn't work is when providing httpCredentials. These are set as part of playwrightOptions and can't be overridden at the journey level, AFAIK. Since they're part of playwrightOptions, they're visible in the clear in the playwrightOptions field when clicking on Edit in the Kibana Synthetics app. Is there any way to override the httpCredentials for an individual journey? This would have the additional benefit of being able to use different credentials for different journeys in the same project.

Thx.

5

it seems like httpCredentials are set when the context is created and that happens before journey call back is executed, So even though there is context available in the journey callback, it seems like it's not possible to set the httpCredentials within the journey.

Playwright used to have this API but it seems to have been deprecated

I see two ways here either we can streamline params permissions or we can allow some sort of API from journey which can interact with context creation which happens in synthetics runner.

Any ideas here @vigneshshanmugam would it be possible to pass httpCredentials to context creation?

6

@DougR actually there is work around you can use monitor.use API

ike this

  journey(`Recorded journey`, async ({ page, context }) => {
    monitor.use({
      id: 'my-browser',
      playwrightOptions:{
        httpCredentials:{
          username: process.env.USERNAME,
          password: process.env.PASSWORD
        }
      }
    });
    step('Go to https://www.fast.com/', async () => {
      await page.goto('https://www.fast.com/');
      await page.fill('input[name="q"]', process.env.TZ);
    });
  });

I hope this helps.

7

That helps greatly. That's actually one of my enhancement requests from many, many months ago, had no idea that it had been implemented. :rofl:

Edit

So...I just tried setting playwrightOptions and it failed. I'm still getting prompted by the browser for username/password while running it locally as a test.

8

@DougR.

We are mixing two scenarios here, I will try to go over them one by one

Running Locally

Playwright Options can be only set at the Runner level through

  • --playwright-options CLI flag
  • Synthetics config file.
npx @elastic/synthetics --playwright-options {}

Pushing Monitors/Running on service

Playwright options can be set at the Runner level and also on individual monitor level.

  • Can be set at the monitor.use({ playwrightOptions: {})
  • CLI flag
  • Synthetics config file.

Could you please let us know on which one you are trying out and getting blocked on?

Thanks,
Vignesh

9

Thank you for the clarification.

I attempted to use monitor.use({ playwrightOptions: {} }) locally on an individual monitor. It failed when running on a local service. I did not realize that this would not work locally, so I tested it locally and as noted, it failed.

Is there documentation on this somewhere? When I look at the documentation, I'm not able to find anything regarding setting playwrightOptions for an individual journey. I will take a look.

Thx.

10

Thank you - works exactly as described. Unfortunately, it still includes the httpCredentials in the playwrightOptions field in Kibana in the clear.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.