# BigQuery to Elasticsearch Self-Managed configuration
**URL:** https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228
**Category:** Elasticsearch
**Created:** [October 20, 2021, 6:45pm UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228 "2021-10-20T18:45:30Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 20, 2021, 6:45pm UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/1 "2021-10-20T18:45:30Z")
Hi All,
Am trying to test out the steps given here: [https://www.elastic.co/blog/ingest-data-directly-from-google-bigquery-into-elastic-using-google-dataflow](https://www.elastic.co/blog/ingest-data-directly-from-google-bigquery-into-elastic-using-google-dataflow)
As per the documentation it says the implementation flow remains the same for " **The integration works for all users, regardless of whether you are using the Elastic Stack on Elastic Cloud, Elastic Cloud in the Google Cloud Marketplace, or a self-managed environment.**"
However when trying to use this dataflow template in big query dataflows it shows that the field Cloud API Key is mandatory. In that case how can we use this dataflow template for Self-Managed Instances with basic auth enabled. If anyone has been able to get this working please help.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 20, 2021, 7:48pm UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/2 "2021-10-20T19:48:56Z")
This is the API Key, it is not related to Elastic Cloud.
You can create an API Key on self-managed without any problems, just go on Kibana, Stack Management, Security, API Keys.
The guide you shared has a link to the part of the documentation on how you can create an API key.
> Fill in the required parameters, including your Cloud ID and Base64-encoded API Key for Elasticsearch. Cloud ID can be found from Elastic Cloud UI as shown below. **API Key can be created using the** [**Create API key API**](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html).
You just need to create an API Key and use it in that field.
---
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 20, 2021, 8:14pm UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/3 "2021-10-20T20:14:20Z")
HI,
I even tried by creating a trial elastic could id and then applying the bigquery template but I get the following error now.
[RuntimeException: org.apache.beam.sdk.util.UserCodeException: java.lang.IllegalArgumentException: Cannot get Elasticsearch version]
---
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 20, 2021, 8:16pm UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/4 "2021-10-20T20:16:39Z")
Also is there any option on the big query template to not use API Key and only use the basic auth creds.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 21, 2021, 1:07am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/5 "2021-10-21T01:07:43Z")
What version of Elasticsearch are you using?
---
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 21, 2021, 11:36am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/6 "2021-10-21T11:36:14Z")
7.10.2
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 21, 2021, 11:55am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/7 "2021-10-21T11:55:03Z")
Are you using the default distribution?
---
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 21, 2021, 11:55am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/8 "2021-10-21T11:55:39Z")
Yes.
---
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 25, 2021, 7:53am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/9 "2021-10-25T07:53:31Z")
Hi,
In this template it has a mandatory parameter for API key. however in the code it looks like its either api key or username/password. Please help on how to process with just using username/password as I cannot use the API key approach.
>
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 25, 2021, 8:05am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/10 "2021-10-25T08:05:11Z")
Out of curiosity, why can you not use the API key approach?
---
### Author:  [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)
#### Post date: [October 25, 2021, 8:52am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/11 "2021-10-25T08:52:41Z")
Am using the OSS build which is mapped behind our internal proxy using which we are creating the security layer and currently in our security layer we only support the Basic auth and don't yet have an API key mech implemented. Hence want to see if there is a way to get it working with Basic Auth.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 22, 2021, 8:52am UTC](https://discuss.elastic.co/t/bigquery-to-elasticsearch-self-managed-configuration/287228/12 "2021-11-22T08:52:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.