Automate setting fleet outputs to remote_elasticsearch in kibana.yml

[0lzi (Oli)]

Hi, Im currently trying to automate kibana setup using ansible.

I can configure the outputs and the fleetServerHosts, but the agents do not come online until i go into the UI to add the service token to the outputs.

current kibana.yml

...
..
.
# Fleet Outputs - Works but needs api/fleet/service_token added as secret                                                                                                                     
xpack.fleet.outputs:                                                                                                                                                                          
  - id: elastic-cluster                                                                                                                                                                       
    name: es-cluster                                                                                                                                                                          
    type: remote_elasticsearch                                                                                                                                                                
    is_default: true                                                                                                                                                                          
    is_default_monitoring: true                                                                                                                                                               
    secrets:                                                                                                                                                                                  
      service_token:                                                                                                                                                                          
        id: "token-1709812465197"                                                                                                                                                             
    hosts:                                                                                                                                                                                    
      - https://es1.example.com:9201                                                                                                                                                
      - https://es2.example.com:9202                                                                                                                                                
      - https://es3.example.com:9203                                                                                                                                                
                                             

xpack.fleet.fleetServerHosts:
  - id: fleet-server
    name: Fleet Server
    is_default: true
    host_urls: [https://fleet.example.com:8220]

I am using ansible to get the service token, and i get a response like :

        "json": {
            "name": "token-1709812465197",
            "value": "AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyLXJlbW90ZS90b2tlbi0xNzA5ODEyNDY1MTk3OmlCNWExV2VrUm1taV9pckw5UUF4VWc"
        },

Is there a way in the kibana.yml to specify the service token thats generated without putting the token value in?

[0lzi (Oli)]

To add to this, If i set these outputs and the fleetServerHosts, the fleet.packages and agent.policies do not seem to be created correctly.

xpack.fleet.packages:
  - name: fleet_server
    version: latest
  - name: system
    version: latest
  - name: elastic_agent
    version: latest
xpack.fleet.agentPolicies:
  - name: Fleet Server policy
    id: fleet-server-policy
    namespace: default
    package_policies:
      - name: fleet_server-1
        package:
          name: fleet_server
  - name: Agent policy 1
    id: agent-policy-1
    namespace: default
    monitoring_enabled:
      - logs
      - metrics
    package_policies:
      - name: system-1
        id: default-system
        package:
          name: system

xpack.fleet.outputs:
  - id: elastic-cluster
    name: es-cluster 
    type: remote_elasticsearch 
    secrets:
      service_token: "AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyLXJlbW90ZS90b2tlbi0xNzA5ODk3MTgyMTA0Ond6LURUaTFDU2VPR29Uc1JrSkRBQ1E"
    hosts:
      - https://es1.example.com:9201
      - https://es2.example.com:9202
      - https://es3.example.com:9203


xpack.fleet.fleetServerHosts:
  - id: fleet-server
    name: Fleet Server
    is_default: true
    host_urls: [https://fleet.example.com:8220]

[Julia_Bardi (Júlia Bárdi)]

Hi Oli, welcome to the community!

You can add the service token value directly in the config like this, and it will be stored as a secret in kibana.

xpack.fleet.outputs:
  - id: elastic-cluster                                                                                                                                                                       
    name: es-cluster                                                                                                                                                                          
    type: remote_elasticsearch                                                                                                                                                                
    is_default: true                                                                                                                                                                          
    is_default_monitoring: true                                                                                                                                                                      
    hosts:                                                                                                                                                                                    
      - https://es1.example.com:9201                                                                                                                                                
      - https://es2.example.com:9202                                                                                                                                                
      - https://es3.example.com:9203 
    secrets:
      service_token: AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyLXJlbW90ZS90b2tlbi0xNzA5ODEyNDY1MTk3OmlCNWExV2VrUm1taV9pckw5UUF4VWc

Reference: [Fleet] Fix preconfigured remote ES outputs with secrets by jillguyonnet · Pull Request #172550 · elastic/kibana · GitHub