How to index a .json file located inside a persistent volume on a kubernetes cluster to Elasticsearch

Elastic Stack Beats
,
1

I have a code that collects data and dumps it in a .json file. In this file, every line is a json object. I was following this guide https://www.elastic.co/guide/en/cloud-on-k8s/current/index.html to run Elastic on kubernetes.
MY QUESTION:
I have a .json file inside a persistent volume path. I just want to get the data inside this file, indexed in Elasticsearch. All examples i'm seeing are for indexing the logs of containers ..

any help please ?

2

Welcome to our community! :smiley:

You will need to mount the persistent volume inside the pod and then tell Filebeat to read from that.

3

Hello Mark, thanks for helping.
Logically, that's the way to solve it. But if that's the yaml file i'm using for indexing the logs, what should I change to index my file.

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: quickstart
spec:
  type: filebeat
  version: 7.14.1
  elasticsearchRef:
    name: quickstart
  config:
    filebeat.inputs:
    - type: container
      paths:
      - /var/log/containers/*.log
  daemonSet:
    podTemplate:
      spec:
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.