Lerna output json

[cskiwi (Glenn Latomme (Kiwi))]

Hi,

I'm trying to setup my logging, for my application, but this is running via lerna.
Which means it outputs first the package name and then the log

Which makes the output look like this:

info cli using local version of lerna
lerna notice cli v4.0.0                                                                                                                                                                                                                                                                                              
lerna notice filter including "@badvlasim/server"                                                                                                                                                                                                                                                                    
lerna info filter [ '@badvlasim/server' ]                                                                                                                                                                                                                                                                            
lerna info Executing command in 1 package: "yarn run start"                                                                                                                                                                                                                                                          
@badvlasim/server: yarn run v1.22.5                                                                                                                                                                                                                                                                                  
@badvlasim/server: $ node -r ts-node/register ./src/server.ts                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
@badvlasim/server: {"@timestamp":"2021-09-10T08:13:05.185Z","log.level":"info","message":"\tRunning migration","ecs":{"version":"1.6.0"},"event":{"dataset":"badman-server.log"},"service":{"name":"badman-server"},"timestamp":"2021-09-10T08:13:05.183Z"}                                                          
@badvlasim/server: {"log.level":"error","@timestamp":"2021-09-10T08:13:05.264Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"metadata requests did not finish, possible deadlock"}                                                                                                                                                                                                                                                                                                                                        
@badvlasim/server: {"@timestamp":"2021-09-10T08:13:06.817Z","log.level":"info","message":"\t🚀 badman-server listening on the port 4000","ecs":{"version":"1.6.0"},"event":{"dataset":"badman-server.log"},"service":{"name":"badman-server"},"timestamp":"2021-09-10T08:13:06.815Z"}  

I was wondering on how I should setup filebeat to work with this?

[ChrsMark (Chris Mark)]

Hi!

Can you redirect this output to a file maybe? Then you can set filebeat to collect logs from this file.

C.

[cskiwi (Glenn Latomme (Kiwi))]

UPDATE: found a way to omit the @badvlasim/server so now the logs are:

info cli using local version of lerna                                                                                                                                      
lerna notice cli v4.0.0                                                                                                                                                    
lerna notice filter including "@badvlasim/server"                                                                                                                          
lerna info filter [ '@badvlasim/server' ]                                                                                                                                  
lerna info Executing command in 1 package: "yarn run start"                                                                                                                
yarn run v1.22.5                                                                                                                                                           
$ node -r ts-node/register ./src/server.ts                                                                                                                                                                                                                                                                             
{"@timestamp":"2021-09-10T11:33:52.938Z","log.level":"info","message":"\tRunning migration","ecs":{"version":"1.6.0"},"event":{"dataset":"badman-server.log"},"service":{" ││ name":"badman-server"},"timestamp":"2021-09-10T11:33:52.936Z"}                                                                                                 
{"@timestamp":"2021-09-10T11:33:54.501Z","log.level":"info","message":"\t🚀 badman-server listening on the port 4000","ecs":{"version":"1.6.0"},"event":{"dataset":"badman ││ -server.log"},"service":{"name":"badman-server"},"timestamp":"2021-09-10T11:33:54.500Z"}

and I've updated the config to the following:

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeat
  namespace: elastic
spec:
  type: filebeat
  version: 7.14.1
  elasticsearchRef:
    name: elastic-search
  kibanaRef:
    name: elastic-kibana
  config:
    filebeat:
      autodiscover:
        providers:
          - type: kubernetes
            namespace: badvla
            node: ${NODE_NAME}
            templates:
              - config:
                  - type: container
                    co.elastic.logs/json.keys_under_root: true
                    co.elastic.logs/json.overwrite_keys: true
                    co.elastic.logs/json.add_error_key: true
                    co.elastic.logs/json.expand_keys: true
                    include_lines: ['^{']
                    paths:
                      - /var/log/containers/*-${data.container.id}.log
    processors:
      - add_host_metadata: {}
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: filebeat
        automountServiceAccountToken: true
        terminationGracePeriodSeconds: 30
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        containers:
          - name: filebeat
            securityContext:
              runAsUser: 0
            volumeMounts:
              - name: varlogcontainers
                mountPath: /var/log/containers
              - name: varlogpods
                mountPath: /var/log/pods
              - name: varlibdockercontainers
                mountPath: /var/lib/docker/containers
            env:
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
        volumes:
          - name: varlogcontainers
            hostPath:
              path: /var/log/containers
          - name: varlogpods
            hostPath:
              path: /var/log/pods
          - name: varlibdockercontainers
            hostPath:
              path: /var/lib/docker/containers

I don't see the error anymore, but I don't see any logs appearing