Filebeat failed to load

shubham.s · November 24, 2022, 11:46am

● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/usr/lib/systemd/system/filebeat.service; disabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Thu 2022-11-24 17:09:22 IST; 7min ago
     Docs: https://www.elastic.co/beats/filebeat
  Process: 15858 ExecStart=/usr/share/filebeat/bin/filebeat --environment systemd $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS (code=exited, status=1/FAILURE)
 Main PID: 15858 (code=exited, status=1/FAILURE)

Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: filebeat.service: main process exited, code=exited, status=1/FAILURE
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: Unit filebeat.service entered failed state.
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: filebeat.service failed.
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: filebeat.service holdoff time over, scheduling restart.
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: start request repeated too quickly for filebeat.service
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: Unit filebeat.service entered failed state.
Nov 24 17:09:22 vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local systemd[1]: filebeat.service failed.

less /etc/filebeat/filebeat.yml

filebeat.inputs:
- type: log
  id: my-filestream-id
  enabled: true
  paths:
    - /opt/elk/log/elasticsearch/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

logging.level: debug
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0640

output.logstash:
  hosts: ["localhost:5044"]
  type: log
  enabled: true
  paths: /opt/elk/log/elasticsearch/*.json
  worker: 1

less {filebeat_home}/module/logstash/module.yml

- module: logstash
  log:
    enabled: true
    var.paths: ["/var/log/logstash/logstash.log*"]
  slowlog:
    enabled: true
    var.paths: ["/var/log/logstash/logstash-slowlog.log*"]

vinit0711 · November 25, 2022, 7:42am

Could you please show the filebeat logs in /var/log/filebeat

shubham.s · November 25, 2022, 8:01am

{"log.level":"info","@timestamp":"2022-11-25T13:26:40.005+0530","log.origin":{"file.name":"instance/beat.go","file.line":708},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.005+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":766},"message":"Beat metadata path: /var/lib/filebeat/meta.json","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.005+0530","log.origin":{"file.name":"instance/beat.go","file.line":716},"message":"Beat ID: ba31b2f8-fc09-4359-b6af-87b2f6e044a6","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.007+0530","log.logger":"conditions","log.origin":{"file.name":"conditions/conditions.go","file.line":98},"message":"New condition contains: map[]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.007+0530","log.logger":"conditions","log.origin":{"file.name":"conditions/conditions.go","file.line":98},"message":"New condition !contains: map[]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.008+0530","log.logger":"docker","log.origin":{"file.name":"docker/client.go","file.line":49},"message":"Docker client will negotiate the API version on the first request.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.008+0530","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":130},"message":"add_cloud_metadata: starting to fetch metadata, timeout=3s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.009+0530","log.logger":"add_docker_metadata","log.origin":{"file.name":"add_docker_metadata/add_docker_metadata.go","file.line":91},"message":"add_docker_metadata: docker environment not detected: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.009+0530","log.logger":"kubernetes","log.origin":{"file.name":"add_kubernetes_metadata/kubernetes.go","file.line":148},"message":"Could not create kubernetes client using in_cluster config: unable to build kube config due to error: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable","service.name":"filebeat","libbeat.processor":"add_kubernetes_metadata","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":167},"message":"add_cloud_metadata: received disposition for digitalocean after 1.148412ms. result=[provider:digitalocean, error=failed with http status code 404, metadata={}]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":167},"message":"add_cloud_metadata: received disposition for gcp after 1.255484ms. result=[provider:gcp, error=failed with http status code 404, metadata={}]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":167},"message":"add_cloud_metadata: received disposition for aws after 1.277405ms. result=[provider:aws, error=<nil>, metadata={\"cloud\":{\"account\":{\"id\":\"252760256550\"},\"availability_zone\":\"ap-south-1b\",\"image\":{\"id\":\"ami-0cf81108a24f29c5a\"},\"instance\":{\"id\":\"i-09b08eb42aabf17d6\"},\"machine\":{\"type\":\"m6a.2xlarge\"},\"provider\":\"aws\",\"region\":\"ap-south-1\",\"service\":{\"name\":\"EC2\"}}}]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":133},"message":"add_cloud_metadata: fetchMetadata ran for 1.323225ms","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":106},"message":"add_cloud_metadata: hosting provider type detected as aws, metadata={\"cloud\":{\"account\":{\"id\":\"252760256550\"},\"availability_zone\":\"ap-south-1b\",\"image\":{\"id\":\"ami-0cf81108a24f29c5a\"},\"instance\":{\"id\":\"i-09b08eb42aabf17d6\"},\"machine\":{\"type\":\"m6a.2xlarge\"},\"provider\":\"aws\",\"region\":\"ap-south-1\",\"service\":{\"name\":\"EC2\"}}}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"processors","log.origin":{"file.name":"processors/processor.go","file.line":121},"message":"Generated new processors: add_host_metadata=[netinfo.enabled=[true], cache.ttl=[5m0s]], condition=!contains: map[], add_cloud_metadata={\"cloud\":{\"account\":{\"id\":\"252760256550\"},\"availability_zone\":\"ap-south-1b\",\"image\":{\"id\":\"ami-0cf81108a24f29c5a\"},\"instance\":{\"id\":\"i-09b08eb42aabf17d6\"},\"machine\":{\"type\":\"m6a.2xlarge\"},\"provider\":\"aws\",\"region\":\"ap-south-1\",\"service\":{\"name\":\"EC2\"}}}, add_docker_metadata=[match_fields=[] match_pids=[process.pid, process.parent.pid]], add_kubernetes_metadata","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":117},"message":"Loading syscall filter","service.name":"filebeat","seccomp_filter":{"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","chmod","chown","clock_gettime","clone","clone3","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchmodat","fchown","fchownat","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rseq","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1082},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"ba31b2f8-fc09-4359-b6af-87b2f6e044a6"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1091},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"f81376bad511929eb90d584d2059c4c8a41fc691","libbeat":"8.5.1","time":"2022-11-09T15:54:41.000Z","version":"8.5.1"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.010+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1094},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":8,"version":"go1.18.7"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.011+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1098},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-11-10T16:39:39+05:30","containerized":false,"name":"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local","ip":["127.0.0.1/8","::1/128","10.90.70.73/24","fe80::6563:c9d6:a4d:c355/64"],"kernel_version":"3.10.0-1160.76.1.el7.x86_64","mac":["0a:b8:f4:b4:83:34"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"IST","timezone_offset_sec":19800,"id":"7859246703b04a8781dc1c81420825c5"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.012+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1127},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null},"cwd":"/","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":7213,"ppid":1,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-11-25T13:26:39.730+0530"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.012+0530","log.origin":{"file.name":"instance/beat.go","file.line":294},"message":"Setup Beat: filebeat; Version: 8.5.1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.013+0530","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":319},"message":"Initializing output plugins","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.877+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/consumer.go","file.line":98},"message":"start pipeline event consumer","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.877+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.877+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/queue_reader.go","file.line":49},"message":"pipeline event consumer queue reader: start","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-11-25T13:26:40.878+0530","log.origin":{"file.name":"beater/filebeat.go","file.line":162},"message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":144},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.878+0530","log.origin":{"file.name":"instance/beat.go","file.line":471},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"test","log.origin":{"file.name":"registrar/migrate.go","file.line":287},"message":"isFile(/var/lib/filebeat/registry) -> false","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"test","log.origin":{"file.name":"registrar/migrate.go","file.line":287},"message":"isFile() -> false","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"test","log.origin":{"file.name":"registrar/migrate.go","file.line":280},"message":"isDir(/var/lib/filebeat/registry/filebeat) -> true","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"test","log.origin":{"file.name":"registrar/migrate.go","file.line":287},"message":"isFile(/var/lib/filebeat/registry/filebeat/meta.json) -> true","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.878+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/migrate.go","file.line":82},"message":"Registry type '1' found","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.883+0530","log.origin":{"file.name":"memlog/store.go","file.line":134},"message":"Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=194","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-11-25T13:26:40.884+0530","log.origin":{"file.name":"beater/filebeat.go","file.line":288},"message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","service.name":"filebeat","ecs.version":"1.6.0"}

shubham.s · November 25, 2022, 8:03am

{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform request:append","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform request:delete","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform request:set","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform response:append","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform response:delete","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform response:set","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform pagination:append","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform pagination:delete","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform pagination:set","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":83},"message":"registering encoder 'application/json': returned error: <nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":86},"message":"registering encoder 'application/x-www-form-urlencoded': returned error: <nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":92},"message":"registering decoder 'application/json': returned error: <nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":95},"message":"registering decoder 'application/x-ndjson': returned error: <nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":98},"message":"registering decoder 'text/csv': returned error: <nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":101},"message":"registering decoder 'application/zip': returned error: <nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.884+0530","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.id filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":140},"message":"Starting Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"cfgwarn","log.origin":{"file.name":"log/input.go","file.line":90},"message":"DEPRECATED: Log input. Use Filestream input instead.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"input","log.origin":{"file.name":"log/config.go","file.line":207},"message":"recursive glob enabled","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":186},"message":"exclude_files: []. Number of states: 1","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"input","log.origin":{"file.name":"file/states.go","file.line":68},"message":"New state added for /opt/elk/log/elasticsearch/VF-ELK.log","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"acker","log.origin":{"file.name":"beater/acker.go","file.line":59},"message":"stateful ack","service.name":"filebeat","count":1,"ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":226},"message":"Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:0001-01-01 00:00:00 +0000 UTC Meta:null Fields:null Private:{Id:native::7501-39 PrevId: Finished:true Fileinfo:<nil> Source:/opt/elk/log/elasticsearch/VF-ELK.log Offset:0 Timestamp:2022-11-25 13:26:39.758078508 +0530 IST TTL:-1ns Type:log Meta:map[] FileStateOS:7501-39 IdentifierName:native} TimeSeries:false}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":262},"message":"Processing 1 events","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":216},"message":"input with previous states loaded: 1","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":229},"message":"Registrar state updates processed. Count: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":172},"message":"Configured paths: [/opt/elk/log/elasticsearch/*.log]","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":252},"message":"Registrar states cleaned up. Before: 1, After: 1, Pending: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.885+0530","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 5209091122028175690)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.886+0530","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/reload.go","file.line":132},"message":"Checking module configs from: /etc/filebeat/modules.d/*.yml","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.886+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":223},"message":"Start next scan","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.886+0530","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/cfgfile.go","file.line":194},"message":"Load config from file: /etc/filebeat/modules.d/logstash.yml","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.886+0530","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/cfgfile.go","file.line":194},"message":"Load config from file: /etc/filebeat/modules.d/system.yml","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.887+0530","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/reload.go","file.line":146},"message":"Number of module configs found: 2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.887+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":473},"message":"Check file for harvesting: /opt/elk/log/elasticsearch/VF-ELK.log","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.887+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":571},"message":"Update existing file for harvesting: /opt/elk/log/elasticsearch/VF-ELK.log, offset: 0","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.887+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":580},"message":"Resuming harvesting of file: /opt/elk/log/elasticsearch/VF-ELK.log, offset: 0, new size: 23285159","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.888+0530","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: logstash (log), logstash (slowlog)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":597},"message":"Setting offset to: 0","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":583},"message":"Setting offset: 0 ","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":658},"message":"newLogFileReader with config.MaxBytes: 10485760","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":213},"message":"Harvester setup successful. Line terminator: 1","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"acker","log.origin":{"file.name":"beater/acker.go","file.line":59},"message":"stateful ack","service.name":"filebeat","count":1,"ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":226},"message":"Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:0001-01-01 00:00:00 +0000 UTC Meta:null Fields:null Private:{Id:native::7501-39 PrevId: Finished:false Fileinfo:0xc000073110 Source:/opt/elk/log/elasticsearch/VF-ELK.log Offset:0 Timestamp:2022-11-25 13:26:40.887491106 +0530 IST m=+0.947848949 TTL:-1ns Type:log Meta:map[] FileStateOS:7501-39 IdentifierName:native} TimeSeries:false}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":492},"message":"Update state (offset: 0).","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":287},"message":"input states cleaned up. Before: 1, After: 1, Pending: 0","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":262},"message":"Processing 1 events","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":229},"message":"Registrar state updates processed. Count: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":310},"message":"Harvester started for paths: [/opt/elk/log/elasticsearch/*.log]","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.889+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.889Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"log\": {\n    \"offset\": 0,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  },\n  \"message\": \"[2022-11-25T00:00:02,461][WARN ][o.e.h.AbstractHttpServerTransport] [ELK-1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/10.90.70.73:9200, remoteAddress=/10.90.70.40:59768}\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false\n  },\n  \"agent\": {\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\"\n  },\n  \"cloud\": {\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    }\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.889Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"cloud\": {\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    }\n  },\n  \"log\": {\n    \"offset\": 232,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  },\n  \"message\": \"io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"host\": {\n    \"os\": {\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\"\n  },\n  \"agent\": {\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"input","log.origin":{"file.name":"log/config.go","file.line":207},"message":"recursive glob enabled","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.890Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"message\": \"\\tat io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:480) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\"\n  },\n  \"agent\": {\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\"\n  },\n  \"cloud\": {\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    }\n  },\n  \"log\": {\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    },\n    \"offset\": 348\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.890Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"host\": {\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\"\n  },\n  \"agent\": {\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\"\n  },\n  \"cloud\": {\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\"\n  },\n  \"log\": {\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    },\n    \"offset\": 445\n  },\n  \"message\": \"\\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":186},"message":"exclude_files: [(?-s:.)gz(?-m:$)]. Number of states: 1","service.name":"filebeat","input_id":"27a03f86-a927-4dc2-ab7c-faff774f9560","ecs.version":"1.6.0"}

shubham.s · November 25, 2022, 8:03am

{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":216},"message":"input with previous states loaded: 0","service.name":"filebeat","input_id":"27a03f86-a927-4dc2-ab7c-faff774f9560","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":172},"message":"Configured paths: [/var/log/logstash/logstash.log*]","service.name":"filebeat","input_id":"27a03f86-a927-4dc2-ab7c-faff774f9560","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.890Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"host\": {\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\"\n    }\n  },\n  \"agent\": {\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"cloud\": {\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\"\n  },\n  \"log\": {\n    \"offset\": 543,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  },\n  \"message\": \"\\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]\"\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.890Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"log\": {\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    },\n    \"offset\": 659\n  },\n  \"message\": \"\\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"os\": {\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ]\n  },\n  \"agent\": {\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\"\n  },\n  \"cloud\": {\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    }\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.890+0530","log.logger":"input","log.origin":{"file.name":"log/config.go","file.line":207},"message":"recursive glob enabled","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.891Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"message\": \"\\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\"\n  },\n  \"agent\": {\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\"\n  },\n  \"cloud\": {\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\"\n  },\n  \"log\": {\n    \"offset\": 775,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":186},"message":"exclude_files: [(?-s:.)gz(?-m:$)]. Number of states: 1","service.name":"filebeat","input_id":"6fe674c4-7df7-4d6f-b8cc-e85cf050a54b","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":216},"message":"input with previous states loaded: 0","service.name":"filebeat","input_id":"6fe674c4-7df7-4d6f-b8cc-e85cf050a54b","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":172},"message":"Configured paths: [/var/log/logstash/logstash-slowlog.log*]","service.name":"filebeat","input_id":"6fe674c4-7df7-4d6f-b8cc-e85cf050a54b","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.891Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"log\": {\n    \"offset\": 889,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  },\n  \"message\": \"\\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"agent\": {\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ]\n  },\n  \"cloud\": {\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.891Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"message\": \"\\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]\",\n  \"log\": {\n    \"offset\": 998,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  },\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\"\n  },\n  \"agent\": {\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\"\n  },\n  \"cloud\": {\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.891Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"message\": \"\\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ]\n  },\n  \"agent\": {\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\"\n  },\n  \"cloud\": {\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    }\n  },\n  \"log\": {\n    \"offset\": 1114,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.891+0530","log.origin":{"file.name":"beater/crawler.go","file.line":155},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.891+0530","log.origin":{"file.name":"beater/crawler.go","file.line":165},"message":"Stopping 1 inputs","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.891Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"message\": \"\\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"agent\": {\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\",\n      \"name\": \"CentOS Linux\"\n    },\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ]\n  },\n  \"cloud\": {\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    },\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    }\n  },\n  \"log\": {\n    \"offset\": 1230,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":170},"message":"Stopping input: 5209091122028175690","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.891+0530","log.origin":{"file.name":"input/input.go","file.line":134},"message":"input ticker stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.891+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":618},"message":"Stopping harvester.","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.892+0530","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-11-25T07:56:40.891Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.5.1\"\n  },\n  \"agent\": {\n    \"type\": \"filebeat\",\n    \"version\": \"8.5.1\",\n    \"ephemeral_id\": \"99b35e07-cec6-402d-b284-e281c87ca70a\",\n    \"id\": \"ba31b2f8-fc09-4359-b6af-87b2f6e044a6\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"cloud\": {\n    \"region\": \"ap-south-1\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-south-1b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"252760256550\"\n    },\n    \"image\": {\n      \"id\": \"ami-0cf81108a24f29c5a\"\n    },\n    \"instance\": {\n      \"id\": \"i-09b08eb42aabf17d6\"\n    },\n    \"machine\": {\n      \"type\": \"m6a.2xlarge\"\n    }\n  },\n  \"log\": {\n    \"offset\": 1330,\n    \"file\": {\n      \"path\": \"/opt/elk/log/elasticsearch/VF-ELK.log\"\n    }\n  },\n  \"message\": \"\\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]\",\n  \"input\": {\n    \"type\": \"log\"\n  },\n  \"host\": {\n    \"os\": {\n      \"name\": \"CentOS Linux\",\n      \"kernel\": \"3.10.0-1160.76.1.el7.x86_64\",\n      \"codename\": \"Core\",\n      \"type\": \"linux\",\n      \"platform\": \"centos\",\n      \"version\": \"7 (Core)\",\n      \"family\": \"redhat\"\n    },\n    \"id\": \"7859246703b04a8781dc1c81420825c5\",\n    \"name\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"containerized\": false,\n    \"ip\": [\n      \"10.90.70.73\",\n      \"fe80::6563:c9d6:a4d:c355\"\n    ],\n    \"mac\": [\n      \"0A-B8-F4-B4-83-34\"\n    ],\n    \"hostname\": \"vl070073-app2-pd-a15-sms-aws-test-mum-in-vf.vfirst.local\",\n    \"architecture\": \"x86_64\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.892+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":628},"message":"Closing file","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.892+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":226},"message":"Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:0001-01-01 00:00:00 +0000 UTC Meta:null Fields:null Private:{Id:native::7501-39 PrevId: Finished:true Fileinfo:0xc000073110 Source:/opt/elk/log/elasticsearch/VF-ELK.log Offset:1437 Timestamp:2022-11-25 13:26:40.887491106 +0530 IST m=+0.947848949 TTL:-1ns Type:log Meta:map[] FileStateOS:7501-39 IdentifierName:native} TimeSeries:false}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":492},"message":"Update state (offset: 1437).","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"input.harvester","log.origin":{"file.name":"log/harvester.go","file.line":639},"message":"harvester cleanup finished.","service.name":"filebeat","input_id":"7341b6c8-c127-4b06-8f77-79aef187f571","source_file":"/opt/elk/log/elasticsearch/VF-ELK.log","state_id":"native::7501-39","finished":false,"os_id":"7501-39","old_source":"/opt/elk/log/elasticsearch/VF-ELK.log","old_finished":true,"old_os_id":"7501-39","harvester_id":"ae63debe-6d2f-4e71-8b8d-45e01daf319d","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":154},"message":"client: closing acker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":159},"message":"client: done closing acker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":161},"message":"client: unlink from queue","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":183},"message":"client: cancelled 0 events","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":163},"message":"client: done unlink","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":166},"message":"client: closing processors","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":171},"message":"client: done closing processors","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.893+0530","log.origin":{"file.name":"beater/crawler.go","file.line":185},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":132},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":166},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":167},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.893+0530","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":137},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.897+0530","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":194},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"id":"filebeat.service","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":33411072}}}},"cpu":{"system":{"ticks":910,"time":{"ms":910}},"total":{"ticks":1060,"time":{"ms":1060},"value":1060},"user":{"ticks":150,"time":{"ms":150}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"99b35e07-cec6-402d-b284-e281c87ca70a","name":"filebeat","uptime":{"ms":948},"version":"8.5.1"},"memstats":{"gc_next":21465960,"memory_alloc":12006608,"memory_sys":29705224,"memory_total":53106792,"rss":83525632},"runtime":{"goroutines":15}},"filebeat":{"events":{"active":15,"added":15,"done":0},"harvester":{"closed":1,"open_files":0,"running":0,"skipped":0,"started":1},"input":{"log":{"files":{"renamed":0,"truncated":0}},"netflow":{"flows":0,"packets":{"dropped":0,"received":0}}}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":0,"scans":0},"output":{"events":{"acked":0,"active":0,"batches":0,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":0},"read":{"bytes":0,"errors":0},"type":"logstash","write":{"bytes":0,"errors":0}},"pipeline":{"clients":0,"events":{"active":12,"dropped":0,"failed":0,"filtered":3,"published":12,"retry":0,"total":15},"queue":{"acked":0,"max_events":4096}}},"registrar":{"states":{"cleanup":0,"current":0,"update":2},"writes":{"fail":0,"success":0,"total":0}},"system":{"cpu":{"cores":8},"load":{"1":0.59,"15":0.84,"5":0.74,"norm":{"1":0.0738,"15":0.105,"5":0.0925}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.898+0530","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":195},"message":"Uptime: 951.971626ms","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.898+0530","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":162},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-25T13:26:40.898+0530","log.origin":{"file.name":"instance/beat.go","file.line":476},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-11-25T13:26:40.899+0530","log.origin":{"file.name":"instance/beat.go","file.line":1057},"message":"Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module system is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}

system · December 23, 2022, 10:03am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Couldnt start filebeat-Please HELP! Beats filebeat	3	3140	November 13, 2019
Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch Beats filebeat	4	1511	March 19, 2022
Filebeat not started Beats filebeat	21	1702	June 16, 2022
Filebeat failed to start Beats filebeat	1	520	July 7, 2023
Filebeat startup fail Beats filebeat	4	1417	April 22, 2019

Filebeat failed to load

Related topics