FIlebeat not feeding logs to logstash

Elastic Stack Beats
1

I currently trying to sending iis logs by filebeat to logstash. Everything were fine when i send asterisk log to logstash but now for somereason filebeat wont sending logs to logstash .
here is my filebeat.yml

filebeat.inputs:
    # Asterisk /var/log/asterisk/debug
    - type: log
      paths:
       - C:/Users/Charles/Desktop/DATA/u_ex191201.txt
      fields:
      document_type: iislog
      include_lines: ["^[[]"]
      fields_under_root: true

output.logstash:
 # The Logstash hosts
  hosts: ["10.64.2.246:5044"]

and it always returnign

2020-01-24T12:35:05.011+0700    INFO    instance/beat.go:607    Home path: [C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64] Config path: [C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64] Data path: [C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data] Logs path: [C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\logs]
2020-01-24T12:35:05.011+0700    DEBUG   [beat]  instance/beat.go:659    Beat metadata path: C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data\meta.json
2020-01-24T12:35:05.012+0700    INFO    instance/beat.go:615    Beat ID: 6098a674-5dd7-4424-9fd9-848442bac485
2020-01-24T12:35:05.012+0700    DEBUG   [seccomp]       seccomp/seccomp.go:96   Syscall filtering is only supported on Linux
2020-01-24T12:35:05.012+0700    INFO    [beat]  instance/beat.go:903    Beat info       {"system_info": {"beat": {"path": {"config": "C:\\Users\\Charles\\Desktop\\DATA\\filebeat-7.4.2-windows-x86_64", "data": "C:\\Users\\Charles\\Desktop\\DATA\\filebeat-7.4.2-windows-x86_64\\data", "home": "C:\\Users\\Charles\\Desktop\\DATA\\filebeat-7.4.2-windows-x86_64", "logs": "C:\\Users\\Charles\\Desktop\\DATA\\filebeat-7.4.2-windows-x86_64\\logs"}, "type": "filebeat", "uuid": "6098a674-5dd7-4424-9fd9-848442bac485"}}}
2020-01-24T12:35:05.012+0700    INFO    [beat]  instance/beat.go:912    Build info      {"system_info": {"build": {"commit": "15075156388b44390301f070960fd8aeac1c9712", "libbeat": "7.4.2", "time": "2019-10-28T19:46:12.000Z", "version": "7.4.2"}}}
2020-01-24T12:35:05.013+0700    INFO    [beat]  instance/beat.go:915    Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":6,"version":"go1.12.9"}}}
2020-01-24T12:35:05.019+0700    INFO    [beat]  instance/beat.go:919    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-01-24T12:17:32.23+07:00","name":"D-ESS2018070164","ip":["2001:d10:8:2:38ca:e1d0:a052:54b0/64","2001:d10:8:2:c5f:abab:5273:fbe6/128","fe80::38ca:e1d0:a052:54b0/64","10.64.66.176/25","::1/128","127.0.0.1/8"],"kernel_version":"10.0.18362.592 (WinBuild.160101.0800)","mac":["4c:ed:fb:40:02:24"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"18362.592"},"timezone":"+07","timezone_offset_sec":25200,"id":"a489adb6-c555-4260-9883-d541391d6804"}}}
2020-01-24T12:35:05.022+0700    INFO    [beat]  instance/beat.go:948    Process info    {"system_info": {"process": {"cwd": "C:\\Users\\Charles\\Desktop\\DATA\\filebeat-7.4.2-windows-x86_64", "exe": "C:\\Users\\Charles\\Desktop\\DATA\\filebeat-7.4.2-windows-x86_64\\filebeat.exe", "name": "filebeat.exe", "pid": 1088, "ppid": 1560, "start_time": "2020-01-24T12:35:04.962+0700"}}}
2020-01-24T12:35:05.022+0700    INFO    instance/beat.go:292    Setup Beat: filebeat; Version: 7.4.2
2020-01-24T12:35:05.022+0700    DEBUG   [beat]  instance/beat.go:318    Initializing output plugins
2020-01-24T12:35:05.023+0700    DEBUG   [publisher]     pipeline/consumer.go:137        start pipeline event consumer
2020-01-24T12:35:05.023+0700    INFO    [publisher]     pipeline/module.go:97   Beat name: D-ESS2018070164
2020-01-24T12:35:05.024+0700    WARN    beater/filebeat.go:152  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-01-24T12:35:05.024+0700    INFO    instance/beat.go:422    filebeat start running.
2020-01-24T12:35:05.024+0700    DEBUG   [service]       service/service_windows.go:72   Windows is interactive: true
2020-01-24T12:35:05.024+0700    INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2020-01-24T12:35:05.025+0700    DEBUG   [test]  registrar/migrate.go:159        isFile(C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data\registry) -> false
2020-01-24T12:35:05.025+0700    DEBUG   [test]  registrar/migrate.go:159        isFile() -> false
2020-01-24T12:35:05.025+0700    DEBUG   [test]  registrar/migrate.go:152        isDir(C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data\registry\filebeat) -> true
2020-01-24T12:35:05.025+0700    DEBUG   [test]  registrar/migrate.go:159        isFile(C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data\registry\filebeat\meta.json) -> true
2020-01-24T12:35:05.025+0700    DEBUG   [registrar]     registrar/migrate.go:51 Registry type '0' found
2020-01-24T12:35:05.026+0700    DEBUG   [registrar]     registrar/registrar.go:125      Registry file set to: C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data\registry\filebeat\data.json
2020-01-24T12:35:05.026+0700    INFO    registrar/registrar.go:145      Loading registrar data from C:\Users\Charles\Desktop\DATA\filebeat-7.4.2-windows-x86_64\data\registry\filebeat\data.json
2020-01-24T12:35:05.026+0700    INFO    registrar/registrar.go:152      States Loaded from registrar: 6
2020-01-24T12:35:05.027+0700    WARN    beater/filebeat.go:368  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-01-24T12:35:05.027+0700    INFO    crawler/crawler.go:72   Loading Inputs: 1
2020-01-24T12:35:05.027+0700    DEBUG   [registrar]     registrar/registrar.go:278      Starting Registrar
2020-01-24T12:35:05.028+0700    DEBUG   [input] log/config.go:204       recursive glob enabled
2020-01-24T12:35:05.029+0700    DEBUG   [input] log/input.go:164        exclude_files: []. Number of stats: 6
2020-01-24T12:35:05.029+0700    DEBUG   [input] log/input.go:185        input with previous states loaded: 0
2020-01-24T12:35:05.029+0700    INFO    log/input.go:152        Configured paths: [C:\Users\Charles\Desktop\DATA\u_ex191201.txt]
2020-01-24T12:35:05.030+0700    INFO    input/input.go:114      Starting input of type: log; ID: 3654584443412608221
2020-01-24T12:35:05.030+0700    INFO    crawler/crawler.go:106  Loading and starting Inputs completed. Enabled inputs: 1
2020-01-24T12:35:05.030+0700    DEBUG   [input] log/input.go:191        Start next scan
2020-01-24T12:35:05.034+0700    DEBUG   [input] log/input.go:212        input states cleaned up. Before: 0, After: 0, Pending: 0

Please help me spot the problem. Thank you

3

Somehow that regex in include_lines seems incorrect. Can you upload some example lines?

4

hi there, thank you for your respond. This problem just solved. I've check the file format. i was entered it as txt because it written as text document. But when i look any closer by looking at the properties it written as .log. And yeah i deleted the include line part. I've faced some other problem can you please check here if you mind

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.