I am not read all log file and am not able to create new index


(nani) #1

hi i am new to elasticsearch and filebeat.

#=========================== Filebeat inputs =============================

filebeat.inputs:

Each - is an input. Most options can be set at the input level, so

you can use different inputs for various configurations.

Below are the input specific configurations.

  • type: log

    Change to true to enable this input configuration.

    enabled: true

    Paths that should be crawled and fetched. Glob based paths.

    paths:
    #- /var/log/.log
    #-C:\Program Files\PostgreSQL\9.6\data\pg_log\postgresql-2018-07-02_205849.log
    #- D:/logs/
    .log

    • D:\service*.log
      #- c:\programdata\elasticsearch\logs*

    Exclude lines. A list of regular expressions to match. It drops the lines that are

    matching any regular expression from the list.

    #exclude_lines: ['^DBG']

    Include lines. A list of regular expressions to match. It exports the lines that are

    matching any regular expression from the list.

    include_lines: ['^ERR', '^WARN' , '^INFO']

    Exclude files. A list of regular expressions to match. Filebeat drops the files that

    are matching any regular expression from the list. By default, no files are dropped.

    #exclude_files: ['.gz$']

    Optional additional fields. These fields can be freely picked

    to add additional information to the crawled log files for filtering

    #fields:

    level: debug

    review: 1

    Multiline options

    Multiline can be used for log messages spanning multiple lines. This is common

    for Java Stack Traces or C-Line Continuation

    The regexp Pattern that has to be matched. The example pattern matches all lines starting with [

    #multiline.pattern: ^[

    Defines if the pattern set under pattern should be negated or not. Default is false.

    #multiline.negate: false

    Match can be set to "after" or "before". It is used to define if lines should be append to a pattern

    that was (not) matched before or after or as long as a pattern is not matched based on negate.

    Note: After is the equivalent to previous and before is the equivalent to to next in Logstash

    #multiline.match: after

#============================= Filebeat modules ===============================

filebeat.config.modules:

Glob pattern for configuration loading

#path: ${path.config}/modules.d/*.yml
path: C:\Program Files\Filebeat\modules.d*.yml

Set to true to enable config reloading

reload.enabled: true

Period on which files under path should be checked for changes

#reload.period: 10s
#============================== Dashboards =====================================

These settings control loading the sample dashboards to the Kibana index. Loading

the dashboards is disabled by default and can be enabled either by setting the

options here, or by using the -setup CLI flag or the setup command.

setup.dashboards.enabled: true

output.elasticsearch:

Array of hosts to connect to.

hosts: ["http://localhost:9200"]
#index: "filebeat-%{[beat.version]}-%{+yyyy.MM.dd}"
#index: "Intelliviewlogs-%{[beat.version]}-%{+yyyy.MM.dd}"
#setup.template.name: "viewlogs"
#setup.template.pattern: "viewlogs-"
#setup.dashboards.index: "viewlogs-
"

indices:

index: "view1-%{[beat.version]}-%{+yyyy.MM.dd}"

when.contains:

 ##   message: "INFO"
  • manage_template: false
    -index: "view1-%{[beat.version]}-%{+yyyy.MM.dd}"
  • document_type : "%{[@view1][type]}"Preformatted text

(David Pilato) #2

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

In the meantime I moved your post to #beats:filebeat


(system) closed #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.