In Logstash when we given start command it is not able to push the data in Elastic Search

Doubt:-1:

  1. I am trying to do ELK demo in Windows. I want to load files into elastic search. I started the beats. First time when i try to load a file in elastic search , it loads successfully. But second time when i try , it is getting stopped in below following line:
    [2018-03-19T15:07:50,167][INFO ][org.logstash.beats.Server] Starting server on port: 5044
    [2018-03-19T15:07:50,271][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}

Why it is not loding when i am trying 2nd time ? Even i tried to restart logstash but it is not working.

  1. Now i want to upload multiple files. So please check the below line in Configuration Files ?

  2. Could you please tell how to get data from tags via grok filters.
    3.1. <data:COUNTRY_CODE>UA</data:COUNTRY_CODE>

Configuation Files:-

logstash-simple.conf:

input {
beats
{
port => "5044"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}

filebeat.yml:

filebeat.prospectors:

Each - is a prospector. Most options can be set at the prospector level, so

you can use different prospectors for various configurations.

Below are the prospector specific configurations.

type: log

Change to true to enable this prospector configuration.

enabled: false

Paths that should be crawled and fetched. Glob based paths.

paths:
#- /var/log/*.log
#- c:\programdata\elasticsearch\logs*
C:\Users\ramals4\Documents\iMap.log

paths:

C:\Users\ramals4\Documents\server.log

#==================== Elasticsearch template setting ==========================

setup.template.settings:
index.number_of_shards: 3

#================================ General =====================================

The name of the shipper that publishes the network data. It can be used to group

all the transactions sent by a single shipper in the web interface.

#name:
fields:
env: staging
type: iMap
generator: iMaplog
server: myserver

fields:
env: staging
type: server
generator: serverlog
server: myserver

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:

Array of hosts to connect to.

hosts: ["localhost:9200"]

First time when i try to load a file in Elasticsearch , it loads successfully. But second time when i try , it is getting stopped in below following line:

Filebeat won't send the same file two times. It keep track of what it has read. If you want to reset the saved state you can delete (or edit) Filebeat's registry file.

Now i want to upload multiple files. So please check the below line in Configuration Files ?

Format the configuration file as preformatted text so it doesn't get mangled when you post. There's a toolbar button for it.

Could you please tell how to get data from tags via grok filters.

To parse XML you should use an xml filter, not a grok filter.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.