My ZXTM logs not shipping to Elastic Search From Logstash


(Kolli Dilip) #1

Dear All,

Can some one help me to resolve the issue, where the ZXTM logs are not being shipped to ES.

The logstash config is as below

input
{
file
{
type => "ZXTMLogs"
path =>[ "F:/UKBlogsstachconfiguration/Logstobestached/ZTMLogpath/*"]
start_position => beginning
sincedb_path => "F:/logstash-5.5.0/bin/.sincedb"
sincedb_write_interval => 10
sindex.refresh_interval => 30s

}

}

filter
{
# Include ZXTM Foramts Below
if [type] in ["ZXTMLogs", "SecureBusinessPage-IISLogs"]
{
grok
{
match => ["message", "(?:%{NUMBER:totalTimeTakenForClient:float}|-) (?:%{IP:clientIpAddress}|-) (?:[%{HTTPDATE:requestTimestamp:datetime}]|-) (?:%{NOTSPACE:VirtualServer}|-) (?:%{HOSTNAME:Domain}|-) (?:%{NOTSPACE:SoapAction}|-) (?:%{WORD:httpmethod}|-) (?:%{URIPATHPARAM:uriPathWithParam}|-) (?:%{NUMBER:httpResponse:int}|-) (?:%{NUMBER:bytesReceivedFromClient:int}|-) (?:%{NUMBER:bytesSentToClient:int}|-) (?:%{NUMBER:timeTakenToConstructResponse:float}|-) (?:%{IP:Connected_Server}:%{POSINT:ConnectedServerPort}|%{HOSTNAME:Connected_Server}:%{POSINT:ConnectedServerPort}|-) (?:%{NOTSPACE:ApplicationPool}|-) (?:%{IP:SessionServer}:%{POSINT:Sessionserverport}|%{HOSTNAME:SessionServer}:%{POSINT:Sessionserverport}|-)"]
}

}

}

output
{
if (!("_grokparsefailure" in [tags]))
{

elasticsearch 
{
hosts => ["localhost:9200"] 
}

} 

}

the folder structure is as below

Directory of F:\logstash-5.5.0
28/07/2017 12:09 bin
28/07/2017 12:08 config

F:\logstash-5.5.0\bin>dir
Volume in drive F is New Volume
Volume Serial Number is 56AD-3436

Directory of F:\logstash-5.5.0\bin

07/07/2017 22:11 38,647,169 .sincedb
30/06/2017 23:56 373 cpdump
30/06/2017 23:56 155 ingest-convert.sh
30/06/2017 23:56 1,949 logstash
30/06/2017 23:56 448 logstash-plugin
30/06/2017 23:56 251 logstash-plugin.b
30/06/2017 23:56 677 logstash.bat
30/06/2017 23:56 5,400 logstash.lib.sh
30/06/2017 23:56 840 ruby
30/06/2017 23:56 2,795 setup.bat
30/06/2017 23:56 3,530 system-install

F:\logstash-5.5.0\config>dir
Volume in drive F is New Volume
Volume Serial Number is 56AD-3436

Directory of F:\logstash-5.5.0\config
30/06/2017 23:56 1,809 jvm.options
30/06/2017 23:56 3,958 log4j2.properties
28/07/2017 12:37 1,293 logstash.conf
07/07/2016 08:41 44 run.bat
30/06/2017 23:56 1,702 startup.options

I have installed Logstash service with nssm, with the service pointing to run.bat in the config folder

Thanks


(Kolli Dilip) #2

Can any one help
me here please


(Mark Walkom) #3

The first step is to add a stdout and make sure they are making it through the pipeline.


(Kolli Dilip) #4

Hi

I have tried it and they are showing up on the Console


(Mark Walkom) #5

Ok, now check Elasticsearch to see what indices have been created. _cat/indices is good for that.


(Kolli Dilip) #6

Hurray, its done

Indexes are created


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.