Logstah optimization and data merge


(Jlogan) #1

hello everyone,i need help!please !
problem1:how to optimize my code?
problem2:how to delete the logdata i don‘t want and merge my data,cause my data has lots of blank and duplicates. i want delete the invalid data,and merge some data to make my events complete. thanks!

my logstash code:
input {
file {
path => "/home/logs/log/testfile1.txt"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
dissect{
mapping => {
"message" => "[%{device_name}]%{} %{}"
}
}
grok{
match => {
"message" => "(?Power\s+1.)"
}
}
grok{
match => {
"message" => "(?Power\s+2.
)"
}
}
dissect{
mapping => {
"power1" => "%{} %{} %{}: %{power_1_state}"
}
}
dissect{
mapping => {
"power2" => "%{} %{} %{}: %{power_2_state}"
}
}
grok {
match => {
"message" => "(?^Uptime.*)"
}}
dissect {
mapping => {
"uptime" => "%{} %{} %{uptime_weeks} %{},%{uptime_days} %{},%{uptime_hours} %{},%{uptime_minus} %{}"
}
}
}
output {
file{
path => "/home/data/cs006.bak"
}
elasticsearch {
hosts => ["172.16.0.20:9200"]
index => "cs005"
}
}

my test-logdata is:
[CS-KP-7506-1]dis ver
[CS-KP-7506-1]dis version
H3C Comware Platform Software
Comware Software, Version 5.20, Release 6305P02
Copyright (c) 2004-2009 Hangzhou H3C Tech. Co., Ltd. All rights reserved.
HC S7506E uptime is 36 weeks, 4 days, 14 hours, 38 minutes

Uptime is 36 weeks,4 days,14 hours,32 minutes
HC S7506E LPU with 1 BCM1122H Processor
DRAM: 512M bytes
FLASH: 0M bytes
NVRAM: 0K bytes
PCB 1 Version: VER.C
Bootrom Version: 205
CPLD 1 Version: 003
Release Version: H3C S7506E-6305P02

[CS-KP-7506-1] dis power
Power 1 State: Normal
Power 2 State: Normal

[CS-KP-7506-1]dis log
Logging buffer configuration and contents:enabled
Allowed max buffer size : 1024
Actual buffer size : 512
Channel number : 4 , Channel name : logbuffer
Dropped messages : 0
Overwritten messages : 9447
Current messages : 512

my results is:


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.