# Logstah optimization and data merge

**URL:** <https://discuss.elastic.co/t/logstah-optimization-and-data-merge/86055>\
**Category:** Logstash\
**Created:** [May 17, 2017, 7:15am UTC](https://discuss.elastic.co/t/logstah-optimization-and-data-merge/86055 "2017-05-17T07:15:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![JIELOGAN](https://avatars.discourse-cdn.com/v4/letter/j/67e7ee/32.png) [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\
**Post date:** [May 17, 2017, 7:15am UTC](https://discuss.elastic.co/t/logstah-optimization-and-data-merge/86055/1 "2017-05-17T07:15:59Z")

</div>

hello everyone，i need help！please ！  
problem1：how to optimize my code？  
problem2：how to delete the logdata i don‘t want and merge my data，cause my data has lots of blank and duplicates. i want delete the invalid data，and merge some data to make my events complete. thanks!

**my logstash code：**  
input {  
file {  
path =\> "/home/logs/log/testfile1.txt"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
}  
}  
filter {  
dissect{  
mapping =\> {  
"message" =\> "[%{device\_name}]%{} %{}"  
}  
}  
grok{  
match =\> {  
"message" =\> "(?Power\s+1._)"  
}  
}  
grok{  
match =\> {  
"message" =\> "(?Power\s+2._)"  
}  
}  
dissect{  
mapping =\> {  
"power1" =\> "%{} %{} %{}: %{power\_1\_state}"  
}  
}  
dissect{  
mapping =\> {  
"power2" =\> "%{} %{} %{}: %{power\_2\_state}"  
}  
}  
grok {  
match =\> {  
"message" =\> "(?^Uptime.\*)"  
}}  
dissect {  
mapping =\> {  
"uptime" =\> "%{} %{} %{uptime\_weeks} %{},%{uptime\_days} %{},%{uptime\_hours} %{},%{uptime\_minus} %{}"  
}  
}  
}  
output {  
file{  
path =\> "/home/data/cs006.bak"  
}  
elasticsearch {  
hosts =\> ["172.16.0.20:9200"]  
index =\> "cs005"  
}  
}

**my test-logdata is：**  
[CS-KP-7506-1]dis ver  
[CS-KP-7506-1]dis version  
H3C Comware Platform Software  
Comware Software, Version 5.20, Release 6305P02  
Copyright (c) 2004-2009 Hangzhou H3C Tech. Co., Ltd. All rights reserved.  
HC S7506E uptime is 36 weeks, 4 days, 14 hours, 38 minutes

Uptime is 36 weeks,4 days,14 hours,32 minutes  
HC S7506E LPU with 1 BCM1122H Processor  
DRAM: 512M bytes  
FLASH: 0M bytes  
NVRAM: 0K bytes  
PCB 1 Version: VER.C  
Bootrom Version: 205  
CPLD 1 Version: 003  
Release Version: H3C S7506E-6305P02

[CS-KP-7506-1] dis power  
Power 1 State: Normal  
Power 2 State: Normal

[CS-KP-7506-1]dis log  
Logging buffer configuration and contents:enabled  
Allowed max buffer size : 1024  
Actual buffer size : 512  
Channel number : 4 , Channel name : logbuffer  
Dropped messages : 0  
Overwritten messages : 9447  
Current messages : 512

**my results is:**

 ![](https://us1.discourse-cdn.com/elastic/original/3X/9/a/9a93600e36c3bd7118f7af2ddd06da741cedd8c3.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 14, 2017, 7:16am UTC](https://discuss.elastic.co/t/logstah-optimization-and-data-merge/86055/2 "2017-06-14T07:16:06Z")

</div>

Closed on June 14, 2017, 7:16am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.