# Logstash filter for varying number of disks info in the logs **URL:** \ **Category:** Logstash\ **Created:** [August 13, 2021, 7:21pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400 "2021-08-13T19:21:41Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Shreesh\_Narayanan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreesh_narayanan/32/87312_2.png) [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Post date:** [August 13, 2021, 7:21pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/1 "2021-08-13T19:21:41Z")
Hello, i have a couple of log lines which go like this , that have varying number of disks ```auto 1628868082.953|7ec93a454940| disk-usage: [1]=46% [2]=44% [3]=52% [4]=52% [5]=46% [6]=40% [7]=45% [8]=48% 1628868082.953|7ec93a454940| disk-usage: [1]=38% [2]=10% 1628868082.953|7ec93a454940| disk-usage: [1]=46% [2]=44% [3]=52% [4]=52% [5]=46% [6]=40% [7]=45% [8]=48% [9]=15% [10]=28% ``` I tried writing a grok filter with multiple patterns , but hasnt worked. ```auto match => {"message" =>["%{NUMBER:epoch_time}%{NOTSPACE:ThreadID} %{WORD:cache}-%{WORD:cache}: %{NOTSPACE:cache}=%{NUMBER:Disk_1}% %{NOTSPACE:cache}=%{NUMBER:Disk_2}% %{NOTSPACE:cache}=%{NUMBER:Disk_3}% %{NOTSPACE:cache}=%{NUMBER:Disk_4}% %{NOTSPACE:cache}=%{NUMBER:Disk_5}% %{NOTSPACE:cache}=%{NUMBER:Disk_6}% %{NOTSPACE:cache}=%{NUMBER:Disk_7}% %{NOTSPACE:cache}=%{NUMBER:Disk_8}%","%{NUMBER:cache_time}%{NOTSPACE:ThreadID} %{WORD:cache}-%{WORD:cache}: %{NOTSPACE:cache}=%{NUMBER:Disk_1}% %{NOTSPACE:cache}=%{NUMBER:Disk_2}%"]} ``` Can someone help? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 13, 2021, 7:46pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/2 "2021-08-13T19:46:18Z")
I would not use grok for that. I would use dissect and kv. ``` dissect { mapping => { "message" => "%{[@metadata][ts]}|%{}| disk-usage: %{[@metadata][restOfLine]}" } } date { match => ["[@metadata][ts]", UNIX ] } kv { trim_value => "%" } ``` will produce ``` "@timestamp" => 2021-08-13T15:21:22.953Z, "1" => "46", "2" => "44", "3" => "52", ``` etc. I would then be tempted to either ``` ruby { code => ' diskUsage = [] event.to_hash.each { |k, v| if k =~ /^\d+$/ diskUsage[k.to_i] = v.to_i; end event.remove(k) } diskUsage.shift event.set("diskUsage", diskUsage) ' } ``` to get ``` "diskUsage" => [ [0] 46, [1] 44, [2] 52, [3] 52, [4] 46, [5] 40, [6] 45, [7] 48 ] ``` or ``` ruby { code => ' diskUsage = [] event.to_hash.each { |k, v| if k =~ /^\d+$/ event.set("diskUsage#{k.to_i}", v.to_i) end event.remove(k) } ' } ``` to get ``` "diskUsage5" => 46, "diskUsage4" => 52, "diskUsage7" => 45, ``` etc. ---
**Author:** ![Shreesh\_Narayanan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreesh_narayanan/32/87312_2.png) [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Post date:** [August 13, 2021, 7:51pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/3 "2021-08-13T19:51:21Z")
Thanks a lot @Badger๐Ÿ™‚ is there some documentation on how rube code works on logstash ? i'd like to understand it ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 13, 2021, 8:28pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/4 "2021-08-13T20:28:50Z")
> [@Shreesh\_Narayanan](#): > > is there some documentation on how rube code works on logstash ? Not that I know of. ---
**Author:** ![Shreesh\_Narayanan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreesh_narayanan/32/87312_2.png) [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Post date:** [August 13, 2021, 8:35pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/5 "2021-08-13T20:35:57Z")
okay ๐Ÿ™‚ i'm using dissect for the first time. so i was wondering if "%{}" can be used for empty spaces ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 13, 2021, 9:02pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/6 "2021-08-13T21:02:15Z")
%{} matches anything, it consumes everything up to the next delimiter. ---
**Author:** ![Shreesh\_Narayanan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreesh_narayanan/32/87312_2.png) [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Post date:** [August 13, 2021, 9:16pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/7 "2021-08-13T21:16:38Z")
Thanks a ton @Badger ๐Ÿ™‚ . The kv + dissect worked perfectly ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 10, 2021, 9:16pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400/8 "2021-09-10T21:16:46Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.