# Filter grok : Few lines differents in log file

**URL:** <https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529>\
**Category:** Logstash\
**Created:** [February 9, 2017, 4:17pm UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529 "2017-02-09T16:17:29Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [February 9, 2017, 4:17pm UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529/1 "2017-02-09T16:17:29Z")

</div>

Hello,

I have a little problem about performing of of my "q\_compt" file (file text)

80% of lines have 4 fields  
10% have 3 fields  
10% have 4 fields but have differents informations in.

Here my filter expression :

```
if [type] == "q_compt" {
     grok {
         match => { "message" => "%{DATA:date}[;]%{DATA:name_compt}[;]%{DATA:applications}[;]%{NUMBER:volume}" }
          }

```

Here an example of lines :

201702;Vol-EDR-2;appli1;15000 (4 fields)  
201702;Vol-EDR-2;appli2;5000 (4 fields)  
201702;Vol-EDR-total;20000 (3 fields)

201702;CA-EDR-2;appli1;850 (4 fields)  
201702;CA-EDR-2;appli2;150 (4 fields)  
201702;CA-EDR-total;1000 (3 fields)

I had thought this :

```
if [type] == "q_compt" {
     grok {
         match => { "message" => "%{DATA:date}[;]%{DATA:name_compt}[;]%{DATA:applications}[;]%{NUMBER:volume}" || "%{DATA:date}[;]%{DATA:name_compt}[;]%{NUMBER:volume_total}" }
 }
```

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [February 10, 2017, 4:12pm UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529/2 "2017-02-10T16:12:46Z")

</div>

I have about 30 differents name\_compt (field n°2) in the one log file text

I had thought cut the main file into as many files as there are name\_compt.

So i should modified the filebeat prospect as :

```
filebeat.prospectors:
  - input_type: log
    - "/log_file_applis/Vol-EDR-2"
         document_type: Vol-EDR-2

    - "/log_file_applis/CA-EDR-2"
         document_type: CA-EDR-2

```

So, in logstash conf :

```
if [type] == "Vol-EDR-2" {
     grok {...}}

if [type] == "CA-EDR-2" {
    grok {...}}

```

But this way of doing involves do write 30 type of prospector and 30 "if [type] in logstash conf.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 13, 2017, 10:42am UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529/3 "2017-02-13T10:42:05Z")

</div>

Why not just have a single grok filter with two expressions, one that matches the four-field case and one that matches the three-field case? Logstash will match them in turn, stopping at the first match.

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [February 13, 2017, 2:46pm UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529/4 "2017-02-13T14:46:26Z")

</div>

I'm stupid, this is a simply way !

Do you know how change octet values to gigaoctet values ? It's in Kibana or directly in logstash conf ?

Thank you for your answer

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 13, 2017, 3:08pm UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529/5 "2017-02-13T15:08:34Z")

</div>

> Do you know how change octet values to gigaoctet values ? It's in Kibana or directly in logstash conf ?

Not sure if you can do it in Kibana but it's definitely possible with Logstash. The units filter seems to be able to do it but otherwise a ruby filter can be used.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 13, 2017, 3:08pm UTC](https://discuss.elastic.co/t/filter-grok-few-lines-differents-in-log-file/74529/6 "2017-03-13T15:08:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.