# Logstash filtering SOLR events

**URL:** <https://discuss.elastic.co/t/logstash-filtering-solr-events/97033>\
**Category:** Logstash\
**Created:** [August 14, 2017, 11:08pm UTC](https://discuss.elastic.co/t/logstash-filtering-solr-events/97033 "2017-08-14T23:08:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sandeep1](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@sandeep1](https://discuss.elastic.co/u/sandeep1)\
**Post date:** [August 14, 2017, 11:08pm UTC](https://discuss.elastic.co/t/logstash-filtering-solr-events/97033/1 "2017-08-14T23:08:50Z")

</div>

I'm trying to parse SOLR logs using logstash. Below i have pasted results from grok debugger but i dont see same results in elastic/kibana.

Here is my logstash filter:

if [fields][log\_type] == "SOLR\_0343"{  
grok {  
match =\> {  
"message" =\> ["%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:LOGLEVEL} %{JAVACLASS:javaclass}: [%{WORD:collection;string}] webapp=/%{WORD:webapp} path=/%{WORD:path} params={%{GREEDYDATA:params}} hits=%{NUMBER:hits} status=%{NUMBER:status} QTime=%{NUMBER:qtime}"]  
}  
}  
date {  
match =\> ["timestamp" , "yyyy-MM-dd HH:mm:ss,SSS"]  
}  
}

}

data i'm parsing :

2017-08-14 23:01:50,340 INFO org.apache.solr.core.SolrCore.Request: [tra\_1\_shard1\_replica1] webapp=/solr path=/old params={q=NUM\_s:3692-|BNUM\_s:444500-|CC\_NM\_s:xxxx-|CD\_s:72-|FUNC\_VAL\_s:AL+DRA-|C\_CD\_s:48-|L\_CD\_s:75080-|CD\_s:444313-|Y\_CD\_s:&qt=/old&sort=priority+asc&rows=1&wt=javabin&version=2} hits=2 status=0 QTime=8

I have verified this in grok debugger and i get expected results:

{  
"timestamp": [  
[  
"2017-08-09 18:51:13,316"  
]  
],  
"YEAR": [  
[  
"2017"  
]  
],  
"MONTHNUM": [  
[  
"08"  
]  
],  
"MONTHDAY": [  
[  
"09"  
]  
],  
"HOUR": [  
[  
"18",  
null  
]  
],  
"MINUTE": [  
[  
"51",  
null  
]  
],  
"SECOND": [  
[  
"13,316"  
]  
],  
"ISO8601\_TIMEZONE": [  
[  
null  
]  
],  
"LOGLEVEL": [  
[  
"INFO"  
]  
],  
"javaclass": [  
[  
"org.apache.solr.core.SolrCore.Request"  
]  
],  
"collection": [  
[  
"tra\_1\_shard1\_replica1"  
]  
],  
"webapp": [  
[  
"solr"  
]  
],  
"path": [  
[  
"old"  
]  
],  
"params": [  
[  
"q=NUM\_s:3692-|BNUM\_s:444500-|CC\_NM\_s:xxxx-|CD\_s:72-|FUNC\_VAL\_s:AL+DRA-|C\_CD\_s:48-|L\_CD\_s:75080-|CD\_s:444313-|Y\_CD\_s:&qt=/old&sort=priority+asc&rows=1&wt=javabin&version=2"  
]  
],  
"hits": [  
[  
"1"  
]  
],  
"BASE10NUM": [  
[  
"1",  
"0",  
"8"  
]  
],  
"status": [  
[  
"0"  
]  
],  
"qtime": [  
[  
"8"  
]  
]  
}

Here is the JSON i see in elastic:

{  
"\_index": "solr-2017.08.14",  
"\_type": "apache",  
"\_id": "AV3i9cHbhCPiQUeDbXWa",  
"\_version": 1,  
"\_score": null,  
"\_source": {  
"offset": 134975933,  
"input\_type": "log",  
"source": "/var/log/solr/solr-cmf-solr2-SOLR\_SERVER-sl55caehdp0343.visa.com.log.out",  
"message": "org.apache.solr.core.SolrCore.Request: [tran\_stamping\_1\_shard1\_replica1] webapp=/solr path=/old params={q=NUM\_s:3692-|BNUM\_s:444500-|CC\_NM\_s:xxxx-|CD\_s:72-|FUNC\_VAL\_s:AL+DRA-|C\_CD\_s:48-|L\_CD\_s:75080-|CD\_s:444313-|Y\_CD\_s:&qt=/old&sort=priority+asc&rows=1&wt=javabin&version=2} hits=1 status=0 QTime=10 ",  
"type": "apache",  
"tags": [  
"beats\_input\_codec\_plain\_applied",  
"version",  
"CDH5.5.1",  
"\_grokparsefailure"  
],  
"@timestamp": "2017-08-14T22:55:41.918Z",  
"@version": "1",  
"beat": {  
"hostname": "xxxxxxx",  
"name": "xxxxxx",  
"version": "5.4.1"  
},  
"host": "xxxxxxxxx",  
"fields": {  
"log\_type": "SOLR\_0343"  
},  
"LOGLEVEL": "INFO",  
"timestamp": "2017-08-14 22:55:41,918"  
},  
"fields": {  
"@timestamp": [  
1502751341918  
]  
},  
"sort": [  
1502751341918  
]  
}

Any help is much appreciated!!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 15, 2017, 5:23am UTC](https://discuss.elastic.co/t/logstash-filtering-solr-events/97033/2 "2017-08-15T05:23:10Z")

</div>

Build your expression gradually, starting with the simplest possible expressions (`^%{TIMESTAMP_ISO8601:timestamp}`). When you've verified that that works, add another token to your expression and continue until it breaks.

---

<div class="post-metadata">

**Author:** ![sandeep1](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@sandeep1](https://discuss.elastic.co/u/sandeep1)\
**Post date:** [August 25, 2017, 8:31am UTC](https://discuss.elastic.co/t/logstash-filtering-solr-events/97033/3 "2017-08-25T08:31:17Z")

</div>

Thanks it works now.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2017, 8:31am UTC](https://discuss.elastic.co/t/logstash-filtering-solr-events/97033/4 "2017-09-22T08:31:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.