Xpack Watcher -- Credit card in logs

Elastic Stack Elasticsearch
1

Hello --

Working on a regex pattern for a watcher that will detect if credit card information is present in our logs. I was hoping that Elastic would have a sample pattern for this as it's one of the specific examples you use here: https://www.elastic.co/products/stack/alerting

-- 
Credit card numbers are visible in your application logs and that's a compliance nightmare. It's time to talk with the application team.

The only other post I found for something similar to this would be from 2017

Additionally

Which regex version are supported with watchers?

--Thanks!

2

Hey,

when you are searching across indexed log files with watcher you could use the regexp. This however is a pretty expensive query and an interesting question might be, if you are able to actually filter for the credit card pattern during ingestion (either within your ingestion tool or using an ingest processor) and flag those documents as potentially containing credit card information. Than your search for such alerts would become much faster and easier.

Hope this helps!

--Alex

3

Hello!

Thank you for your response. I do not expect to find much CC information in our logs as we already try to strip that before it gets ingested. This is more for reassurance than anything else. My plan would be to run a regex to find any string of numbers between 14-16 characters (VISA) in logs and store that in a payload and then run a second regex against the payload for actual credit card parameters. Hoping this will reduce the cost of the regex.

4

Watcher Config so far:

"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"creditcardtest"
],
"types": ,
"body": {
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "/([0-9]{13,16}|[0-9]{4}.[0-9]{4}.[0-9]{4}.[0-9]{4})/"

I was able to find a regex that will consistently grab numbers strings between 13 and 16 characters long:

"query_string": {
"query": "/([0-9]{13,16}|[0-9]{4}.[0-9]{4}.[0-9]{4}.[0-9]{4})/" }

The docs from my test index hold messages like so:
American Express,376978891168364,Elijah Hall,Court Street 108,New Zealand,111,3367$,01/2023
342346589207954
6011-2940-4688-2620
5462 8434 9581 9941

I am able to find all the docs using this pattern.

Now I'd like to load the payload that these strings are saved to and then run a regex on that data for CC matches. Is that possible?
Any help is appreciated.

--Thanks!

5

Any chance you know if I am able to grep out info, load to payload, and then search through the payload only?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.