Hi,
I have a case when I need to use a translate filter with a very large YAML dictionary (around 2.5 million of entries).
I was reading this post
where it recommends to use memcached in order to speed up the translation.
But my case is I use the regex => true
translate{
field => "a_key"
destination => "data_A_Billing"
regex => true
exact => true
dictionary_path => '/data/sbc/tables/rates_a_billing.yaml'
fallback => "NF|NF|-1|Destination Not Found|Origin Not Found|NF"
}
where a_key is a combination of the destination prefix and an originating number in a phone call
For example:
34655#376688785
That will match the following line in the yaml
'^34655#376[0-9]*$': "34655|376|0,083200|Spain -Mob ORANGE|ZONE 3|A"
The dictionary file is sorted from more specific to more general regex.
Originally the dictionary was of 400.000 entries, but now is 7 times bigger and latency in processing events are now a drawback.
Is there a better way to implement this?
Any feddback will be appreciated
Thank you!