Change the regexes useragent filter plugin uses by default

mikewillis · March 5, 2020, 12:11pm

This question is about version of Logstash 5.6 onwards as myself and colleagues are still working on moving away from 5 (yes we know it's EOL).

At time of writing the Logstash useragent filter plugin identifies Chromium based Edge as Chrome instead of Edge. This has led to me looking at making it use a newer version of the regexes.yaml from

github.com

ua-parser/uap-core/blob/master/regexes.yaml

user_agent_parsers:
  #### SPECIAL CASES TOP ####

  # CFNetwork Podcast catcher Applications
  - regex: '^(Luminary)[Stage]+/(\d+) CFNetwork'
  - regex: '(ESPN)[%20| ]+Radio/(\d+)\.(\d+)\.(\d+) CFNetwork'
  - regex: '(Antenna)/(\d+) CFNetwork'
    family_replacement: 'AntennaPod'
  - regex: '(TopPodcasts)Pro/(\d+) CFNetwork'
  - regex: '(MusicDownloader)Lite/(\d+)\.(\d+)\.(\d+) CFNetwork'
  - regex: '^(.*)-iPad\/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)(?:\.(\d+)|) CFNetwork'
  - regex: '^(.*)-iPhone/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)(?:\.(\d+)|) CFNetwork'
  - regex: '^(.*)/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)(?:\.(\d+)|) CFNetwork'

  # Podcast catchers
  - regex: '^(Luminary)/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)'
  - regex: '(espn\.go)'
    family_replacement: 'ESPN'
  - regex: '(espnradio\.com)'
    family_replacement: 'ESPN'

This file has been truncated. show original

But using a newer version of regexes.yaml seems to mean modifying every call to the useragent filter to include

regexes => "path_to_regexes_file"

This is quite tedious and creates the possibility of new calls not being created in a way which means they use the same regexes file as existing calls, because people are fallible.

Instead of modifying a whole bunch of existing calls and hope people don't create inconsistency in future, what I'd really like to do is replace the regexes.yaml the plugin uses by default. As far as I can tell that's not possible, but can anyone tell me otherwise?

system · April 2, 2020, 12:11pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
User_agent filter regexes.yaml location Logstash	3	479	December 23, 2020
User-agent filter does not handle "+" as space in ua-strings from IIS Logstash	5	1603	July 6, 2017
Custom userAgent regexes.yaml Logstash	2	855	December 4, 2018
Useragent filter plugin not working for subfields? Logstash	8	1058	September 20, 2018
Useragent filter plugin fields don't match ECS Logstash	6	1450	June 5, 2019

Change the regexes useragent filter plugin uses by default

Related topics