Logstash and Json, How to ingest this fragment:

I'm trying to ingest json data with this format:

{"info":{"page":1,"pages":53431,"results":53431},"plugins":[{"name":"Contact Form 7","slug":"contact-form-7","version":"5.4.2","author":"<a href=\"https:\/\/ideasilo.wordpress.com\/\">Takayuki Miyoshi<\/a>","author_profile":"https:\/\/profiles.wordpress.org\/takayukister","requires":"5.5","tested":"5.8","requires_php":false,"rating":82,"ratings":{"5":1320,"4":133,"3":59,"2":49,"1":342},"num_ratings":1903,"support_threads":495,"support_threads_resolved":151,"active_installs":5000000,"downloaded":186259370,"last_updated":"2021-07-14 6:48am GMT","added":"2007-08-02","homepage":"https:\/\/contactform7.com\/","short_description":"Just another contact form plugin. Simple but flexible.","description":"<p>Contact Form 7 can manage multiple contact forms, plus you can customize the form and the mail contents flexibly with simple markup. The form supports Ajax-powered submitting, CAPTCHA, Akismet spam filtering and so on.<\/p>\n<h4>Docs and support<\/h4>\n<p>You can find <a href=\"https:\/\/contactform7.com\/docs\/\" rel=\"nofollow ugc\">docs<\/a>, <a href=\"https:\/\/contactform7.com\/faq\/\" rel=\"nofollow ugc\">FAQ<\/a> and more detailed information about Contact Form 7 on <a href=\"https:\/\/contactform7.com\/\" rel=\"nofollow ugc\">contactform7.com<\/a>. When you can&#8217;t find the answer to your question on the FAQ or in any of the documentation, check the <a href=\"https:\/\/wordpress.org\/support\/plugin\/contact-form-7\/\">support forum<\/a> on WordPress.org. If you can&#8217;t locate any topics that pertain to your particular issue, post a new topic for it.<\/p>\n<h4>Contact Form 7 needs your support<\/h4>\n<p>It is hard to continue development and support for this free plugin without contributions from users like you. If you enjoy using Contact Form 7 and find it useful, please consider <a href=\"https:\/\/contactform7.com\/donate\/\" rel=\"nofollow ugc\">making a donation<\/a>. Your donation will help encourage and support the plugin&#8217;s continued development and better user support.<\/p>\n<h4>Privacy notices<\/h4>\n<p>With the default configuration, this plugin, in itself, does not:<\/p>\n<ul>\n<li>track users by stealth;<\/li>\n<li>write any user personal data to the database;<\/li>\n<li>send any data to external servers;<\/li>\n<li>use cookies.<\/li>\n<\/ul>\n<p>If you activate certain features in this plugin, the contact form submitter&#8217;s personal data, including their IP address, may be sent to the service provider. Thus, confirming the provider&#8217;s privacy policy is recommended. These features include:<\/p>\n<ul>\n<li>reCAPTCHA (<a href=\"https:\/\/policies.google.com\/?hl=en\" rel=\"nofollow ugc\">Google<\/a>)<\/li>\n<li>Akismet (<a href=\"https:\/\/automattic.com\/privacy\/\" rel=\"nofollow ugc\">Automattic<\/a>)<\/li>\n<li>Constant Contact (<a href=\"https:\/\/www.endurance.com\/privacy\" rel=\"nofollow ugc\">Endurance International Group<\/a>)<\/li>\n<li><a href=\"https:\/\/www.sendinblue.com\/legal\/privacypolicy\/\" rel=\"nofollow ugc\">Sendinblue<\/a><\/li>\n<\/ul>\n<h4>Recommended plugins<\/h4>\n<p>The following plugins are recommended for Contact Form 7 users:<\/p>\n<ul>\n<li><a href=\"https:\/\/wordpress.org\/plugins\/flamingo\/\">Flamingo<\/a> by Takayuki Miyoshi &#8211; With Flamingo, you can save submitted messages via contact forms in the database.<\/li>\n<li><a href=\"https:\/\/wordpress.org\/plugins\/bogo\/\">Bogo<\/a> by Takayuki Miyoshi &#8211; Bogo is a straight-forward multilingual plugin that doesn&#8217;t cause headaches.<\/li>\n<\/ul>\n<h4>Translations<\/h4>\n<p>You can <a href=\"https:\/\/contactform7.com\/translating-contact-form-7\/\" rel=\"nofollow ugc\">translate Contact Form 7<\/a> on <a href=\"https:\/\/translate.wordpress.org\/projects\/wp-plugins\/contact-form-7\" rel=\"nofollow ugc\">translate.wordpress.org<\/a>.<\/p>\n","download_link":"https:\/\/downloads.wordpress.org\/plugin\/contact-form-","tags":{"contact":"contact","contact-form":"contact form","email":"email","feedback":"feedback","form":"form"},"donate_link":"https:\/\/contactform7.com\/donate\/","icons":{"1x":"https:\/\/ps.w.org\/contact-form-7\/assets\/icon.svg?rev=2339255","2x":"https:\/\/ps.w.org\/contact-form-7\/assets\/icon-256x256.png?rev=2279696","svg":"https:\/\/ps.w.org\/contact-form-7\/assets\/icon.svg?rev=2339255"}}]}

I've tried with one line, multiline, config.support_escapes: true, etc.. and always getting same result: syntax error, I've validated Json; its fine, I don't know what/where is exactly the problem, Break Lines? Windows vs Linux formats? This is my Logstash conf file (one of them):

# Input section
input {
  file {
		start_position => "beginning"
		path => "D:/path/*.json"
		sincedb_path => "NUL"

#filter section

filter {
  json {
    source => "message"
	target => "tes]"
		mutate { 
		  remove_field => ["host", "@version", "message", "type", "path", "tags", "@timestamp"]			

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "databalbal"
    user => "user" 
    password => "pwd"
     stdout {}

Any help is apreciated.

I would expect that to fail with

:exception=>"Invalid FieldReference: `tes]`"

You need the square brackets to be balanced.

Sorry it's a typo error while copying.

Don't know the reason but I'm having errors like:

#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
"; line: 1, column: 17]>}      "tags":{

It's a valid Json, I'm using a Json config and I'm reciving Json errors, :frowning:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.