HTTP_POLLER Plugin for Logstash

Elastic Stack Logstash
21

Hi magnusbaeck,

I've tried to put the code you gave me, but still don't work.

input {
 http_poller {
    urls => {
        jira => {
        method => get
        user => " "
        password => " "
        url => "my_url"
        headers => {
        Accept => "application/json"
        }
     }
    }
	request_timeout => 60
	schedule => { cron => "*/01 * * * * UTC"}
	metadata_target => "http_poller_metadata"
  }
}

filter {
	  ruby{
	    code => 
		userlist = []
		event.get('users').each { |k, v|
  		userlist << {"name" => k, "weeks" => v}
		}
		event.set('users', userlist)
	  }
	}
}
output {
   	elasticsearch {
	hosts => ["localhost:9200"] 
	index => "json_10meilleurs_logs_2"
	user => "elastic"
	password => "changeme"	
	}
}

I've tried with the following code and I get a one single document but can't read it in kibana.

input {
 http_poller {
    urls => {
        jira => {
        method => get
        user => " "
        password => " "
        url => "my_url"
        headers => {
        Accept => "application/json"
        }
     }
    }
	request_timeout => 60
	schedule => { cron => "*/01 * * * * UTC"}
	metadata_target => "http_poller_metadata"
  }
}

filter {

	json{
		source => "message"
		}
}
output {
   	elasticsearch {
	hosts => ["localhost:9200"] 
	index => "json_10meilleurs_logs"
	user => "elastic"
	password => "changeme"	
	}
}
22

The Ruby code needs to be surrounded by quotes (in this case double quotes). Compare what you have against examples in e.g. the documentation and it should be obvious what to do.

23

Hi, have done it too, have done some changes on it since yesterday according to the errors got and finished with this code:

filter {
	  ruby{
	   	code => " 
			userlist = [],
			event.get('users').each { |k, v|, 
			userlist.concat(["name" => k, "weeks" => v])
			}
			event.set('users', userlist) "
	  }
}

But still doesn't work :frowning:

24

Oh. You can't have double quotes in the Ruby code if you surround the whole code block with double quotes. Just use single quotes throughout the code.

25

ok

corrected with:

filter {
	  ruby{
	   	code => " 
			userlist = [],
			event.get('users').each { |k, v|, 
			userlist.concat(['name' => k, 'weeks' => v])
			}
			event.set('users', userlist) "
	  }
}

and got a syntax error:

SyntaxError: (ruby filter code):3: syntax error, unexpected ','
			event.get('users').each { |k, v|, 
                       ^
26

Don't end the lines with commas.

27

Hello,
Here I am again, still cannot solve the problem.
Have taken off the commas at the end of the lines. So get this filter code:

filter {
	  ruby{
	   	code => " 
			userlist = []
			event.get('users').each { |k, v| 
			userlist.concat(['name' => k, 'weeks' => v])
			}
			event.set('users', userlist) "
	  }
}

And I get thir error:

ruby_code
ruby_code.png1154×83 7.33 KB

By the way, by reading back our exchanges, I'd like to change a small thing. There's no "users" field. It starts directly by the name of the users.
So the structure of the document is more like:

{
    "user1": {
      "6": { ... },
      "5": { ... },
      ...
      "1": { ... },
    },
    "user2": {
      "6": { ... },
      "5": { ... },
      ...
      "1": { ... },
    },
    ...
    "user2": {
      "6": { ... },
      "5": { ... },
      ...
      "1": { ... },
    },
}
28

Please don't post screenshots if you can copy/paste plain text.

The error message means that event.get('users') returned nil, which in turn means that the event had no users field.

29

Ok, got it :slight_smile:

Ok
So, could you please tell me where can I find the documentation about the pattern of an event ?

30

So, could you please tell me where can I find the documentation about the pattern of an event ?

I don't think I understand the question, but the event API is documented here: Event API | Logstash Reference [8.11] | Elastic

31

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.