Use API field as a variable in logstash input


What I am trying to do seems quite similar to what @dorj1234 was asking help for a few months ago in this topic but the answer provided is not enough for me so I hope you won't mind me opening a similar topic.

What I would like to get is information about Jenkins builds configuration. From the API, I can have some information about all builds, but if I want to know more about a specific build, I have to add its name in the URL. So my idea is to make a first API request with logstash, maybe store the builds name somewhere (file? local variable?), then use logstash again for each build.

I use exec plugin to curl the Jenkins API and then I would like to GET jenkins/job/name-of-job/config.xml. Is there any way I could do that?

Thanks for the help :slight_smile:

Well the answer is much the same. You can write a ruby filter, which would perhaps use Net::HTTP to do the second GET. Something like this, which uses the whole of message as a filename, and puts the response into somefield.

    ruby {
        code => '
            require "net/http"
            uri = URI( "" + event.get("message") )
            response = Net::HTTP.get(uri)
            event.set("somefield", response)

Error handling left as an exercise for the reader.

1 Like

Ok thanks, I'll try that! I must admit I'm not really familiar with ruby, so thank you for giving the whole syntax :slight_smile:

Edit: I looked a little further and came across the Nokogiri gem, which seems to be better to get specific elements of the XML file. What do you think?

Edit 2: I have tried a few things but I can't fix the issue I have. The address I'm trying to get the XML file from is HTTPS and I always get this error:
Ruby exception occurred: Non-HTTP proxy URI: proxy.********.fr:****

Here's my ruby filter:

ruby {
    code => '
      require "nokogiri"
      require "open-uri"
      config = URI.parse(event.get("[jobs][url]") + "config.xml").read
      jdk_version = config.xpath("//jdk")
      event.set("jdk_version", jdk_version)

Logstash has an xml filter, which is a wrapper around nokogiri.

I guess I shouldn't have tried to do everything within the Ruby filter...
I used your code to get all the XML in a field and now I just have to use the XML filter to keep what I need.

I think I should be able to do it by myself now, thanks a lot!

Edit: It's working perfectly, thanks again :wink:

Actually, I still have a problem...
I need to authenticate to access the config.xml file but I can't figure out how to do it with the https request. I have seen lots of "solutions" but only for http requests. Obviously, I don't want to use username and password, but rather the private token. I have tried to put it in the header but can't make it work.
Any idea for this?

Here's my code:

	ruby {
		# Get the config.xml for each job
		code => '
			require "net/https"
			url = event.get("[jobs][url]") + "config.xml"
			uri = URI(url)

			Net::HTTP.start(, uri.port, :use_ssl => true) do |http|
				request =
				request["authorization"] = "Token token=abc123"
				response = http.request(request)
				event.set("config", response.body)

I used the other syntax for HTTPS from the NET::HTTP class because it seemed to handle the header, but it still doesn't work.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.