GIT Pipelines Integration


#1

Hi -
We are trying to integrate with GIT CI/CD with ELK , I'd like to pull the Pipelines job status and send it to Elastic Search and then display it on the dashboard. Would somone please provide your comments on this.

Thanks !


(Magnus Bäck) #2

Are you talking about Jenkins? When do you want to pull the status? What dashboard are you talking about?


#3

Hi @magnusbaeck,

Its for the GIT Continuous Integration (https://about.gitlab.com/features/gitlab-ci-cd/).
It has the Pipeline Jobs executing, wanted to pull the status of those jobs.
We are using Kibana as Dashboard.


(Magnus Bäck) #4

Oh, GitLab. I suppose you could use an http_poller input in Logstash to periodically query for the pipeline's status.


#5

Thanks for your reply, @magnusbaeck.

We have the authorization for the HTTP requests with the Access Token, how do I give it the request ?
I tried with the below request, but it throws unauthorized 401 error, but with the same access token works over Ruby Rest calls.

input {
http_poller {
urls => {
test2 => {

       url => "https://172.232.121:8080/api/v4/projects/pi-uy-tt%2Ftestjobs/pipelines"
       method => get
       header => {
		   Accept => "application/json"
           private_token => "TnTLejtCS63SfmQzLMCG"
		}
	}

}
request_timeout => 60
codec => "json"
schedule => { cron => "* * * * * UTC"}
}
}

Could you please help. Thanks !


(Magnus Bäck) #6

Aren't you supposed to use the token as the password for the user that requested the token rather than stuff it in a private_token header?


#7

I believe, it should be as a Private Token, it worked when I gave along with the URL.
@magnusbaeck
And Could you please let me know how do I fix this issue :slight_smile:

image


(Magnus Bäck) #8

I believe, it should be as a Private Token, it worked when I gave along with the URL.

Exactly what worked? What URL? What headers?


#9

@magnusbaeck
url => "https://gitlbtesting.com/pipelines?private_token=yc6ZgsGb-P_88sSRASP6"

The above URL worked.

Could you please how to resolve the below issue:

image


(Magnus Bäck) #10

The above URL worked.

Okay, so add the token to the URL then. The header option is for adding HTTP headers.

Could you please how to resolve the below issue:

OTOH I don't know what's wrong there.


#11

Hi @magnusbaeck

This is the new URL, am trying to poll the HTTP requests and get the data:

Here is the Conf file:

input {
http_poller {
urls => {
test1 => {
url => "https://10.12.22.110/statistics.json"
method => get
user => "Admin"
password => "Pass"
headers => {
Accept => "application/json"
}
}

}

ssl_certificate_validation => false
request_timeout => 60
codec => "json"
schedule => { cron => "* * * * * UTC"}
}
}

output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => "localhost:9200"
index => "hppp"
workers => 1
}
}

I see logstash is getting stuck, not able to get any data. :frowning: could you please help.


#12

@magnusbaeck - Is there anything wrong with my ConF file. The URL returns a data in the format of JSON.
Please let me know.


(system) #13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.