GIT Pipelines Integration

Elastic Stack Logstash
1

Hi -
We are trying to integrate with GIT CI/CD with ELK , I'd like to pull the Pipelines job status and send it to Elastic Search and then display it on the dashboard. Would somone please provide your comments on this.

Thanks !

2

Are you talking about Jenkins? When do you want to pull the status? What dashboard are you talking about?

3

Hi @magnusbaeck,

Its for the GIT Continuous Integration (https://about.gitlab.com/features/gitlab-ci-cd/).
It has the Pipeline Jobs executing, wanted to pull the status of those jobs.
We are using Kibana as Dashboard.

4

Oh, GitLab. I suppose you could use an http_poller input in Logstash to periodically query for the pipeline's status.

5

Thanks for your reply, @magnusbaeck.

We have the authorization for the HTTP requests with the Access Token, how do I give it the request ?
I tried with the below request, but it throws unauthorized 401 error, but with the same access token works over Ruby Rest calls.

input {
http_poller {
urls => {
test2 => {

       url => "https://172.232.121:8080/api/v4/projects/pi-uy-tt%2Ftestjobs/pipelines"
       method => get
       header => {
		   Accept => "application/json"
           private_token => "TnTLejtCS63SfmQzLMCG"
		}
	}

}
request_timeout => 60
codec => "json"
schedule => { cron => "* * * * * UTC"}
}
}

Could you please help. Thanks !

6

Aren't you supposed to use the token as the password for the user that requested the token rather than stuff it in a private_token header?

7

I believe, it should be as a Private Token, it worked when I gave along with the URL.
@magnusbaeck
And Could you please let me know how do I fix this issue :slight_smile:

image

8

I believe, it should be as a Private Token, it worked when I gave along with the URL.

Exactly what worked? What URL? What headers?

9

@magnusbaeck
url => "https://gitlbtesting.com/pipelines?private_token=yc6ZgsGb-P_88sSRASP6"

The above URL worked.

Could you please how to resolve the below issue:

image

10

The above URL worked.

Okay, so add the token to the URL then. The header option is for adding HTTP headers.

Could you please how to resolve the below issue:

OTOH I don't know what's wrong there.

11

Hi @magnusbaeck

This is the new URL, am trying to poll the HTTP requests and get the data:

Here is the Conf file:

input {
http_poller {
urls => {
test1 => {
url => "https://10.12.22.110/statistics.json"
method => get
user => "Admin"
password => "Pass"
headers => {
Accept => "application/json"
}
}

}

ssl_certificate_validation => false
request_timeout => 60
codec => "json"
schedule => { cron => "* * * * * UTC"}
}
}

output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => "localhost:9200"
index => "hppp"
workers => 1
}
}

I see logstash is getting stuck, not able to get any data. :frowning: could you please help.

12

@magnusbaeck - Is there anything wrong with my ConF file. The URL returns a data in the format of JSON.
Please let me know.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.