Approach to send metrics to Elasticsearch in a new module

[madou23]

Hi all,

I'm trying to create a new metricbeat module for Jenkins. Actually, I'm working with this wonderful API : Jenkins API Client for Go
I'm trying to send all jobs information include it's builds but I'm facing a problem to implement it in module "fetch" method
Below how build response returned from this API

type BuildResponse struct {
	Actions   []generalObj
	Artifacts []struct {
		DisplayPath  string `json:"displayPath"`
		FileName     string `json:"fileName"`
		RelativePath string `json:"relativePath"`
	} `json:"artifacts"`
	Building  bool   `json:"building"`
	BuiltOn   string `json:"builtOn"`
	ChangeSet struct {
		Items []struct {
			AffectedPaths []string `json:"affectedPaths"`
			Author        struct {
				AbsoluteUrl string `json:"absoluteUrl"`
				FullName    string `json:"fullName"`
			} `json:"author"`
			Comment  string `json:"comment"`
			CommitID string `json:"commitId"`
			Date     string `json:"date"`
			ID       string `json:"id"`
			Msg      string `json:"msg"`
			Paths    []struct {
				EditType string `json:"editType"`
				File     string `json:"file"`
			} `json:"paths"`
			Timestamp int64 `json:"timestamp"`
		} `json:"items"`
		Kind      string `json:"kind"`
		Revisions []struct {
			Module   string
			Revision int
		} `json:"revision"`
	} `json:"changeSet"`
	Culprits          []culprit   `json:"culprits"`
	Description       interface{} `json:"description"`
	Duration          int64       `json:"duration"`
	EstimatedDuration int64       `json:"estimatedDuration"`
	Executor          interface{} `json:"executor"`
	FullDisplayName   string      `json:"fullDisplayName"`
	ID                string      `json:"id"`
	KeepLog           bool        `json:"keepLog"`
	Number            int64       `json:"number"`
	QueueID           int64       `json:"queueId"`
	Result            string      `json:"result"`
	Timestamp         int64       `json:"timestamp"`
	URL               string      `json:"url"`
	MavenArtifacts    interface{} `json:"mavenArtifacts"`
	MavenVersionUsed  string      `json:"mavenVersionUsed"`
	FingerPrint       []FingerPrintResponse
	Runs              []struct {
		Number int64
		URL    string
	} `json:"runs"`
}

What's the correct approach to convert this info into a Json object sent to Elasticsearch ?
I have already tried to extract some field as mentioned in this documentation Creating a Metricset and everything working well but I noticed that it only support "common.MapStr" object, how can I deal with arrays nested in the above-mentioned struct ??

[andrewkroh]

@madou23

how can I deal with arrays nested in the above-mentioned struct ??

You could try leaving them as is depending on how you plan to use the fields from the objects in the arrays. The main limitation of leaving them as is is that Kibana has trouble rending the nested objects (but you can still query based on the fields).