# Code Coverage for System Tests

**URL:** <https://discuss.elastic.co/t/code-coverage-for-system-tests/66233>\
**Category:** Beats\
**Created:** [November 16, 2016, 9:46am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233 "2016-11-16T09:46:54Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 16, 2016, 9:46am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/1 "2016-11-16T09:46:54Z")

</div>

I followed the elastic beats blog for finding coverage for the system tests. However my coverage.cov is not getting generated when i run the binary. Need help.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 16, 2016, 3:17pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/2 "2016-11-16T15:17:22Z")

</div>

Could you provide a [Short, Self Contained, Correct (Compilable), Example](http://www.sscce.org/) and show the exact commands you are using to compile and test. Then maybe one of us can spot a problem.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 17, 2016, 4:30am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/3 "2016-11-17T04:30:15Z")

</div>

This is the content inside main\_test.go:

```auto
 package main
// This file is mandatory as otherwise the filebeat.test binary is not generated correctly.
import (
  "testing"
  "flag"
)
var systemTest *bool
func init() {
  systemTest = flag.Bool("systemTest", true, "Set to true when running system tests")
}
// Test started when the test binary is started. Only calls main.
func TestSystem(t *testing.T) {
  if *systemTest {
     main()
  }
}

```

The command i use to create a binary is:

`go test -c -covermode=count -coverpkg ./...`

The command i use to run the binary:

`./binary.test -systemTest -test.coverprofile coverage.cov`

When i run it, there are some errors which i get(in the binary output) but it keeps running and i'm able to hit the APIs as well. When i stop the binary and see for the coverage file, it is not generated.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 17, 2016, 5:55am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/4 "2016-11-17T05:55:41Z")

</div>

Also, i get the following warning when i try to build a test binary

warning: no packages being tested depend on /path/to/packages. Does this have an impact on the resulting binary?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 17, 2016, 1:15pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/5 "2016-11-17T13:15:25Z")

</div>

Can you print a message after `main()` returns in the `TestSystem()` function and check if that gets logged. The test case must return normally (i.e. main() cannot exit) in order for the coverage file to be written.

```auto
func TestSystem(t *testing.T) {
  if *systemTest {
     main()
     t.Log("main returned")
  }
}

```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 18, 2016, 12:34pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/6 "2016-11-18T12:34:19Z")

</div>

You can ignore these warnings. Which golang version are you using?

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 21, 2016, 4:57am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/7 "2016-11-21T04:57:29Z")

</div>

I tried logging a message but i'm not able to see it. I would also like to tell you that i exit from binary using Ctrl-C after performing the tests. But i don't find the coverage file. I also keep checking the directory where it's supposed to be generated while the tests are running. The file is not generated. I have also fixed the errors which i used to get while running the binary.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 21, 2016, 4:57am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/8 "2016-11-21T04:57:49Z")

</div>

Go version which i'm using is : 1.6.2

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 21, 2016, 1:05pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/9 "2016-11-21T13:05:29Z")

</div>

Can you check what the exit code is after you hit CTRL-C? It should be 0 as @andrewkroh mentioned as otherwise the report is not generated.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 21, 2016, 1:57pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/10 "2016-11-21T13:57:41Z")

</div>

You probably need to [install](https://golang.org/pkg/os/signal/#example_Notify) your own signal handler to make the program exit gracefully. Then you should see the log message and have a code coverage file.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 22, 2016, 3:56am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/11 "2016-11-22T03:56:02Z")

</div>

Yes i already tried using signal handler and the code snippet looks like this.

func TestSystem(t \*testing.T) {  
if \*systemTest {  
term := make(chan os.Signal, 1)  
signal.Notify(term, os.Interrupt, syscall.SIGTERM)

```
    go func() {
     select {
            case <-term:
              log.Println("==> Stopping services")
              runtime.GC()
              os.Exit(0)
            }
    }()

    main()

```

}  
}

When i hit ctrl-c, i get this: **2016/11/22 11:50:24 ==\> Stopping services** and the binary stops. But still the coverage file is not generated.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 22, 2016, 1:34pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/12 "2016-11-22T13:34:07Z")

</div>

Instead of using `os.Exit(0)` try to just use `<-term` as a blocking channel in main and it will naturally finish the program when the channel is closed. As far as I remember `os.Exit(0)` does not work as it exits directly from this point and will not execute the additional code which was added to the binary.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 23, 2016, 9:01am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/13 "2016-11-23T09:01:30Z")

</div>

Could you please show the same(using \<-term) using the code snippet which i shared above ?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 23, 2016, 4:24pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/14 "2016-11-23T16:24:05Z")

</div>

I would need to see your `main()` to advise. In general you need to architect your `main()` function to `return` when finished and not call `os.Exit()`. This means that when the interrupt signal is received that it should stop doing what ever work it's doing and return.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 28, 2016, 10:56am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/15 "2016-11-28T10:56:28Z")

</div>

Thanks @andrewkroh, @ruflin for your support. I am now able to generate coverage for my system tests 🙂

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 28, 2016, 11:18am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/16 "2016-11-28T11:18:24Z")

</div>

@imharishd Glad to hear. It would be nice if you could share in 2-3 sentence with the community what the final solution to your problem was, or where the problem was.

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [November 28, 2016, 11:51am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/17 "2016-11-28T11:51:11Z")

</div>

we call beego.Run in our main function. I used signal handler to return from main and also placed the beego.Run inside a go routine. This did the job for me. 🙂

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [December 5, 2016, 12:34pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/18 "2016-12-05T12:34:38Z")

</div>

I was able to generate two coverage files and merge them. I am not able to understand the final coverage file that is generated. Please find below the results.

Coverage1: (71.4%)

 ![](https://us1.discourse-cdn.com/elastic/original/2X/b/b15fabd1aa1a9f25881b3239e4f94a230b664344.png)

Coverage2: (72.5%)

 ![](https://us1.discourse-cdn.com/elastic/original/2X/8/8a217d73bfa69580488931b7eb67cb6ce3e75bb1.png)

FinalSystemCoverage: (72%)

 ![](https://us1.discourse-cdn.com/elastic/original/2X/6/675581c1b24e996075d78256fa84b6d66b9344ff.png)

Can someone please interpret the above results and tell how its calculating the final system coverage ? And also, the reason why the covered lines in coverage1 doesn't appear as covered in final coverage ?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [December 5, 2016, 4:16pm UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/19 "2016-12-05T16:16:34Z")

</div>

How did you merge the two?

We use this: [https://github.com/elastic/beats/blob/master/dev-tools/aggregate\_coverage.py](https://github.com/elastic/beats/blob/master/dev-tools/aggregate_coverage.py)

---

<div class="post-metadata">

**Author:** ![imharishd](https://avatars.discourse-cdn.com/v4/letter/i/bc8723/32.png) [@imharishd](https://discuss.elastic.co/u/imharishd)\
**Post date:** [December 6, 2016, 3:28am UTC](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233/20 "2016-12-06T03:28:25Z")

</div>

mkdir coverage  
echo 'mode: count' \> ./coverage/system.cov  
tail -q -n +2 ./coverage/\*.cov \>\> ./coverage/system.cov

This way i merged the cov files. The above method was listed in the blog. ( [https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests](https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests))

[Next page](https://discuss.elastic.co/t/code-coverage-for-system-tests/66233.md?page=2)