Contents

Go - Go Routines, Wait Groups and Channels

What are those notes about?

With those notes, I want to give you some ideas on how to use and not use Go Routines, Wait Groups, and Channels.

Go Routines

Go routines are there to help the developer schedule tasks to run in threads.

Concurrency vs. Parallelism

In the context of a CPU:

Concurrency happens when you have only one resource and many tasks to be executed.

Each task has a limited amount of time to execute, and when that time is up, another task will be taken. If the previous task was not yet finished, it will wait until the resource has some time for it.

In parallelism, we have more resources, and the tasks are spitted through those resources. It is important to notice that each resource is still working in concurrency mode by giving each task a limited amount of time and moving on to the next one when the time is up.

Go Scheduler

Go Scheduler is part of what is called Go Runtime, and it is responsible for managing your Go application’s routines. Including the main go routine.

Wait Groups

A wait group is a mechanism that helps the developer be sure that a collection of go routines are finished. It gives the developer to decide to wait and to decide where to wait.

Channels

A channel is the way a go routine has to communicate with other go routines.

Make use of the channel

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
package main

import (
	"fmt"
	"time"
)

func main() {
    // We create a channel buffer of type string
    // For this example we are going to have 3 extra go routines
    message := make(chan string, 3)

    fmt.Println("main: waiting for goroutine to wake up")

    // We initialize a go routine with the keyword go
    // We give the function access to the communication channel
    go goToSleep(2*time.Second, message, "sleeper 1")
    go goToSleep(3*time.Second, message, "sleeper 2")
    go goToSleep(4*time.Second, message, "sleeper 3")

    fmt.Println("main: waiting sleepers feedback")

    // here is where things get a bit more interesting
    //
    // by default, the main routine will not stop and wait for the other
    // routines to finish.
    //
    // if you don't wait there is a huge change the main routine will finish
    // and exit before the other routines finish and their work will be in vain
    //
    // to avoid it we need to know where to wait and how to wait
    // for this example we say:
    // We will be waiting for a message to come from the communication channel
    // and only after the message arrives we move to the next line of code

    // 1st go routine to finish
    awake1 := <-message
    fmt.Println("main: got one feedback from", awake1)

    // 2nd go routine to finish
    awake2 := <-message
    fmt.Println("main: got one feedback from", awake2)

    // 3rd go routine to finish
    awake3 := <-message
    fmt.Println("main: got one feedback from", awake3)

    // now that we know all the go routines are finished
    // we can finally move on and finish the main routine after printing
    // a message
    fmt.Println("main: all good. We are all well rested and done")
}

// The goToSleep function will receive the time to sleep, the communication channel
// and a identification name for the log output
func goToSleep(t time.Duration, c chan string, name string) {
	fmt.Printf("goroutine %s: Go for a nap of %v seconds\n", name, t)
	time.Sleep(t)
	fmt.Printf("goroutine %s: Waking up from nap feeling refreshed\n", name)
	fmt.Printf("goroutine %s: telling main I'm done with my nap\n", name)
    // goToSleep is ready to use the communication channel
    // it does it by sending (<-) any string to it
    // in our case we use the identification name
	c <- name
}

And the output will be something like

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
main: waiting for goroutine to wake up
main: waiting sleepers feedback
goroutine sleeper 1: Go for a nap of 2s seconds
goroutine sleeper 2: Go for a nap of 3s seconds
goroutine sleeper 3: Go for a nap of 4s seconds
goroutine sleeper 1: Waking up from nap feeling refreshed
goroutine sleeper 1: telling main I'm done with my nap
main: got one feedback from sleeper 1
goroutine sleeper 2: Waking up from nap feeling refreshed
goroutine sleeper 2: telling main I'm done with my nap
main: got one feedback from sleeper 2
goroutine sleeper 3: Waking up from nap feeling refreshed
goroutine sleeper 3: telling main I'm done with my nap
main: got one feedback from sleeper 3

Waiting for a channel that is not being used

If you are waiting for a channel that does not bring any information for some time, your application will panic

1
2
3
4
5
6
7
8
9
package main

import "fmt"

func main() {
    c := make(chan bool)
    <-c
    fmt.Println("channel expired")
}

Output:

1
fatal error: all goroutines are asleep - deadlock!

Making usage of Wait Group

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
    // We create a variable to represent our wait group
	var wg sync.WaitGroup

    fmt.Println("main: waiting for goroutine to wake up")

    // We know we will have 3 go routines
    // So, we share that knowledge with our wait group
    wg.Add(3)

    // We initialize a go routine with the keyword go
    // We give the function access to the wait group
    // It is important to be sure we are giving the function
    // a reference to the wait group by using `&`
    go goToSleepWg(2*time.Second, &wg, "sleeper 1")
    go goToSleepWg(3*time.Second, &wg, "sleeper 2")
    go goToSleepWg(4*time.Second, &wg, "sleeper 3")

    fmt.Println("main: waiting all sleepers to wake up")

    // Now we tell the main routine to wait
    // It will be waiting until all 3 extra go routines are finished
    wg.Wait()

    // Super! Now we are ready to move on with the main go routine flow
    // Print the message and finish the go main routine
    fmt.Println("main: all good. We are all well rested and done")
}

// The goToSleep function will receive the time to sleep, the reference to the
// wait group and a identification name for the log output
func goToSleepWg(t time.Duration, wg *sync.WaitGroup, name string) {
    // That is a very important step when working with wait groups
    // We need to be sure that we are going to call the wg.Done()
    // that call will inform the wait group that this specific go routine
    // finished the work
    // we use defer because defer give us the garante that the function Done will
    // be called when the goToSleepWp is finished.
	defer wg.Done()

	fmt.Printf("goroutine %s: Go for a nap of %v seconds\n", name, t)
	time.Sleep(t)
	fmt.Printf("goroutine %s: Waking up from nap feeling refreshed\n", name)
	fmt.Printf("goroutine %s: I'm done with my nap. I won't tell main but I will tell wg I'm done.\n", name)
}

wg.Add(x) - Will increment a counter telling the wait group how many wg.Done() are being expected to be called. wg.Done() - Will decrement the counter wg.Wait() - Will be waiting until the counter is 0.

In case the wait group counter turns negative. Wait Counter will panic

The output is similar to the example with channel

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
main: waiting for goroutine to wake up
main: waiting all sleepers to wake up
goroutine sleeper 2: Go for a nap of 3s seconds
goroutine sleeper 3: Go for a nap of 4s seconds
goroutine sleeper 1: Go for a nap of 2s seconds
goroutine sleeper 1: Waking up from nap feeling refreshed
goroutine sleeper 1: I'm done with my nap. I won't tell main but I will tell wg I'm done.
goroutine sleeper 2: Waking up from nap feeling refreshed
goroutine sleeper 2: I'm done with my nap. I won't tell main but I will tell wg I'm done.
goroutine sleeper 3: Waking up from nap feeling refreshed
goroutine sleeper 3: I'm done with my nap. I won't tell main but I will tell wg I'm done.
main: all good. We are all well rested and done

How to limit resource usage with wait group

We can use wait group to work with limited resources.

The example bellow simulates a hostel with a long queue and only 3 beds are available.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
    // We create a variable to represent our wait group
    var wg sync.WaitGroup

    // We set some variables to control our hostel
    const availableBeds = 3
    bedsInUse := 0

    // We have a queue of people in need for some rest
    inNeedOfSleep := []string{
        "Ali Younger",
        "Marcus Schulze",
        "Laurel Shaw",
        "Jimmie Medina",
        "Aniya Cochrane",
        "Johnson Schofield",
        "Mary Benner",
        "Brendan Whelan",
        "Ibrahim Jacobs",
        "Aaliyah Colby",
        "Carolina Currier",
        "Arnold Kiser",
        "Taya Pettit",
        "Denise Arsenault",
        "Ramiro Greiner",
        "Jessie Owen",
        "Trent McGraw",
        "Bobby Gandy",
        "Jadon Correia",
        "Konner Delossantos",
        "Allen Butterfield",
        "Truman Kimble",
        "Xzavier Decker",
        "Jenna Hauser",
        "Emerson Solorzano",
        "Keshawn Oates",
        "Leland Barton",
        "Piper Cordell",
        "Jayson Counts",
        "Javion Ogle",
        "The tired hostel employee1",
        "The tired hostel employee2",
        "The tired hostel employee3",
        "The hostel owner",
    }

    fmt.Println("----- Opening the hostel doors -----")

    // We let our wait group knows how much we can handle per time
    wg.Add(availableBeds)

    // Now we go person by person
    for i := 0; i < len(inNeedOfSleep); i++ {
        fmt.Printf("----- We have %d beds available -----\n", availableBeds-bedsInUse)
        bedsInUse++

        fmt.Println("----- Getting the next person on the line ----")
        fmt.Printf("----- %s got a bed to have a nap -----\n", inNeedOfSleep[i])

        // Here is where the person goes to bed.
        // We share the wait group reference and the person name
        // so the function can simulate the person's nap
        go func(wg *sync.WaitGroup, p string) {
            // remember the Done? We can't forget about it!
            // here is where we tell the wait group that we are done with our nap
            defer wg.Done()
            fmt.Printf("%s felt sleep :O\n", p)
            time.Sleep(500 * time.Millisecond)
            fmt.Printf("%s woke up and the bed is now free again\n", p)
        }(&wg, inNeedOfSleep[i])

        // In case we are in full capacity
        // we wait for people to finish their nap
        // so we can clean the beds
        // and go back to our loop to give good news
        // to 3 more on the line
        if bedsInUse == availableBeds {
            fmt.Println("----- We are in full capacity -----")
            fmt.Println("----- Waiting beds to be available to clean  -----")

            // Be quiet. People are sleeping :D. We wait!
            wg.Wait()

            fmt.Println("----- Cleaning used beds -----")
            time.Sleep(1 * time.Second)

            fmt.Println("----- Beds are clean and available to use again -----")
            // beds are clean so we update our management variables
            bedsInUse = 0
            // we let the wait group know we have 3 more spots left
            wg.Add(availableBeds)
        }
    }
    // Uhu! Everyone is rested!
    fmt.Println("----- Closing the hostel doors -----")
}

The output is a bit long :) so I’m going to post only part of it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
----- Opening the hostel doors -----
----- We have 3 beds available -----
----- Getting the next person on the line ----
----- Ali Younger got a bed to have a nap -----
----- We have 2 beds available -----
----- Getting the next person on the line ----
----- Marcus Schulze got a bed to have a nap -----
----- We have 1 beds available -----
----- Getting the next person on the line ----
----- Laurel Shaw got a bed to have a nap -----
----- We are in full capacity -----
----- Waiting beds to be available to clean  -----
Laurel Shaw felt sleep :O
Ali Younger felt sleep :O
Marcus Schulze felt sleep :O
Marcus Schulze woke up and the bed is now free again
Ali Younger woke up and the bed is now free again
Laurel Shaw woke up and the bed is now free again
----- Cleaning used beds -----
----- Beds are clean and available to use again -----
----- We have 3 beds available -----
----- Getting the next person on the line ----
----- Jimmie Medina got a bed to have a nap -----
----- We have 2 beds available -----
----- Getting the next person on the line ----
----- Aniya Cochrane got a bed to have a nap -----
----- We have 1 beds available -----
----- Getting the next person on the line ----
----- Johnson Schofield got a bed to have a nap -----
Aniya Cochrane felt sleep :O
----- We are in full capacity -----
----- Waiting beds to be available to clean  -----
Johnson Schofield felt sleep :O
Jimmie Medina felt sleep :O
Jimmie Medina woke up and the bed is now free again
Johnson Schofield woke up and the bed is now free again
Aniya Cochrane woke up and the bed is now free again
----- Cleaning used beds -----
----- Beds are clean and available to use again -----
----- We have 3 beds available -----
----- Closing the hostel doors -----

Some things to have in mind when using Wait Groups

Negative counter

If the Wait Group counter gets Negative, it will panic.

Check some examples bellow:

1
2
3
4
5
6
7
8
9
package main

import "sync"

func main() {
    // We create a variable to represent our wait group
    var wg sync.WaitGroup // counter: 0
    wg.Done() // counter: -1 [panic]
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
package main

import "sync"

func main() {
    // We create a variable to represent our wait group
    var wg sync.WaitGroup // counter: 0
    wg.Add(1) // counter: 1
    wg.Done() // counter: 0
    wg.Done() // counter: -1 [panic]
}
1
2
3
4
5
6
7
8
9
package main

import "sync"

func main() {
    // We create a variable to represent our wait group
    var wg sync.WaitGroup // counter: 0
    wg.Add(-2) // counter: -2 [panic]
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
	// We create a variable to represent our wait group
	var wg sync.WaitGroup // counter: 0

	go func(wg *sync.WaitGroup) {
		defer wg.Done() // counter: -1 [panic]
		fmt.Println("Go routine is running")
	}(&wg)

    // we have a sleep here to be sure that the
    // go routine will finish before the main
    // go routine exits
	time.Sleep(3 * time.Second)

    // if the extra go routine finishes
    // this part of the code won't be executed
    // because the Done was called and the app
    // panic
	wg.Wait()
	fmt.Println("Done with main go routine")
}

Output:

1
panic: sync: negative WaitGroup counter

You forgot to use Add(x)

If you don’t let your wait group knows how many go routines you have running it won’t be able to wait

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
package main

import "sync"

func main() {
    // We create a variable to represent our wait group
    var wg sync.WaitGroup // counter: 0
	go func(wg *sync.WaitGroup) {
        // Just for the example to work
        // We are not calling done here to avoid panic
		fmt.Println("Go routine is running")
	}(&wg)

    wg.Wait() // counter: 0 [it won't wait because it is already 0]
}

Output is inconsistent. Depending on how fast the go routine runs it will or will not print to the output. You can run this example multiple times to see the output

1
2
3
4
# First try
❯ ./example
# The go main routine finished before the extra go routine
❯
1
2
3
4
5
# Second try
❯ ./example
Done with main go routine
# The extra go routine was lucky enough to finish before the main go routine was finished
❯

Not calling Done when the go routine finish

If you don’t call done by using defer wg.Done() at the beginning of your function. Wait group won’t have a way to know that the function was done and, We get a deadlock

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
package main

import "sync"

func main() {
    // We create a variable to represent our wait group
    var wg sync.WaitGroup // counter: 0
    // We let our wait group know we have one extra go routine to run
    wg.Add(1)

	go func() {
        // We are not letting the wait group know we are done executing our extra go routine
		fmt.Println("Go routine is running")
	}()

    wg.Wait() // counter: 1
}

Output

1
fatal error: all goroutines are asleep - deadlock!

Calling Add(x) after calling Done()

It will panic :O

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
package main

import (
	"fmt"
	"sync"
)

func main() {
	// We create a variable to represent our wait group
	var wg sync.WaitGroup // counter: 0

	wg.Add(1)
	go func(wg *sync.WaitGroup) {
		fmt.Println("Go routine is running")
		// We let wait group know we are done
		wg.Done()
		// We try to add to wait group [panic]
		wg.Add(1)
	}(&wg)

	wg.Wait()
}

Output

1
panic: sync: WaitGroup is reused before previous Wait has returned