最近在学习Go语言,阅读Go语言并发之道(第四章),学习context包源码。
因为对context理解不深刻,所以写一篇文章总结一下;Go源码基于1.14.4版本。
1、为什么要用context,context能做什么
原因是:存在A调用B,B再调用C的场景,若此时A调用B取消了,那也要取消B调用C。
书上最初是通过Done Channel来引入“取消”机制的,然后再引入了context机制。
在Go1.7中,context包被引入标准库中,它成为考虑并发问题时的一个标准风格。
2、理解Context包
参考链接:快速掌握 Golang context 包,简单示例
Context应该是一个链式的调用,通过WithCancel,WithDeadline,WithTimeout或WithValue派生出新的Context。
当父Context被取消时,其派生的所有子Context都将被取消。
调用context.WithXXX函数返回值有两个,一个是新的Context(后文以“子Context”代指),另一个是CancelFunc取消函数。
如果调用CancelFunc将取消子Context,移除父Context对子Context的引用,并停止所有子Context定时器。
举个例子:父Context->子Context1->子Context2
如果我们调用了 子Context1 的CancelFunc,那么子Context1 和 子Context2 都会被取消掉,同时父Context对子Context的引用也会被移除,就只剩下父Context了。如果没有调用CancelFunc,那么将泄漏子Context,直到父Context被取消或定时器触发为止。
go vet 工具检查所有流程,控制路径上使用 CancelFuncs。
P.S. CancelFunc 的定义如下,本质就是 func() .
1 2 3 4 5 6 7 |
// go ver1.14.4, File: context.go, Line 220 - 224 // A CancelFunc tells an operation to abandon its work. // A CancelFunc does not wait for the work to stop. // A CancelFunc may be called by multiple goroutines simultaneously. // After the first call, subsequent calls to a CancelFunc do nothing. type CancelFunc func() |
Context接口:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
// go ver1.14.4, File: context.go, Line 58 - 179 // A Context carries a deadline, a cancellation signal, and other values across // API boundaries. // // Context's methods may be called by multiple goroutines simultaneously. type Context interface { // Deadline returns the time when work done on behalf of this context // should be canceled. Deadline returns ok==false when no deadline is // set. Successive calls to Deadline return the same results. Deadline() (deadline time.Time, ok bool) // Done returns a channel that's closed when work done on behalf of this // context should be canceled. Done may return nil if this context can // never be canceled. Successive calls to Done return the same value. // The close of the Done channel may happen asynchronously, // after the cancel function returns. // // WithCancel arranges for Done to be closed when cancel is called; // WithDeadline arranges for Done to be closed when the deadline // expires; WithTimeout arranges for Done to be closed when the timeout // elapses. // // Done is provided for use in select statements: // // // Stream generates values with DoSomething and sends them to out // // until DoSomething returns an error or ctx.Done is closed. // func Stream(ctx context.Context, out chan<- Value) error { // for { // v, err := DoSomething(ctx) // if err != nil { // return err // } // select { // case <-ctx.Done(): // return ctx.Err() // case out <- v: // } // } // } // // See https://blog.golang.org/pipelines for more examples of how to use // a Done channel for cancellation. Done() <-chan struct{} // If Done is not yet closed, Err returns nil. // If Done is closed, Err returns a non-nil error explaining why: // Canceled if the context was canceled // or DeadlineExceeded if the context's deadline passed. // After Err returns a non-nil error, successive calls to Err return the same error. Err() error // Value returns the value associated with this context for key, or nil // if no value is associated with key. Successive calls to Value with // the same key returns the same result. // // Use context values only for request-scoped data that transits // processes and API boundaries, not for passing optional parameters to // functions. // // A key identifies a specific value in a Context. Functions that wish // to store values in Context typically allocate a key in a global // variable then use that key as the argument to context.WithValue and // Context.Value. A key can be any type that supports equality; // packages should define keys as an unexported type to avoid // collisions. // // Packages that define a Context key should provide type-safe accessors // for the values stored using that key: // // // Package user defines a User type that's stored in Contexts. // package user // // import "context" // // // User is the type of value stored in the Contexts. // type User struct {...} // // // key is an unexported type for keys defined in this package. // // This prevents collisions with keys defined in other packages. // type key int // // // userKey is the key for user.User values in Contexts. It is // // unexported; clients use user.NewContext and user.FromContext // // instead of using this key directly. // var userKey key // // // NewContext returns a new Context that carries value u. // func NewContext(ctx context.Context, u *User) context.Context { // return context.WithValue(ctx, userKey, u) // } // // // FromContext returns the User value stored in ctx, if any. // func FromContext(ctx context.Context) (*User, bool) { // u, ok := ctx.Value(userKey).(*User) // return u, ok // } Value(key interface{}) interface{} } |
Context接口关注以下四个函数:
Deadline()函数,Deadline() (deadline time.Time, ok bool) 返回截止时间time和ok值,当没有deadline的情况下,返回ok==false
Done() 函数,Done() <-chan struct{} 返回一个channel,当timeout或调用CancelFunc时channel会被close掉。
Err()函数,Err() error 返回一个错误,该Context为什么会被取消掉。
Value()函数,Value(key interface{}) interface{} 返回Context中与Key关联的Value,如果没有则返回nil
所有方法:
1 2 3 4 5 6 7 |
func Background() Context func TODO() Context func WithCancel(parent Context) (ctx Context, cancel CancelFunc) func WithDeadline(parent Context, deadline time.Time) (Context, CancelFunc) func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) func WithValue(parent Context, key, val interface{}) Context |
Background 和 TODO方法:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
// go ver1.14.4, File: context.go, Line 199 - 218 var ( background = new(emptyCtx) todo = new(emptyCtx) ) // Background returns a non-nil, empty Context. It is never canceled, has no // values, and has no deadline. It is typically used by the main function, // initialization, and tests, and as the top-level Context for incoming // requests. func Background() Context { return background } // TODO returns a non-nil, empty Context. Code should use context.TODO when // it's unclear which Context to use or it is not yet available (because the // surrounding function has not yet been extended to accept a Context // parameter). func TODO() Context { return todo } |
Background 和 TODO 方法返回emptyCtx(后文介绍)。
TODO的预期目的是作为一个占位符,当你不知道使用哪个Context的时候就可以用它。
一般我们将它们作为Context的根,来往下派生。
emptyCtx其实是一个int, 注意 type emptyCtx int 这一句代码。 其实现部分源码如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
// go ver1.14.4, File: context.go, Line 169 - 197 // An emptyCtx is never canceled, has no values, and has no deadline. It is not // struct{}, since vars of this type must have distinct addresses. type emptyCtx int func (*emptyCtx) Deadline() (deadline time.Time, ok bool) { return } func (*emptyCtx) Done() <-chan struct{} { return nil } func (*emptyCtx) Err() error { return nil } func (*emptyCtx) Value(key interface{}) interface{} { return nil } func (e *emptyCtx) String() string { switch e { case background: return "context.Background" case todo: return "context.TODO" } return "unknown empty Context" } |
可以看到,emptyCtx本质是一个int,然后通过方法实现了Context接口的Deadline、Done、Err和Value四个接口函数。
根据介绍,它是一个永远不被取消的Context。
WithCancel方法:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
// go ver1.14.4, File: context.go, Line 226 - 241 // WithCancel returns a copy of parent with a new Done channel. The returned // context's Done channel is closed when the returned cancel function is called // or when the parent context's Done channel is closed, whichever happens first. // // Canceling this context releases resources associated with it, so code should // call cancel as soon as the operations running in this Context complete. func WithCancel(parent Context) (ctx Context, cancel CancelFunc) { c := newCancelCtx(parent) propagateCancel(parent, &c) return &c, func() { c.cancel(true, Canceled) } } // newCancelCtx returns an initialized cancelCtx. func newCancelCtx(parent Context) cancelCtx { return cancelCtx{Context: parent} } |
WithCancel函数返回 cancelCtx(它包含父Context的拷贝 + new done channel…) 和 CancelFunc.
这里注意 newCancelCtx 的实现,同时结合下文的cancelCtx的实现来看。
简单理解调用propagateCancel函数的作用是将父Context和子Context的Cancel函数绑定起来。
这样父Context取消的时候,调用子Context的CancelFunc。
propagateCancel函数具体源码位置在: context.go Line 247行。
其内部实现用到了一个children map来存储这个关系(这个map在下文cancelCtx结构体内)。
cancelCtx解析:
1 2 3 4 5 6 7 8 9 10 11 12 |
// go ver1.14.4, File: context.go, Line 339 - 348 // A cancelCtx can be canceled. When canceled, it also cancels any children // that implement canceler. type cancelCtx struct { Context mu sync.Mutex // protects following fields done chan struct{} // created lazily, closed by first cancel call children map[canceler]struct{} // set to nil by the first cancel call err error // set to non-nil by the first cancel call } |
《Go程序设计语言》Page81. Go允许我们定义不带名称的结构体成员,只需要指定类型即可;这种结构体成员成为匿名成员。
Context 就是 cancelCtx的匿名成员,这里不展示cancelCtx的具体四个方法的实现。
我们关注一下 children 这个map结构,它是我们上文提到的父子Context绑定的存储map.
WithCancel示例代码:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
package main import ( "context" "fmt" ) func main() { gen := func(ctx context.Context) <-chan int { dst := make(chan int) go func() { defer close(dst) n := 1 for { select { case <-ctx.Done(): return case dst <- n: n++ } } }() return dst } ctx, cancel := context.WithCancel(context.Background()) defer cancel() for n := range gen(ctx) { fmt.Println(n) if n == 10 { break } } } |
这里gen函数的意义是返回一个自动生成器,它会一直生产int整数到它返回的channel里面,直到它被取消。
示例函数结束时,defer 调用 cancel 方法,gen goroutine 将返回,不会泄漏。
同时注意,gen goroutine里面的dst也需要defer close.
WithDeadline方法:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
// go ver1.14.4, File: context.go, Line 418 - 450 // WithDeadline returns a copy of the parent context with the deadline adjusted // to be no later than d. If the parent's deadline is already earlier than d, // WithDeadline(parent, d) is semantically equivalent to parent. The returned // context's Done channel is closed when the deadline expires, when the returned // cancel function is called, or when the parent context's Done channel is // closed, whichever happens first. // // Canceling this context releases resources associated with it, so code should // call cancel as soon as the operations running in this Context complete. func WithDeadline(parent Context, d time.Time) (Context, CancelFunc) { if cur, ok := parent.Deadline(); ok && cur.Before(d) { // The current deadline is already sooner than the new one. return WithCancel(parent) } c := &timerCtx{ cancelCtx: newCancelCtx(parent), deadline: d, } propagateCancel(parent, c) dur := time.Until(d) if dur <= 0 { c.cancel(true, DeadlineExceeded) // deadline has already passed return c, func() { c.cancel(false, Canceled) } } c.mu.Lock() defer c.mu.Unlock() if c.err == nil { c.timer = time.AfterFunc(dur, func() { c.cancel(true, DeadlineExceeded) }) } return c, func() { c.cancel(true, Canceled) } } |
WithDeadline函数返回timerCtx(父Context的一个拷贝 + deadline) 和 CancelFunc.
这里需要注意,如果父Context的截止时间已经早于d,其语义等效为父Context截止。
另外,timerCtx的期限时间满后、取消函数调用、父Context的DoneChannel关闭,任一发生都会取消子Context。
我们来看一下timerCtx的实现:
1 2 3 4 5 6 7 8 9 10 11 |
// go ver1.14.4, File: context.go, Line 452 - 460 // A timerCtx carries a timer and a deadline. It embeds a cancelCtx to // implement Done and Err. It implements cancel by stopping its timer then // delegating to cancelCtx.cancel. type timerCtx struct { cancelCtx timer *time.Timer // Under cancelCtx.mu. deadline time.Time } |
它拥有一个匿名的 cancelCtx, 另外需要注意的是,它的定时器是基于cancelCtx的Mutex来实现同步的。
WithDeadline示例代码:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
package main import ( "context" "fmt" "time" ) func main() { d := time.Now().Add(50 * time.Millisecond) ctx, cancel := context.WithDeadline(context.Background(), d) defer cancel() select { case <-time.After(1 * time.Second): fmt.Println("overslept") case <-ctx.Done(): fmt.Println(ctx.Err()) } } |
其实很简单,timerCtx设置了Deadline为当前时间的50ms后,select语句等1秒或者timerCtx到期。
WithTimeout方法:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
// go ver1.14.4, File: context.go, Line 486 - 498 // WithTimeout returns WithDeadline(parent, time.Now().Add(timeout)). // // Canceling this context releases resources associated with it, so code should // call cancel as soon as the operations running in this Context complete: // // func slowOperationWithTimeout(ctx context.Context) (Result, error) { // ctx, cancel := context.WithTimeout(ctx, 100*time.Millisecond) // defer cancel() // releases resources if slowOperation completes before timeout elapses // return slowOperation(ctx) // } func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) { return WithDeadline(parent, time.Now().Add(timeout)) } |
这里可以很明显的看出,WithTimeout的实现是基于 Deadline来实现的。
直接给出示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
package main import ( "context" "fmt" "time" ) func main() { ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) defer cancel() select { case <-time.After(1 * time.Second): fmt.Println("overslept") case <-ctx.Done(): fmt.Println(ctx.Err()) } } |
运行之后会发现,Timeout 和 Deadline的输出是一致的,都是 context deadline exceeded
WithValue方法:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
// go ver1.14.4, File: context.go, Line 500 - 528 // WithValue returns a copy of parent in which the value associated with key is // val. // // Use context Values only for request-scoped data that transits processes and // APIs, not for passing optional parameters to functions. // // The provided key must be comparable and should not be of type // string or any other built-in type to avoid collisions between // packages using context. Users of WithValue should define their own // types for keys. To avoid allocating when assigning to an // interface{}, context keys often have concrete type // struct{}. Alternatively, exported context key variables' static // type should be a pointer or interface. func WithValue(parent Context, key, val interface{}) Context { if key == nil { panic("nil key") } if !reflectlite.TypeOf(key).Comparable() { panic("key is not comparable") } return &valueCtx{parent, key, val} } // A valueCtx carries a key-value pair. It implements Value for that key and // delegates all other calls to the embedded Context. type valueCtx struct { Context key, val interface{} } |
可以看到,返回的是一个ValueCtx,它本质是一个Context + key-value。
WithValue示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
package main import ( "context" "fmt" ) func main() { type favContextKey string f := func(ctx context.Context, k favContextKey) { if v := ctx.Value(k); v != nil { fmt.Println("found value:", v) return } fmt.Println("key not found:", k) } k := favContextKey("language") ctx := context.WithValue(context.Background(), k, "Go") f(ctx, k) f(ctx, favContextKey("color")) ctx2 := context.WithValue(ctx, k, "Go2") f(ctx2, k) f(ctx, favContextKey("color")) ctx3 := context.WithValue(ctx2, favContextKey("color"), "White") f(ctx3, k) f(ctx3, favContextKey("color")) } |
具体的好处应该是:这样我们不仅保留了根节点原有的值,还在子孙节点中加入了新的值(继承)。
注意:若存在Key相同,则会被覆盖,这里具体看一下Value函数的实现方案:
1 2 3 4 5 6 |
func (c *valueCtx) Value(key interface{}) interface{} { if c.key == key { return c.val } return c.Context.Value(key) } |
它先判断自己的key是否符合,如果不符合就去找父Context的key,如果还不符合会继续往上找(P.S. 其他context都是这样类似的实现)
所以相同的Key会被覆盖,不同的key则会继承。
原理小结:
context包通过构建树形结构的Context,实现了上层goroutine对下层goroutine的控制。
使用原则:
- 不要把Context存在一个结构体当中。Context变量应该作为第一个参数显式地传入函数,一般命名为ctx。
- 即使方法允许,也不要传递一个空的Context,如果不确定需要什么Context,就传递一个context.TODO()。
- 使用context的Value相关方法只应该用于在程序和接口中传递请求相关的数据,不要用它传递一些可选参数。
- 同一个Context可以传递到不同的goroutine中,Context在多个goroutine中是安全的。
- 接收子Context的goroutine中,应该对子Context的Done Channel进行监控,一旦信道关闭就应当停止处理,释放资源并返回。