mirror
/
ants
mirror of https://github.com/panjf2000/ants.git
1
0
Fork 1
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'develop'

This commit is contained in:
Andy Pan 2018-12-06 00:50:46 +08:00
parent aad3999c75 08469e8dc2
commit a1a68f278d
3 changed files with 39 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
README.md
View File

@ -13,7 +13,7 @@
[中文项目说明](README_ZH.md) | [Project Tutorial](http://blog.taohuawu.club/article/goroutine-pool)
Package ants implements a fixed goroutine pool for managing and recycling a massive number of goroutines, allowing developers to limit the number of goroutines that created by your concurrent programs.
Package `ants` implements a fixed goroutine pool for managing and recycling a massive number of goroutines, allowing developers to limit the number of goroutines that created in your concurrent programs.
## Features:
@ -36,7 +36,7 @@ glide get github.com/panjf2000/ants
```
## How to use
If your program will generate a massive number of goroutines and you don't want them to consume a vast amount of memory, with ants, all you need to do is to import ants package and submit all your tasks to the default limited pool created when ants was imported:
If your program will generate a massive number of goroutines and you don't want them to consume a vast amount of memory, with `ants`, all you need to do is to import `ants` package and submit all your tasks to the default limited pool created when `ants` was imported:
``` go
package main
@ -68,7 +68,7 @@ func main() {
runTimes := 1000
// use the common pool
// Uses the common pool
var wg sync.WaitGroup
for i := 0; i < runTimes; i++ {
wg.Add(1)
@ -81,14 +81,14 @@ func main() {
fmt.Printf("running goroutines: %d\n", ants.Running())
fmt.Printf("finish all tasks.\n")
// use the pool with a function
// set 10 the size of goroutine pool and 1 second for expired duration
// Uses the pool with a function,
// sets 10 to the size of goroutine pool and 1 second for expired duration
p, _ := ants.NewPoolWithFunc(10, func(i interface{}) {
myFunc(i)
wg.Done()
})
defer p.Release()
// submit tasks
// Submits tasks
for i := 0; i < runTimes; i++ {
wg.Add(1)
p.Serve(int32(i))
@ -141,7 +141,7 @@ func main() {
request := &Request{Param: param, Result: make(chan []byte)}
// Throttle the requests with ants pool. This process is asynchronous and
// Throttles the requests with ants pool. This process is asynchronous and
// you can receive a result from the channel defined outside.
if err := pool.Serve(request); err != nil {
http.Error(w, "throttle limit error", http.StatusInternalServerError)
@ -154,34 +154,34 @@ func main() {
}
```
## Submit tasks
## Submits tasks
Tasks can be submitted by calling `ants.Submit(func())`
```go
ants.Submit(func(){})
```
## Custom limited pool
Ants also supports custom limited pool. You can use the `NewPool` method to create a pool with the given capacity, as following:
## Customizes limited pool
`ants` also supports customizing limited pool. You can use the `NewPool` method to create a pool with the given capacity, as following:
``` go
// set 10000 the size of goroutine pool
// Sets 10000 the size of goroutine pool
p, _ := ants.NewPool(10000)
// submit a task
// Submits a task
p.Submit(func(){})
```
## Readjusting pool capacity
You can change ants pool capacity at any time with `ReSize(int)`:
## Tuning pool capacity
You can change `ants` pool capacity at any time with `ReSize(int)`:
``` go
pool.ReSize(1000) // Readjust its capacity to 1000
pool.ReSize(100000) // Readjust its capacity to 100000
pool.ReSize(1000) // Tunes its capacity to 1000
pool.ReSize(100000) // Tunes its capacity to 100000
```
Don't worry about the synchronous problems in this case, this method is thread-safe.
## About sequence
All the tasks submitted to ants pool will not be guaranteed to be processed in order, because those tasks distribute among a series of concurrent workers, thus those tasks are processed concurrently.
All the tasks submitted to `ants` pool will not be guaranteed to be processed in order, because those tasks distribute among a series of concurrent workers, thus those tasks are processed concurrently.
## Benchmarks
@ -195,11 +195,11 @@ Go1.9
<div align="center"><img src="ants_benchmarks.png"/></div>
In that benchmark-picture, the first and second benchmarks performed test with 1M tasks and the rest of benchmarks performed test with 10M tasks, both unlimited goroutines and ants pool, and the capacity of this ants goroutine-pool was limited to 50K.
In that benchmark-picture, the first and second benchmarks performed test with 1M tasks and the rest of benchmarks performed test with 10M tasks, both unlimited goroutines and `ants` pool, and the capacity of this `ants` goroutine-pool was limited to 50K.
- BenchmarkGoroutine-4 represent the benchmarks with unlimited goroutines in golang.
- BenchmarkGoroutine-4 represents the benchmarks with unlimited goroutines in golang.
- BenchmarkPoolGroutine-4 represent the benchmarks with a ants pool.
- BenchmarkPoolGroutine-4 represents the benchmarks with a `ants` pool.
The test data above is a basic benchmark and the more detailed benchmarks will be uploaded later.
@ -207,7 +207,7 @@ The test data above is a basic benchmark and the more detailed benchmarks will
![](benchmark_pool.png)
In that benchmark-picture, the first and second benchmarks performed test with 1M tasks and the rest of benchmarks performed test with 10M tasks, both unlimited goroutines and ants pool, and the capacity of this ants goroutine-pool was limited to 50K.
In that benchmark-picture, the first and second benchmarks performed test with 1M tasks and the rest of benchmarks performed test with 10M tasks, both unlimited goroutines and `ants` pool, and the capacity of this `ants` goroutine-pool was limited to 50K.
**As you can see, `ants` can up to 2x faster than goroutines without pool (10M tasks) and it only consumes half memory comparing with goroutines without pool. (both 1M and 10M tasks)**
@ -215,7 +215,7 @@ In that benchmark-picture, the first and second benchmarks performed test with 1
![](ants_bench_poolwithfunc.png)
### Throughput ( situation for only submitting tasks and need not waiting for all the tasks to be completed)
### Throughput (it is suitable for scenarios where asynchronous tasks are submitted without concern for results)
#### 100K tasks
@ -231,7 +231,7 @@ In that benchmark-picture, the first and second benchmarks performed test with 1
There was only the test of `ants` Pool because my computer was crash when it reached 10M goroutines without pool.
**As you can see, `ants` can up to 2x~6x faster than goroutines without pool and the memory consumption is reduced by 10 to 20 times.**
**In conclusion, `ants` can up to 2x~6x faster than goroutines without pool and the memory consumption is reduced by 10 to 20 times.**
[1]: https://travis-ci.com/panjf2000/ants.svg?branch=master
[2]: https://travis-ci.com/panjf2000/ants

24
README_ZH.md
View File

@ -67,7 +67,7 @@ func main() {
runTimes := 1000
// use the common pool
// Uses the common pool
var wg sync.WaitGroup
for i := 0; i < runTimes; i++ {
wg.Add(1)
@ -80,14 +80,14 @@ func main() {
fmt.Printf("running goroutines: %d\n", ants.Running())
fmt.Printf("finish all tasks.\n")
// use the pool with a function
// set 10 the size of goroutine pool and 1 second for expired duration
// Uses the pool with a function,
// sets 10 to the size of goroutine pool and 1 second for expired duration
p, _ := ants.NewPoolWithFunc(10, func(i interface{}) {
myFunc(i)
wg.Done()
})
defer p.Release()
// submit tasks
// Submits tasks
for i := 0; i < runTimes; i++ {
wg.Add(1)
p.Serve(int32(i))
@ -96,7 +96,7 @@ func main() {
fmt.Printf("running goroutines: %d\n", p.Running())
fmt.Printf("finish all tasks, result is %d\n", sum)
}
```
```
## 与http server集成
```go
@ -140,7 +140,7 @@ func main() {
request := &Request{Param: param, Result: make(chan []byte)}
// Throttle the requests with ants pool. This process is asynchronous and
// Throttles the requests with ants pool. This process is asynchronous and
// you can receive a result from the channel defined outside.
if err := pool.Serve(request); err != nil {
http.Error(w, "throttle limit error", http.StatusInternalServerError)
@ -163,9 +163,9 @@ ants.Submit(func(){})
`ants`支持实例化使用者自己的一个 Pool ，指定具体的池容量；通过调用 `NewPool` 方法可以实例化一个新的带有指定容量的 Pool ，如下：
``` go
// set 10000 the size of goroutine pool
// Sets 10000 the size of goroutine pool
p, _ := ants.NewPool(10000)
// submit a task
// Submits a task
p.Submit(func(){})
```
@ -173,8 +173,8 @@ p.Submit(func(){})
需要动态调整协程池容量可以通过调用`ReSize(int)`：
``` go
pool.ReSize(1000) // Readjust its capacity to 1000
pool.ReSize(100000) // Readjust its capacity to 100000
pool.ReSize(1000) // Tuning its capacity to 1000
pool.ReSize(100000) // Tuning its capacity to 100000
```
该方法是线程安全的。
@ -213,7 +213,7 @@ Go1.9
**因为`PoolWithFunc`这个Pool只绑定一个任务函数，也即所有任务都是运行同一个函数，所以相较于`Pool`对原生goroutine在执行速度和内存消耗的优势更大，上面的结果可以看出，执行速度可以达到原生goroutine的300%，而内存消耗的优势已经达到了两位数的差距，原生goroutine的内存消耗达到了`ants`的35倍且原生goroutine的每次执行的内存分配次数也达到了`ants`45倍，1000w的任务量，`ants`的初始分配容量是5w，因此它完成了所有的任务依旧只使用了5w个goroutine！事实上，`ants`的Goroutine Pool的容量是可以自定义的，也就是说使用者可以根据不同场景对这个参数进行调优直至达到最高性能。**
### 吞吐量测试（使用于那种只管提交异步任务而无须关心结果的场景）
### 吞吐量测试（适用于那种只管提交异步任务而无须关心结果的场景）
#### 10w 任务量
@ -229,7 +229,7 @@ Go1.9
1000w任务量的场景下，我的电脑已经无法支撑 golang 的原生 goroutine 并发，所以只测出了使用`ants`池的测试结果。
**从该demo测试吞吐性能对比可以看出，使用ants的吞吐性能相较于原生goroutine可以保持在2-6倍的性能压制，而内存消耗则可以达到10-20倍的节省优势。**
**从该demo测试吞吐性能对比可以看出，使用`ants`的吞吐性能相较于原生goroutine可以保持在2-6倍的性能压制，而内存消耗则可以达到10-20倍的节省优势。**
[1]: https://travis-ci.com/panjf2000/ants.svg?branch=master
[2]: https://travis-ci.com/panjf2000/ants

8
examples/main.go
View File

@ -49,7 +49,7 @@ func main() {
runTimes := 1000
// use the common pool
// Uses the common pool
var wg sync.WaitGroup
for i := 0; i < runTimes; i++ {
wg.Add(1)
@ -62,14 +62,14 @@ func main() {
fmt.Printf("running goroutines: %d\n", ants.Running())
fmt.Printf("finish all tasks.\n")
// use the pool with a function
// set 10 the size of goroutine pool and 1 second for expired duration
// Uses the pool with a function,
// sets 10 to the size of goroutine pool and 1 second for expired duration
p, _ := ants.NewPoolWithFunc(10, func(i interface{}) {
myFunc(i)
wg.Done()
})
defer p.Release()
// submit tasks
// Submits tasks
for i := 0; i < runTimes; i++ {
wg.Add(1)
p.Serve(int32(i))