df54454102
Feature: support operate the underlying incoming data for uploading large file
...
For file uploading, default behaviour will `ReadForm` and write data to new temporary file if oversize `maxMemory`.
If temporary directory size is too small, it will fail for uploading large file and even consume OS memory since copy to buffer.
Inspired by Flask, Flask `request.stream.read` will read underlying socket data directly if payload found. Refer to: https://werkzeug.palletsprojects.com/en/2.0.x/wsgi/#werkzeug.wsgi.LimitedStream
This method makes assumptions because the request payload exists for operating `POST/PATCH/PUT`.
A simple demo for uploading 5G large file in low memory based on docker:
```
[root@control-master tmp]# docker run -ti --rm --memory 1G --tmpfs /tmp:rw,size=1G,mode=1777 -v $(pwd)/go:/go golang:1.16.5 /bin/bash
root@b672a98e5314:/go/testmodules# cat main.go
package main
import (
"fmt"
"github.com/gin-gonic/gin"
"io"
"os"
"path/filepath"
)
func main() {
engine := gin.Default()
engine.POST("/streamupload", func(c *gin.Context) {
if c.GetHeader("Content-Type") != "application/octet-stream" {
err := fmt.Errorf("required octet-stream")
c.AbortWithStatusJSON(400, map[string]string{"message": err.Error()})
return
}
info, err := os.Create(filepath.Join("/home", "foo"))
if err != nil {
c.AbortWithStatusJSON(400, gin.H{"message": "Create: "+err.Error()})
return
}
defer info.Close()
_, err = io.Copy(info, c.Request.Body)
if err != nil {
c.AbortWithStatusJSON(400, gin.H{"message": "Copy: "+err.Error()})
return
}
c.JSON(200, map[string]string{"message": "ok stream"})
})
if err := engine.Run("0.0.0.0:19090"); err != nil {
panic(err)
}
}
root@b53810b3e294:/go/testmodules# GOTMPDIR=/opt/ go run main.go
[root@control-master ~]# dd if=/dev/zero of=5G bs=1M count=5170 status=progress
[root@control-master ~]# curl -vvv -H "Content-Type:application/octet-stream" -T 5G -X POST 172.17.0.2:19090/streamupload
```
Signed-off-by: Chenyang Yan <memory.yancy@gmail.com>
2022-01-26 21:26:32 +08:00
580e7da6ee
Remove incorrect comments about context.Bind() and improve docs ( #3028 )
2022-01-20 22:33:35 +08:00
1b28e2b030
Fix typo ( #3023 )
...
Co-authored-by: lin.song <lin.song@rakuten.com>
2022-01-12 22:12:32 +08:00
336ce0d475
Bump github.com/goccy/go-json from 0.8.1 to 0.9.0 ( #3021 )
...
Bumps [github.com/goccy/go-json](https://github.com/goccy/go-json ) from 0.8.1 to 0.9.0.
- [Release notes](https://github.com/goccy/go-json/releases )
- [Changelog](https://github.com/goccy/go-json/blob/master/CHANGELOG.md )
- [Commits](https://github.com/goccy/go-json/compare/v0.8.1...v0.9.0 )
---
updated-dependencies:
- dependency-name: github.com/goccy/go-json
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-11 08:15:12 +08:00
6868ed18cc
Bump github.com/go-playground/validator/v10 from 10.9.0 to 10.10.0 ( #3013 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-08 17:25:31 +08:00
94153d1e19
test: expose performRequest func ( #3012 )
2022-01-02 19:07:44 +08:00
01363191be
fix: typo ( #3006 )
2022-01-02 14:04:07 +08:00
d062a6a615
docs: redirect to the correct line of code ( #2998 )
2021-12-26 08:02:01 +08:00
1538ece13f
Update README.md ( #2997 )
2021-12-23 07:51:12 +08:00
8a0f95cc71
fix: typo ( #2993 )
2021-12-20 17:42:54 +08:00
92a988d761
fix: description error ( #2988 )
...
* fix:typo
* fix: typo
* fix: wrong when wildcard follows named param
* fix: description error
* update author list
* fix:typo
* fix: description error
2021-12-18 19:40:47 +08:00
fb5f045417
fix: description error ( #2986 )
2021-12-15 23:27:23 +08:00
7d189814cb
fix: wrong when wildcard follows named param ( #2983 )
2021-12-12 13:30:33 +08:00
e56d18f19d
Update README.md ( #2985 )
2021-12-11 16:24:10 +08:00
3973c77263
Bump github.com/goccy/go-json from 0.7.10 to 0.8.1 ( #2981 )
...
Bumps [github.com/goccy/go-json](https://github.com/goccy/go-json ) from 0.7.10 to 0.8.1.
- [Release notes](https://github.com/goccy/go-json/releases )
- [Changelog](https://github.com/goccy/go-json/blob/master/CHANGELOG.md )
- [Commits](https://github.com/goccy/go-json/compare/v0.7.10...v0.8.1 )
---
updated-dependencies:
- dependency-name: github.com/goccy/go-json
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-07 08:09:09 +08:00
ba7e58989c
fix: typo ( #2977 )
...
* fix:typo
* fix: typo
2021-12-05 08:41:25 +08:00
504ec594f8
fix:typo ( #2975 )
2021-12-03 14:49:51 +08:00
0be805a675
TrustedProxies: Add default IPv6 support and refactor ( #2967 )
2021-12-03 14:49:16 +08:00
830a63d244
fix: typo ( #2973 )
...
* fix: typo
* fix: grammar error
2021-12-02 18:00:24 +08:00
bc2417fc40
fix: description error ( #2968 )
2021-11-30 08:36:36 +08:00
a06d546f5c
prettify error message for catch-all conflict with existing path segment ( #2934 )
2021-11-28 09:26:17 +08:00
f068099d0d
Update README.md ( #2954 )
2021-11-28 09:24:14 +08:00
ffb3b73430
fix: description error ( #2961 )
2021-11-26 16:38:10 +08:00
823adfc91a
fix: typo ( #2958 )
2021-11-25 18:12:08 +08:00
4d7c4ec36f
chore(docs): Bump to v1.7.7 ( #2952 )
...
* Perfect TrustedProxies feature
* some typo fix
* fix some typo
* bump to v1.7.6
* remove 'retract'
* Revert "remove 'retract'"
This reverts commit 3fc2ab44b3
2021-11-24 21:52:52 +08:00
57ede9c95a
Export struct sliceValidateError to allow error casting and rename it as ( #2777 )
...
SliceValidationError
Co-authored-by: Emeric de Bernis <emeric.debernis@adevinta.com>
2021-11-21 21:45:03 +08:00
6aee45608d
Fix grammar ( #2933 )
2021-11-11 08:07:55 +08:00
89a159bdd9
Bump golangci-lint version ( #2929 )
2021-11-03 22:13:24 +08:00
efa3175007
Update version.go ( #2923 )
2021-11-01 08:33:38 +08:00
cbdd47a7e1
fix tsr with mixed static and wildcard paths ( #2924 )
2021-11-01 08:21:37 +08:00
d4e72a17f7
Fix typo ( #2926 )
2021-10-31 09:23:08 +08:00
1c2aa59b20
fix the misplacement of adding slashes ( #2847 )
2021-10-26 18:15:29 +08:00
eb75ce0ff5
adjust the routergroup Any method ( #2701 )
2021-10-24 09:31:13 +08:00
2d3d6d2f13
Provide custom options of TrustedPlatform for another CDN services ( #2906 )
...
* refine TrustedPlatform and docs
* refactor for switch
* refactor switch to if statement
2021-10-24 08:34:03 +08:00
3fe928994b
Update the code logic for latestNode in tree.go ( #2897 )
2021-10-23 11:58:57 +08:00
34ae7777ca
README.md: fix a typo ( #2913 )
2021-10-22 21:26:14 +08:00
464535ff94
Bump github.com/goccy/go-json from 0.7.9 to 0.7.10 ( #2905 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-19 07:55:14 +08:00
5929d52171
ClientIP: check every proxy for trustiness ( #2844 )
2021-10-09 08:38:51 +08:00
21125bbb3f
Bump github.com/goccy/go-json from 0.7.8 to 0.7.9 ( #2891 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-06 09:38:42 +08:00
39181329de
Tidy: Complete TrustedProxies feature ( #2887 )
2021-10-06 09:37:25 +08:00
1a2bc0e7cb
setted typo fix: There’s no such word as `setted`, `set` is set, set, setting ( #2886 )
2021-09-30 10:04:28 +08:00
6d75aba83f
Quick Fix c.ClientIP() mistakely parsing to 127.0.0.1 for who not using r.Run() to run http server ( #2832 )
2021-09-29 19:26:02 +08:00
97b3c0d88a
Fix grammatical and spelling errors in context.go ( #2883 )
2021-09-29 07:35:06 +08:00
f469c1be39
Add gosec ( #2882 )
2021-09-28 11:37:31 +08:00
d6534ccf38
turn on HandleMethodNotAllowed when using NoMethod #2871 ( #2872 )
2021-09-28 09:45:50 +08:00
ef168679ce
Correcting grammatical errors in README file ( #2880 )
2021-09-28 08:57:36 +08:00
e052bf31aa
Bump github.com/json-iterator/go from 1.1.11 to 1.1.12 ( #2865 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-21 15:23:31 +08:00
71f7087097
golangci(lint) : more linters ( #2870 )
2021-09-21 15:22:21 +08:00
3a6f18f32f
fixed SetOutput() panics on go 1.17 ( #2861 )
...
* fixed SetOutput() panics on go 1.17
* update go.sum
2021-09-08 11:30:55 +08:00
ae349b4015
Fix typo ( #2860 )
2021-09-07 13:05:19 +08:00
Chenyang Yan