Commit Graph

317 Commits

Author SHA1 Message Date
Michalis Kargakis 68b71565dd
Add README section for GCS Fs 2021-12-22 11:03:34 +01:00
Michalis Kargakis 21b977b251
Merge branch 'master' into gcs 2021-12-22 11:03:13 +01:00
Raphael Pour 949437f3b7 Readme: fix quotes of step 3
Single quotes are reserved for runes and therefore not allowed for strings in Go: `more than one character in rune literal`
2021-11-10 15:04:45 +01:00
Michalis Kargakis f0ee686ef7
Merge pull request #312 from tklauser/bump-deps
go.mod: bump dependencies to latest version
2021-11-10 10:36:00 +01:00
Tobias Klauser d461d4f875 go.mod: bump dependencies to latest version
github.com/pkg/sftp to v1.13.1 (this fixes a security vulnerability)
golang.org/x/crypto to v0.0.0-20211108221036-ceb1ce70b4fa
golang.org/x/text to v0.3.7
2021-11-10 10:29:42 +01:00
Martin Karlsch 090e652a76 read GOOGLE_APPLICATION_CREDENTIALS_JSON if it is set 2021-10-05 18:17:13 +02:00
VOvchinnikov c1220175da Added implementation to cover the new `Mkdir` & `MkdirAll` cases 2021-08-05 18:38:36 +02:00
VOvchinnikov 13ee679c8e Added test cases for `Mkdir` & `MkdirAll` to cover empty folder names 2021-08-05 18:24:24 +02:00
Vasily Ovchinnikov 78acf3b074 Changed abstraction to operate on client, not bucket 2021-04-14 13:08:40 +02:00
Vasily Ovchinnikov 1a0f20777f Add GCS Fs implementation 2021-04-12 20:07:21 +02:00
Michail Kargakis bc94f58bed
Merge pull request #287 from tdakkota/fix/issue-286
Fix incorrect (*mem.File).WriteAt
2021-03-20 15:22:17 +01:00
Michail Kargakis b9650fb6c8
Merge pull request #291 from xakep666/feature/iofs
Add adapter to allow afero.Fs usage as io/fs.FS
2021-03-20 15:18:02 +01:00
Vladimir Stolyarov 164e24d3b1
Add reverse wrapper from io.FS to afero.Fs
afero.Fs/File requires mutating methods implementation but they will always return error because fs.FS is read-only.
ReadAt and Seek will return error if underlying fs.File doesn't implement them.
2021-03-13 22:50:37 +03:00
Vladimir Stolyarov c597fe5aa8
Add adapter to allow afero.Fs usage as io/fs.FS
It will not affect users of go older than 1.16 because files with `io/fs` import marked with build constraints
2021-03-13 21:09:09 +03:00
tdakkota 7b47fea3fc
fix(mem): do not overwrite old data in some cases (fixes #286) 2021-02-12 13:02:47 +03:00
Michail Kargakis 32b5faae5b
Merge pull request #280 from mfuterko/master
Fix a typo
2020-12-09 13:17:20 +01:00
Mike Futerko f3723fe04d Fix a typo 2020-12-08 20:57:36 +02:00
Michail Kargakis 816954d9ff
Remove go1.13 from travis build matrix 2020-12-07 20:53:32 +01:00
Michail Kargakis 89dd0ada86
Merge pull request #279 from mfuterko/implement-chown-functionality
Implement chown functionality
2020-12-07 20:50:45 +01:00
Mike Futerko cdee88cb2d Add go1.15 to the build matrix 2020-12-07 19:36:56 +02:00
Mike Futerko 5207eb96b7 Fix chown pull request errors 2020-12-07 17:24:35 +02:00
Mike Futerko cb1d580bf4 Fix Chown() pull request errors 2020-12-07 15:19:45 +02:00
Michail Kargakis 736b98e92c
Merge pull request #273 from rzajac/mem-file-truncate-use-lock
mem.File.Truncate should use lock
2020-11-15 14:23:41 +01:00
Rafal Zajac d175ebcca8 mem.File.Truncate should use lock
Fixes: #272
2020-11-13 22:59:22 +01:00
Michail Kargakis 6c0630556d
Merge pull request #271 from sreekanth370/master
Added power support for the travis.yml file with ppc64le.
2020-11-07 18:17:24 +01:00
sreekanth370 5dc7152bc6
test 2020-10-20 14:58:32 +05:30
Michail Kargakis efa411e911
Merge pull request #268 from hasheddan/tarfs-panic
Do not panic on passing empty tar reader to tarfs
2020-10-06 10:07:03 +02:00
hasheddan 9fa7c3b3c6
Do not panic on passing empty tar reader to tarfs
Corrects a bug where passing an empty tar reader to tarfs would cause
adding the pseudoroot to panic with assignment to entry in nil map.

Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
2020-10-05 17:23:18 -05:00
L. Alberto Giménez a4ea980f2d
Add tarfs implementation (reopen #265) (#266)
* Initial commit for tarfs

* tarfs: reword "open" status field

* tarfs: use TestMain for FS setup

We want to have the FS variable available through all the tests, so we
we use a common "setup" function to initialise it.

* tarfs: test: early exit for nonexisting files

* tarfs: create test for filesystem Open

* tarfs: implement File.Stat

* tarfs: implement Fs.Open

* tarfs: return error on non-supported methods

As tarfs is a read-only filesystem backend, we return EROFS (Read-only
file system) from any method that makes modifications.

* tarfs: implement File.data as bytes.Reader

Most of the operations that we want to implement for tarfs.File are
already defined in bytes.Reader.

We could use a plain slice and implement all the seeking manually, but I
think using this is more convenient.

* tarfs: short format for simple methods

* tarfs: add missing closing brace in tests

* tarfs: add test for File.ReadAt

* tarfs: test File.ReadAt

* tarfs: add tests for File.Read

* tarfs: implement File.Read

* tarfs: add tests for File.Seek

* tarfs: implement File.Seek

* tarfs: add tests for File.Name

* tarfs: implement File.Name

* tarfs: add tests for File.Close

* tarfs: implement File.Close

* tarfs: add tests for OpenFile

* tarfs: fix test for Fs.OpenFile

If the call fails, we don't have to close the file

* tarfs: remove code not needed after using filepath.Clean

* tarfs: Open: return a copy of the internal structure

As we modify the struct fields when closing, we don't want to lose the
internal representation of the file, in case we want to reopen it.

Return a copy of the File, although we keep using the same pointers to
tar.Header and buffer.Reader. Maybe we will need to change that in the
future.

* tarfs: implement Fs.OpenFile

* tarfs: use Fatalf for unexpected error in TestFsOpen

* tarfs: add tests for Fs.Stat

* tarfs: implement Fs.Stat

* tarfs: remove TestNewFs

That test depends too much on the internal imlementation, and it is
easier to break if we change it.

* tarfs: remove unused code

* tarfs: change internal implementation

To be able to handle directories (File.Readdir, File.Readdirnames), the
naive single-map implementation makes it a bit harder to implement.

Inspired by the zipfs backend, switch to an internal implementation of a
map of directories that contains a map of files, so the directory
methods are easier to implement.

Also, treat the "virtual" filesystem as absolute, just like zipfs does.

* tarfs: use Fatal errors to avoid panics

* tarfs: add pseudoroot

* tarfs: add tests for File.Readdir

* tarfs: add pointer Fs in the File structure

For directory-related operations we will need to access the internal
structure in the Fs.

As Readdir and Readdirnames are File methods, we need to access such
structure from the File.

* tarfs: fix error

* tarfs: use just the names for TestReaddir, easier than using fill os.FileInfo entries

* tarfs: create a copy of the original entry when opening a file

We added the fs field in the File struct to reference the underlying Fs
object, but in the Open cal we were not passing it, making all the
opened files to have a nil pointer in that field.

Change to make a copy of the original file, and returning that

* tarfs: implement File.Readdir

* tarfs: add tests for File.Readdirnames

* tarfs: implement Readdirnames

* tarfs: add test for File.Name

* tarfs: change tests to use the Afero interface instead

* tarfs: add tests for Glob from zipfs

* tarfs: update main repo references to tarfs

* tarfs: use OS-specific file separator for pseudoroot

* tarfs: fix path handling in Windows systems
2020-09-14 20:44:56 +02:00
Michail Kargakis 277f220898
Merge pull request #264 from LandonTClipp/memmapfs_lstatifpossible
Make MemMapFs implement Lstater
2020-09-14 20:42:22 +02:00
LandonTClipp bf960e8dcb Removing use of testify 2020-09-13 14:01:19 -05:00
LandonTClipp cb8b6bc2ff Make MemMapFs implement Lstater 2020-09-07 00:05:27 -05:00
Michail Kargakis 27c9ee0498
Merge pull request #260 from std0/union-readdir-fix
Fix amount of files read in UnionFile.Readdir
2020-09-02 22:07:05 +02:00
Michail Kargakis 215b26b36a
Merge pull request #263 from yerden/fix-mmap-fs-seeker
mem: fix incorrect offset in Write
2020-09-02 22:00:11 +02:00
Yerden Zhumabekov 84f9c83b7e mem: fix Write setting wrong offset
Signed-off-by: Yerden Zhumabekov <yerden.zhumabekov@gmail.com>
2020-09-02 11:39:23 +06:00
Yerden Zhumabekov d2521ec52d mem: fix code style in Seek
Signed-off-by: Yerden Zhumabekov <yerden.zhumabekov@gmail.com>
2020-09-02 11:39:23 +06:00
Yerden Zhumabekov 9e9dd69b09 mem: add test for Write/Seek
Signed-off-by: Yerden Zhumabekov <yerden.zhumabekov@gmail.com>
2020-09-02 11:39:23 +06:00
std0 066c8b09c5 Fix amount of files read in UnionFile.Readdir
To match behavior of os.File.Readdir, argument c in UnionFile.Readdir
should control amount of files read, not the upper bound of buffered
files.

Fixes #259
2020-08-23 02:36:25 +03:00
Michail Kargakis 64002605e4
Merge pull request #257 from cksriprajittichai/fix-typos-in-readme-code
Fix typos in README code samples
2020-08-12 22:36:11 +02:00
Chase Sriprajittichai 0caf874268 Fix typos in README code samples 2020-08-11 18:39:03 -07:00
Michail Kargakis 799ff74c0f
Merge pull request #254 from tklauser/bump-x-text
go.mod: bump golang.org/x/text to v0.3.3
2020-08-09 22:32:41 +02:00
Tobias Klauser 7686d4fd58 go.mod: bump golang.org/x/text to v0.3.3
golang.org/x/text v0.3.0 has a known vulnerability [1], [2], [3] which
is resolved in v0.3.3

[1] https://nvd.nist.gov/vuln/detail/CVE-2020-14040
[2] https://groups.google.com/forum/#!topic/golang-announce/bXVeAmGOqz0
[3] https://github.com/golang/go/issues/39491
2020-08-06 16:04:07 +02:00
Michail Kargakis 238028b461
Merge pull request #250 from anishathalye/improve-mode
Improve handling of mode bits
2020-08-02 15:26:12 +02:00
Anish Athalye b598fbbf55 Improve handling of mode bits
- "/" should have mode `os.ModeDir|0755`, not `0000`. Among other
  things, this had resulted in `mode.IsDir()` returning false for root
  prior to this patch.
- `Mkdir`, `MkdirAll`, and `OpenFile` shouldn't be allowed to set
  permissions that are otherwise illegal through `Chmod`. This mirrors
  what Go's `os` package does: it calls `syscallMode(mode)`, which
  effectively clears out the same bits that are disallowed by `Chmod`.
- `MkdirAll` should use the given permissions for all intermediate
  directories that are created, not just for the final directory. Prior
  to this patch, intermediate directories were created with mode bits
  `0000`. Besides the permission bits being wrong, `mode.IsDir()` would
  return false for these directories prior to this patch.
2020-07-15 15:43:42 -04:00
Michail Kargakis 5d9e780691
Merge pull request #249 from JohnStarich/bugfix/memfs-chmod-perm-only
Prevent MemMapFs.Chmod from changing all mode bits
2020-07-13 19:39:06 +02:00
John Starich d443df9ff3 Fix test for Windows 2020-07-11 20:57:46 -05:00
John Starich 57ab25aed5 Use more appropriate func name for unrestricted mode changes 2020-07-11 20:42:35 -05:00
John Starich 7b70cb1dbc Fix MemMapFs.Chmod changing non-perm bits 2020-07-11 18:32:33 -05:00
Michail Kargakis 64b7ddd95f
Merge pull request #247 from JohnStarich/bugfix/zipfs-read-small-file
Fix panic when not filling up zipfs's read buffer
2020-06-28 19:14:13 +02:00
John Starich e76d1f3bb5 Fix panic when not filling up zipfs's read buffer 2020-06-25 23:27:17 -05:00