Compare commits

...

21 Commits

Author SHA1 Message Date
h4sh5 1af815b7dc remove README.rst file since README.md file exists 2024-02-27 14:04:42 +10:00
Thomas Merz 683b8c2454 👷 use latest major version for actions/checkout (#289) 2023-12-22 18:05:32 +00:00
Haoxi Tan 70c8fd9031 Merge pull request #277 from g-easy/master
Implement connect to vsock.
2023-11-06 19:55:14 +10:00
h4sh 47a580dd77 test improvements 2023-09-23 23:05:21 +10:00
h4sh e5f4fcaad7 github action build+test 2023-09-23 23:00:12 +10:00
Haoxi Tan 8c97da5b16 Update README.md 2023-09-23 22:52:14 +10:00
Haoxi Tan 551752c3a5 No longer orphaned 2023-09-23 22:51:16 +10:00
Nikolaus Rath c91eb9a9a9 Released 3.7.3 2022-05-26 14:23:35 +01:00
Peter Wienemann 103c6ba68b Fix typo: occured -> occurred (#280) 2022-03-07 19:41:43 +00:00
easy 3aa3efcb52 Implement connect to vsock.
"sshfs -o vsock=CID:PORT" will cause sshfs to connect directly to the
given vsock, bypassing ssh, and allowing high performance sshfs mounts
of a VM guest.
2022-02-15 14:45:26 +11:00
Matthew Berginski a2054a2b73 Typo Fix (#274) 2021-12-27 11:27:40 +00:00
Kim Brose fd02499b4c add missing backtick (#270) 2021-10-30 13:04:15 +01:00
Antonio Rojas c2715f7453 Fix typo in ssh_opts (#269)
Add a missing comma that prevents using the PubkeyAcceptedKeyTypes option
2021-10-06 08:19:22 +01:00
Nikolaus Rath 1abde6e779 Clarify the need for libfuse3. 2021-09-24 13:36:56 +01:00
Cam Cope a181b9b60b add .git-blame-ignore-revs (#261) 2021-08-30 18:08:07 +01:00
Cam Cope d54c7ecbd6 Fixup whitespace and configure CI to keep it that way 2021-08-30 15:35:33 +01:00
a1346054 803e0e65cf Fix script issues identified through shellcheck (#258)
* Fix spelling

* Fix shellcheck-identified warnings in shell scripts
2021-08-25 14:45:42 +02:00
Nikolaus Rath 9700b35370 Released 3.7.2 2021-06-08 09:52:08 +01:00
Andrew Stone 6c1b92df81 Fix deadlock in conn cleanup (#244)
Calling through to request_free() from clean_req() causes deadlock since
sshfs.lock is already held by the caller of clean_req().
2021-02-25 12:13:30 +00:00
Nikolaus Rath d18869a307 Update to newer CI environment.
The current one causes build failures since recent pytest versions are incompatble with
Python 3.5.
2021-01-19 10:17:10 +00:00
Peter Belm dfd4cba385 Workaround for mkdir on existing directory (#242)
Added a secondary check so if a mkdir request fails with EPERM an access request will be tried - returning EEXIST if the access was successful. This matches the correct behaviour expected by applications such as git.

Co-authored-by: Peter Belm <peter.belm@dataalchemist.co.uk>
2021-01-19 10:13:09 +00:00
23 changed files with 577 additions and 287 deletions
+1
View File
@@ -0,0 +1 @@
d54c7ecbd618afb4df524e0d96dec7fe7cc2935d
+1 -1
View File
@@ -11,7 +11,7 @@ PLEASE READ BEFORE REPORTING AN ISSUE
SSHFS does not have any active, regular contributors or developers. The current maintainer continues to apply pull requests and tries to make regular releases, but unfortunately has no capacity to do any development beyond addressing high-impact issues. When reporting bugs, please understand that unless you are including a pull request or are reporting a critical issue, you will probably not get a response.
To prevent the issue tracker from being flooded with issues that no-one is intending to work on, and to give more visibilty to critical issues that users should be aware of and that most urgently need attention, I will also close most bug reports once they've been inactive for a while.
To prevent the issue tracker from being flooded with issues that no-one is intending to work on, and to give more visibility to critical issues that users should be aware of and that most urgently need attention, I will also close most bug reports once they've been inactive for a while.
Please note that this isn't meant to imply that you haven't found a bug - you most likely have and I'm grateful that you took the time to report it. Unfortunately, SSHFS is a purely volunteer driven project,
and at the moment there simply aren't any volunteers.
+51
View File
@@ -0,0 +1,51 @@
name: build and test
on:
push:
pull_request:
workflow_dispatch: # this is a nice option that will enable a button w/ inputs
inputs:
git-ref:
description: Git Ref (Optional)
required: false
jobs:
build-and-test:
name: Build and test
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- uses: actions/setup-python@v4
- name: Install build dependencies
run: |
sudo apt-get update
sudo apt-get install valgrind gcc ninja-build meson libglib2.0-dev libfuse3-dev
- name: Install meson
run: pip3 install meson pytest
- name: build
run: |
mkdir build; cd build
meson ..
ninja
# cd does not persist across steps
- name: upload build artifact
uses: actions/upload-artifact@v3
with:
name: sshfs
path: build/sshfs
- name: make ssh into localhost without prompt possible for tests
run: |
ssh-keygen -b 2048 -t rsa -f ~/.ssh/id_rsa -q -N ""
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
- name: run tests
run: |
cd build
python3 -m pytest test/
+14
View File
@@ -0,0 +1,14 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.0.1
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/jumanjihouse/pre-commit-hooks
rev: 2.1.5
hooks:
- id: shellcheck
+20 -10
View File
@@ -1,21 +1,31 @@
sudo: required
dist: xenial
dist: focal
language: c
cache:
- pip
language:
- c
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- shellcheck
- valgrind
- clang
- gcc
- gcc-6
- clang
- python-docutils
- python3-pip
- python3-setuptools
- ninja-build
install: test/travis-install.sh
script: test/travis-build.sh
- meson
- python3-pytest
- libglib2.0-dev
install: test/travis-install.sh
jobs:
include:
- name: Lint
script: ./test/lint.sh
install: skip
- name: Build + Test
script: test/travis-build.sh
+10 -2
View File
@@ -1,27 +1,31 @@
Current Maintainer
------------------
Nikolaus Rath <Nikolaus@rath.org>
None.
Past Maintainers
----------------
Nikolaus Rath <Nikolaus@rath.org> (until 05/2022)
Miklos Szeredi <miklos@szeredi.hu> (until 12/2015)
Contributors (autogenerated list)
---------------------------------
a1346054 <36859588+a1346054@users.noreply.github.com>
Alan Jenkins <alan.christopher.jenkins@gmail.com>
Alexander Neumann <alexander@bumpern.de>
Anatol Pomozov <anatol.pomozov@gmail.com>
Andrew Stone <a@stne.dev>
Antonio Rojas <arojas@archlinux.org>
Benjamin Fleischer <fleiben@gmail.com>
Berserker <berserker.troll@yandex.com>
Bill Zissimopoulos <billziss@navimatics.com>
bjoe2k4 <bjoe2k4@users.noreply.github.com>
Brandon Carter <b-carter@users.noreply.github.com>
Cam Cope <github@camcope.me>
Chris Wolfe <cwolfe@chromium.org>
Clayton G. Hobbs <clay@lakeserv.net>
Daniel Lublin <daniel@lublin.se>
@@ -41,6 +45,8 @@ Julio Merino <jmmv@meroh.net>
Junichi Uekawa <dancerj@gmail.com>
Junichi Uekawa <dancer@netfort.gr.jp>
kalvdans <github@kalvdans.no-ip.org>
Kim Brose <kim.brose@rwth-aachen.de>
Matthew Berginski <matthew.berginski@gmail.com>
Michael Forney <mforney@mforney.org>
Mike Kelly <mike@pair.com>
Mike Salvatore <mike.s.salvatore@gmail.com>
@@ -49,6 +55,8 @@ Miklos Szeredi <mszeredi@suse.cz>
mssalvatore <mike.s.salvatore@gmail.com>
Nikolaus Rath <Nikolaus@rath.org>
Percy Jahn <email@percyjahn.de>
Peter Belm <peterbelm@gmail.com>
Peter Wienemann <peter.wienemann@uni-bonn.de>
Qais Patankar <qaisjp@gmail.com>
Quentin Rameau <quinq@fifth.space>
Reid Wagner <wagnerreid@gmail.com>
+30 -7
View File
@@ -1,3 +1,26 @@
Release 3.7.3 (2022-05-26)
--------------------------
* Minor bugfixes.
* This is the last release from the current maintainer. SSHFS is now no longer maintained
or developed. Github issue tracking and pull requests have therefore been disabled. The
mailing list (see below) is still available for use.
If you would like to take over this project, you are welcome to do so. Please fork it
and develop the fork for a while. Once there has been 6 months of reasonable activity,
please contact Nikolaus@rath.org and I'll be happy to give you ownership of this
repository or replace with a pointer to the fork.
Release 3.7.2 (2021-06-08)
--------------------------
* Added a secondary check so if a mkdir request fails with EPERM an access request will be
tried - returning EEXIST if the access was successful.
Fixes: https://github.com/libfuse/sshfs/issues/243
Release 3.7.1 (2020-11-09)
--------------------------
@@ -17,7 +40,7 @@ Release 3.7.0 (2020-01-03)
needed anymore. If you depend on this workaround, please let the SSHFS maintainers know,
otherwise support for the workaround will be removed completely in a future version.
Release 3.6.0 (2019-11-03)
--------------------------
@@ -100,13 +123,13 @@ Release 3.1.0 (2017-08-04)
* For improved backwards compatibility, SSHFS now also silently
accepts the old ``-o cache_*`` options.
Release 3.0.0 (2017-07-08)
--------------------------
* sshfs now requires libfuse 3.1.0 or newer.
* When supported by the kernel, sshfs now uses writeback caching.
* The `cache` option has been renamed to `dir_cache` for clarity.
* The `cache` option has been renamed to `dir_cache` for clarity.
* Added unit tests
* --debug now behaves like -o debug_sshfs, i.e. it enables sshfs
debugging messages rather than libfuse debugging messages.
@@ -121,7 +144,7 @@ Release 3.0.0 (2017-07-08)
* Removed support for `-o workaround=all`. Workarounds should always
enabled explicitly and only when needed. There is no point in always
enabling a potentially changing set of workarounds.
Release 2.9 (2017-04-17)
------------------------
@@ -160,14 +183,14 @@ Release 2.4 (2012-03-08)
------------------------
* New `slave` option.
* New `idmap`, `uidmap` and `gidmap` options.
* New `idmap`, `uidmap` and `gidmap` options.
* Various small bugfixes.
Release 2.3 (2011-07-01)
------------------------
* Support hard link creation if server is OpenSSH 5.7 or later
* Small improvements and bug fixes
* Small improvements and bug fixes
* Check mount point and options before connecting to ssh server
* New 'delay_connect' option
@@ -180,7 +203,7 @@ Release 2.2 (2008-10-20)
Release 2.1 (2008-07-11)
------------------------
* Small improvements and bug fixes
* Small improvements and bug fixes
Release 2.0 (2008-04-23)
------------------------
+55 -46
View File
@@ -1,100 +1,109 @@
SSHFS
=====
# SSHFS
About
-----
## About
SSHFS allows you to mount a remote filesystem using SFTP. Most SSH
servers support and enable this SFTP access by default, so SSHFS is
very simple to use - there's nothing to do on the server-side.
Development Status
------------------
## Development Status
SSHFS is shipped by all major Linux distributions and has been in
production use across a wide range of systems for many years. However,
at present SSHFS does not have any active, regular contributors, and
there are a number of known issues (see the bugtracker). The current
maintainer continues to apply pull requests and makes regular
there are a number of known issues (see the [bugtracker](https://github.com/libfuse/sshfs/issues)).
The current maintainer continues to apply pull requests and makes regular
releases, but unfortunately has no capacity to do any development
beyond addressing high-impact issues. When reporting bugs, please
understand that unless you are including a pull request or are
reporting a critical issue, you will probably not get a response.
How to use
----------
## How to use
Once sshfs is installed (see next section) running it is very simple::
sshfs [user@]hostname:[directory] mountpoint
Once sshfs is installed (see next section) running it is very simple:
```
sshfs [user@]hostname:[directory] mountpoint
```
It is recommended to run SSHFS as regular user (not as root). For
this to work the mountpoint must be owned by the user. If username is
omitted SSHFS will use the local username. If the directory is
omitted, SSHFS will mount the (remote) home directory. If you need to
enter a password sshfs will ask for it (actually it just runs ssh
which ask for the password if needed).
which asks for the password if needed).
Also many ssh options can be specified (see the manual pages for
*sftp(1)* and *ssh_config(5)*), including the remote port number
(``-oport=PORT``)
(`-oport=PORT`)
To unmount the filesystem::
To unmount the filesystem:
fusermount -u mountpoint
```
fusermount -u mountpoint
```
On BSD and macOS, to unmount the filesystem::
On BSD and macOS, to unmount the filesystem:
umount mountpoint
```
umount mountpoint
```
## Installation
Installation
------------
First, download the latest SSHFS release from
https://github.com/libfuse/sshfs/releases. On Linux and BSD, you will
also need to install libfuse_ 3.1.0 or newer. On macOS, you need
OSXFUSE_ instead. Finally, you need the Glib_ library with development
headers (which should be available from your operating system's
package manager).
https://github.com/libfuse/sshfs/releases. You also need [libfuse](http://github.com/libfuse/libfuse) 3.1.0 or newer (or a
similar library that provides a libfuse3 compatible interface for your operating
system). Finally, you need the [Glib](https://developer.gnome.org/glib/stable/) library with development headers (which should be
available from your operating system's package manager).
To build and install, we recommend to use Meson_ (version 0.38 or
newer) and Ninja_. After extracting the sshfs tarball, create a
(temporary) build directory and run Meson::
To build and install, we recommend to use [Meson](http://mesonbuild.com/) (version 0.38 or
newer) and [Ninja](https://ninja-build.org/). After extracting the sshfs tarball, create a
(temporary) build directory and run Meson:
$ mkdir build; cd build
$ meson ..
```
$ mkdir build; cd build
$ meson ..
```
Normally, the default build options will work fine. If you
nevertheless want to adjust them, you can do so with the *mesonconf*
command::
command:
$ mesonconf # list options
$ mesonconf -D strip=true # set an option
```
$ mesonconf # list options
$ mesonconf -D strip=true # set an option
```
To build, test and install SSHFS, you then use Ninja (running the
tests requires the `py.test`_ Python module)::
tests requires the [py.test](http://www.pytest.org/) Python module):
$ ninja
$ python3 -m pytest test/ # optional, but recommended
$ sudo ninja install
```
$ ninja
$ python3 -m pytest test/ # optional, but recommended
$ sudo ninja install
```
.. _libfuse: http://github.com/libfuse/libfuse
.. _OSXFUSE: https://osxfuse.github.io/
.. _Glib: https://developer.gnome.org/glib/stable/
.. _Meson: http://mesonbuild.com/
.. _Ninja: https://ninja-build.org/
.. _`py.test`: http://www.pytest.org/
## Getting Help
Getting Help
------------
If you need help, please ask on the <fuse-sshfs@lists.sourceforge.net>
mailing list (subscribe at
https://lists.sourceforge.net/lists/listinfo/fuse-sshfs).
Please report any bugs on the GitHub issue tracker at
https://github.com/libfuse/libfuse/issues.
https://github.com/libfuse/sshfs/issues.
## Packaging Status
<a href="https://repology.org/project/fusefs:sshfs/versions">
<img src="https://repology.org/badge/vertical-allrepos/fusefs:sshfs.svg" alt="Packaging status" >
</a>
+6 -6
View File
@@ -256,7 +256,7 @@ static void *cache_init(struct fuse_conn_info *conn,
{
void *res;
res = cache.next_oper->init(conn, cfg);
// Cache requires a path for each request
cfg->nullpath_ok = 0;
@@ -318,9 +318,9 @@ static int cache_releasedir(const char *path, struct fuse_file_info *fi)
{
int err;
struct file_handle *cfi;
cfi = (struct file_handle*) fi->fh;
if(cfi->is_open) {
fi->fh = cfi->fs_fh;
err = cache.next_oper->releasedir(path, fi);
@@ -365,7 +365,7 @@ static int cache_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
struct node *node;
assert(offset == 0);
pthread_mutex_lock(&cache.lock);
node = cache_lookup(path);
if (node != NULL && node->dir != NULL) {
@@ -391,8 +391,8 @@ static int cache_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
}
cfi->is_open = 1;
cfi->fs_fh = fi->fh;
}
}
ch.path = path;
ch.buf = buf;
ch.filler = filler;
+1 -1
View File
@@ -18,7 +18,7 @@ extern "C" {
/**
* Option description
*
* This structure describes a single option, and and action associated
* This structure describes a single option, and an action associated
* with it, in case it matches.
*
* More than one such match may occur, in which case the action for
-1
View File
@@ -27,4 +27,3 @@ gpg --armor --detach-sign "${TAG}.tar.xz"
PREV_TAG="$(git tag --list 'sshfs-3*' --sort=-taggerdate --merged "${TAG}^"| head -1)"
echo "Contributors from ${PREV_TAG} to ${TAG}:"
git log --pretty="format:%an <%aE>" "${PREV_TAG}..${TAG}" | sort -u
+2 -3
View File
@@ -1,4 +1,4 @@
project('sshfs', 'c', version: '3.7.1',
project('sshfs', 'c', version: '3.7.3',
meson_version: '>= 0.40',
default_options: [ 'buildtype=debugoptimized' ])
@@ -26,7 +26,7 @@ endif
rst2man = find_program('rst2man', 'rst2man.py', required: false)
cfg = configuration_data()
cfg.set_quoted('PACKAGE_VERSION', meson.project_version())
@@ -69,4 +69,3 @@ meson.add_install_script('utils/install_helper.sh',
subdir('test')
+98 -35
View File
@@ -50,6 +50,9 @@
# include <libgen.h>
# include <darwin_compat.h>
#endif
#ifdef __linux__
# include <linux/vm_sockets.h>
#endif
#include "cache.h"
@@ -141,7 +144,7 @@
/*
Handling of multiple SFTP connections
--------------------------------------
An SFTP server is free to return responses to outstanding requests in arbitrary
order. However, execution of requests may only be re-ordered and parallelized as long
as "the results in the responses will be the same as if [the client] had sent the
@@ -158,11 +161,11 @@
involved) wait for the other requests to complete. This means that e.g. a readdir
request would have to block on most other activity in the same directory, eliminating a
major advantage of using multiple connections.
In practice, we can luckily take advantage of the knowledge that most FUSE requests are
the result of (synchronous) syscalls from userspace that will block until the
corresponding FUSE response has been sent.
If -o sshfs_sync is used, SSHFS always waits for the SFTP server response before
returning a FUSE response to userspace. If userspace makes concurrent system calls,
there is no ordering guarantee in the first place, so we do not have to worry about
@@ -349,6 +352,7 @@ struct sshfs {
pthread_mutex_t lock;
unsigned int randseed;
int max_conns;
char *vsock;
struct conn *conns;
int ptyfd;
int ptypassivefd;
@@ -432,7 +436,7 @@ static const char *ssh_opts[] = {
"ProxyCommand",
"ProxyJump",
"ProxyUseFdpass",
"PubkeyAcceptedKeyTypes"
"PubkeyAcceptedKeyTypes",
"PubkeyAuthentication",
"RekeyLimit",
"RevokedHostKeys",
@@ -504,6 +508,7 @@ static struct fuse_opt sshfs_opts[] = {
SSHFS_OPT("dir_cache=no", dir_cache, 0),
SSHFS_OPT("direct_io", direct_io, 1),
SSHFS_OPT("max_conns=%u", max_conns, 1),
SSHFS_OPT("vsock=%s", vsock, 0),
SSHFS_OPT("-h", show_help, 1),
SSHFS_OPT("--help", show_help, 1),
@@ -523,7 +528,7 @@ static struct fuse_opt sshfs_opts[] = {
/* For backwards compatibility */
SSHFS_OPT("cache=yes", dir_cache, 1),
SSHFS_OPT("cache=no", dir_cache, 0),
FUSE_OPT_KEY("writeback_cache=no", FUSE_OPT_KEY_DISCARD),
FUSE_OPT_KEY("unreliable_append", FUSE_OPT_KEY_DISCARD),
@@ -1073,7 +1078,7 @@ static struct conn* get_conn(const struct sshfs_file *sf,
if (sshfs.max_conns == 1)
return &sshfs.conns[0];
if (sf != NULL)
return sf->conn;
@@ -1281,6 +1286,60 @@ static int connect_to(struct conn *conn, char *host, char *port)
return 0;
}
static int connect_vsock(struct conn *conn, char *vsock)
{
#ifndef __linux__
fprintf(stderr, "vsock is not available\n");
return -1;
#else
int err;
int sock;
struct sockaddr_vm addr;
unsigned int cid;
unsigned int port;
char *delim;
delim = strchr(vsock, ':');
if (delim == NULL) {
fprintf(stderr, "invalid vsock, expecting CID:PORT\n");
return -1;
}
*delim = '\0';
errno = 0;
cid = strtoul(vsock, NULL, 10);
if (errno) {
perror("invalid cid");
return -1;
}
errno = 0;
port = strtoul(delim + 1, NULL, 10);
if (errno) {
perror("invalid port");
return -1;
}
sock = socket(AF_VSOCK, SOCK_STREAM, 0);
if (sock == -1) {
perror("failed to create socket");
return -1;
}
memset(&addr, 0, sizeof(addr));
addr.svm_family = AF_VSOCK;
addr.svm_cid = cid;
addr.svm_port = port;
err = connect(sock, (const struct sockaddr *)&addr, sizeof(addr));
if (err == -1) {
perror("failed to connect vsock");
close(sock);
return -1;
}
conn->rfd = sock;
conn->wfd = sock;
return 0;
#endif
}
static int do_write(struct conn *conn, struct iovec *iov, size_t count)
{
int res;
@@ -1401,9 +1460,11 @@ static int sftp_read(struct conn *conn, uint8_t *type, struct buffer *buf)
static void request_free(struct request *req)
{
pthread_mutex_lock(&sshfs.lock);
if (req->end_func)
req->end_func(req);
req->conn->req_count--;
pthread_mutex_unlock(&sshfs.lock);
buf_free(&req->reply);
sem_destroy(&req->ready);
g_free(req);
@@ -1449,11 +1510,9 @@ static int clean_req(void *key, struct request *req, gpointer user_data)
req->error = -EIO;
if (req->want_reply)
sem_post(&req->ready);
else {
if (req->end_func)
req->end_func(req);
else
request_free(req);
}
return TRUE;
}
@@ -1515,12 +1574,9 @@ static int process_one_request(struct conn *conn)
if (req->want_reply)
sem_post(&req->ready);
else {
if (req->end_func) {
pthread_mutex_lock(&sshfs.lock);
req->end_func(req);
pthread_mutex_unlock(&sshfs.lock);
}
pthread_mutex_lock(&sshfs.lock);
request_free(req);
pthread_mutex_unlock(&sshfs.lock);
}
} else
buf_free(&buf);
@@ -1836,6 +1892,8 @@ static int connect_remote(struct conn *conn)
err = connect_passive(conn);
else if (sshfs.directport)
err = connect_to(conn, sshfs.host, sshfs.directport);
else if (sshfs.vsock)
err = connect_vsock(conn, sshfs.vsock);
else
err = start_ssh(conn);
if (!err)
@@ -1909,7 +1967,7 @@ static void *sshfs_init(struct fuse_conn_info *conn,
// SFTP only supports 1-second time resolution
conn->time_gran = 1000000000;
return NULL;
}
@@ -1970,12 +2028,9 @@ static int sftp_request_wait(struct request *req, uint8_t type,
}
out:
if (req->end_func) {
pthread_mutex_lock(&sshfs.lock);
req->end_func(req);
pthread_mutex_unlock(&sshfs.lock);
}
pthread_mutex_lock(&sshfs.lock);
request_free(req);
pthread_mutex_unlock(&sshfs.lock);
return err;
}
@@ -2377,6 +2432,13 @@ static int sshfs_mkdir(const char *path, mode_t mode)
// Commutes with pending write(), so we can use any connection
err = sftp_request(get_conn(NULL, NULL), SSH_FXP_MKDIR, &buf, SSH_FXP_STATUS, NULL);
buf_free(&buf);
if (err == -EPERM) {
if (sshfs.op->access(path, R_OK) == 0) {
return -EEXIST;
}
}
return err;
}
@@ -2393,7 +2455,7 @@ static int sshfs_mknod(const char *path, mode_t mode, dev_t rdev)
// Commutes with pending write(), so we can use any connection
conn = get_conn(NULL, NULL);
buf_init(&buf, 0);
buf_add_path(&buf, path);
buf_add_uint32(&buf, SSH_FXF_WRITE | SSH_FXF_CREAT | SSH_FXF_EXCL);
@@ -2531,7 +2593,7 @@ static int sshfs_rename(const char *from, const char *to, unsigned int flags)
}
pthread_mutex_unlock(&sshfs.lock);
}
return err;
}
@@ -2588,12 +2650,12 @@ static int sshfs_chmod(const char *path, mode_t mode,
buf_init(&buf, 0);
if (sf == NULL)
buf_add_path(&buf, path);
else
else
buf_add_buf(&buf, &sf->handle);
buf_add_uint32(&buf, SSH_FILEXFER_ATTR_PERMISSIONS);
buf_add_uint32(&buf, mode);
/* FIXME: really needs LSETSTAT extension (debian Bug#640038) */
// Commutes with pending write(), so we can use any connection
// if the file is not open.
@@ -2617,7 +2679,7 @@ static int sshfs_chown(const char *path, uid_t uid, gid_t gid,
if (!sshfs_file_is_conn(sf))
return -EIO;
}
if (sshfs.remote_uid_detected) {
if (uid == sshfs.local_uid)
uid = sshfs.remote_uid;
@@ -2634,7 +2696,7 @@ static int sshfs_chown(const char *path, uid_t uid, gid_t gid,
buf_init(&buf, 0);
if (sf == NULL)
buf_add_path(&buf, path);
else
else
buf_add_buf(&buf, &sf->handle);
buf_add_uint32(&buf, SSH_FILEXFER_ATTR_UIDGID);
buf_add_uint32(&buf, uid);
@@ -2684,7 +2746,7 @@ static int sshfs_utimens(const char *path, const struct timespec tv[2],
buf_init(&buf, 0);
if (sf == NULL)
buf_add_path(&buf, path);
else
else
buf_add_buf(&buf, &sf->handle);
buf_add_uint32(&buf, SSH_FILEXFER_ATTR_ACMODTIME);
buf_add_uint32(&buf, asec);
@@ -2739,7 +2801,7 @@ static int sshfs_open_common(const char *path, mode_t mode,
if (fi->flags & O_APPEND)
pflags |= SSH_FXF_APPEND;
sf = g_new0(struct sshfs_file, 1);
list_init(&sf->write_reqs);
pthread_cond_init(&sf->write_finished, NULL);
@@ -2749,7 +2811,7 @@ static int sshfs_open_common(const char *path, mode_t mode,
pthread_mutex_lock(&sshfs.lock);
sf->modifver= sshfs.modifver;
if (sshfs.max_conns > 1) {
ce = g_hash_table_lookup(sshfs.conntab, path);
ce = g_hash_table_lookup(sshfs.conntab, path);
if (!ce) {
ce = g_malloc(sizeof(struct conntab_entry));
ce->refcount = 0;
@@ -3353,7 +3415,7 @@ static int sshfs_truncate(const char *path, off_t size,
sshfs_inc_modifver();
if (sshfs.truncate_workaround)
return sshfs_truncate_workaround(path, size, fi);
buf_init(&buf, 0);
if (sf != NULL)
@@ -3396,7 +3458,7 @@ static int sshfs_getattr(const char *path, struct stat *stbuf,
buf_add_buf(&buf, &sf->handle);
err = sftp_request(sf->conn, SSH_FXP_FSTAT, &buf,
SSH_FXP_ATTRS, &outbuf);
}
}
if (!err) {
err = buf_get_attrs(&outbuf, stbuf, NULL);
#ifdef __APPLE__
@@ -3644,6 +3706,7 @@ static void usage(const char *progname)
" -o no_check_root don't check for existence of 'dir' on server\n"
" -o password_stdin read password from stdin (only for pam_mount!)\n"
" -o max_conns=N open parallel SSH connections\n"
" -o vsock=CID:PORT connect to the given vsock\n"
" -o SSHOPT=VAL ssh options (see man ssh_config)\n"
"\n"
"FUSE Options:\n",
+13 -10
View File
@@ -156,6 +156,9 @@ Options
-o directport=PORT
directly connect to PORT bypassing ssh
-o vsock=CID:PORT
directly connect using a vsock to CID:PORT bypassing ssh
-o passive
communicate over stdin and stdout bypassing network. Useful for
mounting local filesystem on the remote side. An example using
@@ -190,17 +193,17 @@ Options
directory cache holds the names of directory entries. Enabling it
allows `readdir(3)` system calls to be processed without network
access.
-o dcache_max_size=N
sets the maximum size of the directory cache.
-o dcache_timeout=N
sets timeout for directory cache in seconds.
-o dcache_{stat,link,dir}_timeout=N
sets separate timeout for {attributes, symlinks, names} in the
directory cache.
-o dcache_clean_interval=N
sets the interval for automatic cleaning of the directory cache.
@@ -209,17 +212,17 @@ Options
when full.
-o direct_io
This option disables the use of page cache (file content cache) in
This option disables the use of page cache (file content cache) in
the kernel for this filesystem.
This has several affects:
1. Each read() or write() system call will initiate one or more read or
write operations, data will not be cached in the kernel.
2. The return value of the read() and write() system calls will correspond
to the return values of the read and write operations. This is useful
2. The return value of the read() and write() system calls will correspond
to the return values of the read and write operations. This is useful
for example if the file size is not known in advance (before reading it).
e.g. /proc filesystem
e.g. /proc filesystem
-o max_conns=N
sets the maximum number of simultaneous SSH connections
@@ -274,7 +277,7 @@ SSHFS hangs for no apparent reason
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In some cases, attempts to access the SSHFS mountpoint may freeze if
no filesystem activity has occured for some time. This is typically
no filesystem activity has occurred for some time. This is typically
caused by the SSH connection being dropped because of inactivity
without SSHFS being informed about that. As a workaround, you can try
to mount with ``-o ServerAliveInterval=15``. This will force the SSH
@@ -311,7 +314,7 @@ interrupted.
Mounting from /etc/fstab
========================
To mount an SSHFS filesystem from ``/etc/fstab``, simply use ``sshfs`
To mount an SSHFS filesystem from ``/etc/fstab``, simply use ``sshfs``
as the file system type. (For backwards compatibility, you may also
use ``fuse.sshfs``).
+3 -2
View File
@@ -1,7 +1,8 @@
#!/bin/bash
set -e
machine=$(uname -m)
mkdir build-$machine
cd build-$machine
mkdir "build-$machine"
cd "build-$machine"
meson ..
ninja
+44 -21
View File
@@ -3,24 +3,28 @@ import pytest
import time
import re
# If a test fails, wait a moment before retrieving the captured
# stdout/stderr. When using a server process, this makes sure that we capture
# any potential output of the server that comes *after* a test has failed. For
# example, if a request handler raises an exception, the server first signals an
# error to FUSE (causing the test to fail), and then logs the exception. Without
# the extra delay, the exception will go into nowhere.
@pytest.mark.hookwrapper
# If a test fails, wait a moment before retrieving the captured stdout/stderr.
# When using a server process, this makes sure that we capture any potential
# output of the server that comes *after* a test has failed. For example, if a
# request handler raises an exception, the server first signals an error to
# FUSE (causing the test to fail), and then logs the exception. Without the
# extra delay, the exception will go into nowhere.
@pytest.hookimpl(hookwrapper=True)
def pytest_pyfunc_call(pyfuncitem):
outcome = yield
failed = outcome.excinfo is not None
if failed:
time.sleep(1)
@pytest.fixture()
def pass_capfd(request, capfd):
'''Provide capfd object to UnitTest instances'''
"""Provide capfd object to UnitTest instances"""
request.instance.capfd = capfd
def check_test_output(capfd):
(stdout, stderr) = capfd.readouterr()
@@ -31,39 +35,57 @@ def check_test_output(capfd):
# Strip out false positives
for (pattern, flags, count) in capfd.false_positives:
cp = re.compile(pattern, flags)
(stdout, cnt) = cp.subn('', stdout, count=count)
(stdout, cnt) = cp.subn("", stdout, count=count)
if count == 0 or count - cnt > 0:
stderr = cp.sub('', stderr, count=count - cnt)
stderr = cp.sub("", stderr, count=count - cnt)
patterns = [ r'\b{}\b'.format(x) for x in
('exception', 'error', 'warning', 'fatal', 'traceback',
'fault', 'crash(?:ed)?', 'abort(?:ed)',
'uninitiali[zs]ed') ]
patterns += ['^==[0-9]+== ']
patterns = [
r"\b{}\b".format(x)
for x in (
"exception",
"error",
"warning",
"fatal",
"traceback",
"fault",
"crash(?:ed)?",
"abort(?:ed)",
"uninitiali[zs]ed",
)
]
patterns += ["^==[0-9]+== "]
for pattern in patterns:
cp = re.compile(pattern, re.IGNORECASE | re.MULTILINE)
hit = cp.search(stderr)
if hit:
raise AssertionError('Suspicious output to stderr (matched "%s")' % hit.group(0))
raise AssertionError(
'Suspicious output to stderr (matched "%s")' % hit.group(0)
)
hit = cp.search(stdout)
if hit:
raise AssertionError('Suspicious output to stdout (matched "%s")' % hit.group(0))
raise AssertionError(
'Suspicious output to stdout (matched "%s")' % hit.group(0)
)
def register_output(self, pattern, count=1, flags=re.MULTILINE):
'''Register *pattern* as false positive for output checking
"""Register *pattern* as false positive for output checking
This prevents the test from failing because the output otherwise
appears suspicious.
'''
"""
self.false_positives.append((pattern, flags, count))
# This is a terrible hack that allows us to access the fixtures from the
# pytest_runtest_call hook. Among a lot of other hidden assumptions, it probably
# relies on tests running sequential (i.e., don't dare to use e.g. the xdist
# plugin)
current_capfd = None
@pytest.yield_fixture(autouse=True)
@pytest.fixture(autouse=True)
def save_cap_fixtures(request, capfd):
global current_capfd
capfd.false_positives = []
@@ -71,7 +93,7 @@ def save_cap_fixtures(request, capfd):
# Monkeypatch in a function to register false positives
type(capfd).register_output = register_output
if request.config.getoption('capture') == 'no':
if request.config.getoption("capture") == "no":
capfd = None
current_capfd = capfd
bak = current_capfd
@@ -82,6 +104,7 @@ def save_cap_fixtures(request, capfd):
assert bak is current_capfd
current_capfd = None
@pytest.hookimpl(trylast=True)
def pytest_runtest_call(item):
capfd = current_capfd
Executable
+4
View File
@@ -0,0 +1,4 @@
#!/bin/bash
set -e
pip3 install --user pre-commit
pre-commit run --all-files --show-diff-on-failure
+1 -1
View File
@@ -2,7 +2,7 @@ test_scripts = [ 'conftest.py', 'pytest.ini', 'test_sshfs.py',
'util.py' ]
custom_target('test_scripts', input: test_scripts,
output: test_scripts, build_by_default: true,
command: ['cp', '-fPp',
command: ['cp', '-fPp',
'@INPUT@', meson.current_build_dir() ])
# Provide something helpful when running 'ninja test'
+2
View File
@@ -1,2 +1,4 @@
[pytest]
addopts = --verbose --assert=rewrite --tb=native -x -r a
markers = uses_fuse: Mark to indicate that FUSE is available on the system running the test
+158 -100
View File
@@ -1,8 +1,9 @@
#!/usr/bin/env python3
if __name__ == '__main__':
if __name__ == "__main__":
import pytest
import sys
sys.exit(pytest.main([__file__] + sys.argv[1:]))
import subprocess
@@ -13,79 +14,123 @@ import stat
import shutil
import filecmp
import errno
from contextlib import contextmanager
from tempfile import NamedTemporaryFile
from util import (wait_for_mount, umount, cleanup, base_cmdline,
basename, fuse_test_marker, safe_sleep)
from util import (
wait_for_mount,
umount,
cleanup,
base_cmdline,
basename,
fuse_test_marker,
safe_sleep,
os_create,
os_open,
)
from os.path import join as pjoin
TEST_FILE = __file__
pytestmark = fuse_test_marker()
with open(TEST_FILE, 'rb') as fh:
with open(TEST_FILE, "rb") as fh:
TEST_DATA = fh.read()
def name_generator(__ctr=[0]):
__ctr[0] += 1
return 'testfile_%d' % __ctr[0]
@pytest.mark.parametrize("debug", (False, True))
@pytest.mark.parametrize("cache_timeout", (0,1))
@pytest.mark.parametrize("sync_rd", (True, False))
@pytest.mark.parametrize("multiconn", (True,False))
def test_sshfs(tmpdir, debug, cache_timeout, sync_rd, multiconn, capfd):
def name_generator(__ctr=[0]) -> str:
"""Generate a fresh filename on each call"""
__ctr[0] += 1
return f"testfile_{__ctr[0]}"
@pytest.mark.parametrize(
"debug",
[pytest.param(False, id="debug=false"), pytest.param(True, id="debug=true")],
)
@pytest.mark.parametrize(
"cache_timeout",
[pytest.param(0, id="cache_timeout=0"), pytest.param(1, id="cache_timeout=1")],
)
@pytest.mark.parametrize(
"sync_rd",
[pytest.param(True, id="sync_rd=true"), pytest.param(False, id="sync_rd=false")],
)
@pytest.mark.parametrize(
"multiconn",
[
pytest.param(True, id="multiconn=true"),
pytest.param(False, id="multiconn=false"),
],
)
def test_sshfs(
tmpdir, debug: bool, cache_timeout: int, sync_rd: bool, multiconn: bool, capfd
) -> None:
# Avoid false positives from debug messages
#if debug:
# if debug:
# capfd.register_output(r'^ unique: [0-9]+, error: -[0-9]+ .+$',
# count=0)
# Avoid false positives from storing key for localhost
# Avoid false positives from storing key for localhost
capfd.register_output(r"^Warning: Permanently added 'localhost' .+", count=0)
# Test if we can ssh into localhost without password
try:
res = subprocess.call(['ssh', '-o', 'KbdInteractiveAuthentication=no',
'-o', 'ChallengeResponseAuthentication=no',
'-o', 'PasswordAuthentication=no',
'localhost', '--', 'true'], stdin=subprocess.DEVNULL,
timeout=10)
res = subprocess.call(
[
"ssh",
"-o",
"StrictHostKeyChecking=no",
"-o",
"KbdInteractiveAuthentication=no",
"-o",
"ChallengeResponseAuthentication=no",
"-o",
"PasswordAuthentication=no",
"localhost",
"--",
"true",
],
stdin=subprocess.DEVNULL,
timeout=10,
)
except subprocess.TimeoutExpired:
res = 1
if res != 0:
pytest.fail('Unable to ssh into localhost without password prompt.')
pytest.fail("Unable to ssh into localhost without password prompt.")
mnt_dir = str(tmpdir.mkdir('mnt'))
src_dir = str(tmpdir.mkdir('src'))
mnt_dir = str(tmpdir.mkdir("mnt"))
src_dir = str(tmpdir.mkdir("src"))
cmdline = base_cmdline + [ pjoin(basename, 'sshfs'),
'-f', 'localhost:' + src_dir, mnt_dir ]
cmdline = base_cmdline + [
pjoin(basename, "sshfs"),
"-f",
f"localhost:{src_dir}",
mnt_dir,
]
if debug:
cmdline += [ '-o', 'sshfs_debug' ]
cmdline += ["-o", "sshfs_debug"]
if sync_rd:
cmdline += [ '-o', 'sync_readdir' ]
cmdline += ["-o", "sync_readdir"]
# SSHFS Cache
if cache_timeout == 0:
cmdline += [ '-o', 'dir_cache=no' ]
cmdline += ["-o", "dir_cache=no"]
else:
cmdline += [ '-o', 'dcache_timeout=%d' % cache_timeout,
'-o', 'dir_cache=yes' ]
cmdline += ["-o", f"dcache_timeout={cache_timeout}", "-o", "dir_cache=yes"]
# FUSE Cache
cmdline += [ '-o', 'entry_timeout=0',
'-o', 'attr_timeout=0' ]
cmdline += ["-o", "entry_timeout=0", "-o", "attr_timeout=0"]
if multiconn:
cmdline += [ '-o', 'max_conns=3' ]
new_env = dict(os.environ) # copy, don't modify
cmdline += ["-o", "max_conns=3"]
new_env = dict(os.environ) # copy, don't modify
# Abort on warnings from glib
new_env['G_DEBUG'] = 'fatal-warnings'
new_env["G_DEBUG"] = "fatal-warnings"
mount_process = subprocess.Popen(cmdline, env=new_env)
try:
wait_for_mount(mount_process, mnt_dir)
@@ -114,30 +159,20 @@ def test_sshfs(tmpdir, debug, cache_timeout, sync_rd, multiconn, capfd):
tst_truncate_path(mnt_dir)
tst_truncate_fd(mnt_dir)
tst_open_unlink(mnt_dir)
except:
except Exception as exc:
cleanup(mount_process, mnt_dir)
raise
raise exc
else:
umount(mount_process, mnt_dir)
@contextmanager
def os_open(name, flags):
fd = os.open(name, flags)
try:
yield fd
finally:
os.close(fd)
def os_create(name):
os.close(os.open(name, os.O_CREAT | os.O_RDWR))
def tst_unlink(src_dir, mnt_dir, cache_timeout):
name = name_generator()
fullname = mnt_dir + "/" + name
with open(pjoin(src_dir, name), 'wb') as fh:
fh.write(b'hello')
with open(pjoin(src_dir, name), "wb") as fh:
fh.write(b"hello")
if cache_timeout:
safe_sleep(cache_timeout+1)
safe_sleep(cache_timeout + 1)
assert name in os.listdir(mnt_dir)
os.unlink(fullname)
with pytest.raises(OSError) as exc_info:
@@ -146,22 +181,24 @@ def tst_unlink(src_dir, mnt_dir, cache_timeout):
assert name not in os.listdir(mnt_dir)
assert name not in os.listdir(src_dir)
def tst_mkdir(mnt_dir):
dirname = name_generator()
fullname = mnt_dir + "/" + dirname
os.mkdir(fullname)
fstat = os.stat(fullname)
assert stat.S_ISDIR(fstat.st_mode)
assert os.listdir(fullname) == []
assert fstat.st_nlink in (1,2)
assert os.listdir(fullname) == []
assert fstat.st_nlink in (1, 2)
assert dirname in os.listdir(mnt_dir)
def tst_rmdir(src_dir, mnt_dir, cache_timeout):
name = name_generator()
fullname = mnt_dir + "/" + name
os.mkdir(pjoin(src_dir, name))
if cache_timeout:
safe_sleep(cache_timeout+1)
safe_sleep(cache_timeout + 1)
assert name in os.listdir(mnt_dir)
os.rmdir(fullname)
with pytest.raises(OSError) as exc_info:
@@ -170,6 +207,7 @@ def tst_rmdir(src_dir, mnt_dir, cache_timeout):
assert name not in os.listdir(mnt_dir)
assert name not in os.listdir(src_dir)
def tst_symlink(mnt_dir):
linkname = name_generator()
fullname = mnt_dir + "/" + linkname
@@ -180,6 +218,7 @@ def tst_symlink(mnt_dir):
assert fstat.st_nlink == 1
assert linkname in os.listdir(mnt_dir)
def tst_create(mnt_dir):
name = name_generator()
fullname = pjoin(mnt_dir, name)
@@ -197,6 +236,7 @@ def tst_create(mnt_dir):
assert fstat.st_nlink == 1
assert fstat.st_size == 0
def tst_chown(mnt_dir):
filename = pjoin(mnt_dir, name_generator())
os.mkdir(filename)
@@ -216,37 +256,38 @@ def tst_chown(mnt_dir):
assert fstat.st_uid == uid_new
assert fstat.st_gid == gid_new
def tst_open_read(src_dir, mnt_dir):
name = name_generator()
with open(pjoin(src_dir, name), 'wb') as fh_out, \
open(TEST_FILE, 'rb') as fh_in:
with open(pjoin(src_dir, name), "wb") as fh_out, open(TEST_FILE, "rb") as fh_in:
shutil.copyfileobj(fh_in, fh_out)
assert filecmp.cmp(pjoin(mnt_dir, name), TEST_FILE, False)
def tst_open_write(src_dir, mnt_dir):
name = name_generator()
fd = os.open(pjoin(src_dir, name),
os.O_CREAT | os.O_RDWR)
fd = os.open(pjoin(src_dir, name), os.O_CREAT | os.O_RDWR)
os.close(fd)
fullname = pjoin(mnt_dir, name)
with open(fullname, 'wb') as fh_out, \
open(TEST_FILE, 'rb') as fh_in:
with open(fullname, "wb") as fh_out, open(TEST_FILE, "rb") as fh_in:
shutil.copyfileobj(fh_in, fh_out)
assert filecmp.cmp(fullname, TEST_FILE, False)
def tst_append(src_dir, mnt_dir):
name = name_generator()
os_create(pjoin(src_dir, name))
fullname = pjoin(mnt_dir, name)
with os_open(fullname, os.O_WRONLY) as fd:
os.write(fd, b'foo\n')
with os_open(fullname, os.O_WRONLY|os.O_APPEND) as fd:
os.write(fd, b'bar\n')
os.write(fd, b"foo\n")
with os_open(fullname, os.O_WRONLY | os.O_APPEND) as fd:
os.write(fd, b"bar\n")
with open(fullname, "rb") as fh:
assert fh.read() == b"foo\nbar\n"
with open(fullname, 'rb') as fh:
assert fh.read() == b'foo\nbar\n'
def tst_seek(src_dir, mnt_dir):
name = name_generator()
@@ -254,20 +295,21 @@ def tst_seek(src_dir, mnt_dir):
fullname = pjoin(mnt_dir, name)
with os_open(fullname, os.O_WRONLY) as fd:
os.lseek(fd, 1, os.SEEK_SET)
os.write(fd, b'foobar\n')
os.write(fd, b"foobar\n")
with os_open(fullname, os.O_WRONLY) as fd:
os.lseek(fd, 4, os.SEEK_SET)
os.write(fd, b'com')
os.write(fd, b"com")
with open(fullname, "rb") as fh:
assert fh.read() == b"\0foocom\n"
with open(fullname, 'rb') as fh:
assert fh.read() == b'\0foocom\n'
def tst_open_unlink(mnt_dir):
name = pjoin(mnt_dir, name_generator())
data1 = b'foo'
data2 = b'bar'
data1 = b"foo"
data2 = b"bar"
fullname = pjoin(mnt_dir, name)
with open(fullname, 'wb+', buffering=0) as fh:
with open(fullname, "wb+", buffering=0) as fh:
fh.write(data1)
os.unlink(fullname)
with pytest.raises(OSError) as exc_info:
@@ -276,11 +318,13 @@ def tst_open_unlink(mnt_dir):
assert name not in os.listdir(mnt_dir)
fh.write(data2)
fh.seek(0)
assert fh.read() == data1+data2
assert fh.read() == data1 + data2
def tst_statvfs(mnt_dir):
os.statvfs(mnt_dir)
def tst_link(mnt_dir, cache_timeout):
name1 = pjoin(mnt_dir, name_generator())
name2 = pjoin(mnt_dir, name_generator())
@@ -299,11 +343,19 @@ def tst_link(mnt_dir, cache_timeout):
# we need to wait until the cached value has expired.
if cache_timeout:
safe_sleep(cache_timeout)
fstat1 = os.lstat(name1)
fstat2 = os.lstat(name2)
for attr in ('st_mode', 'st_dev', 'st_uid', 'st_gid',
'st_size', 'st_atime', 'st_mtime', 'st_ctime'):
for attr in (
"st_mode",
"st_dev",
"st_uid",
"st_gid",
"st_size",
"st_atime",
"st_mtime",
"st_ctime",
):
assert getattr(fstat1, attr) == getattr(fstat2, attr)
assert os.path.basename(name2) in os.listdir(mnt_dir)
assert filecmp.cmp(name1, name2, False)
@@ -316,6 +368,7 @@ def tst_link(mnt_dir, cache_timeout):
os.unlink(name1)
def tst_readdir(src_dir, mnt_dir):
newdir = name_generator()
src_newdir = pjoin(src_dir, newdir)
@@ -331,7 +384,7 @@ def tst_readdir(src_dir, mnt_dir):
listdir_is = os.listdir(mnt_newdir)
listdir_is.sort()
listdir_should = [ os.path.basename(file_), os.path.basename(subdir) ]
listdir_should = [os.path.basename(file_), os.path.basename(subdir)]
listdir_should.sort()
assert listdir_is == listdir_should
@@ -340,11 +393,12 @@ def tst_readdir(src_dir, mnt_dir):
os.rmdir(subdir)
os.rmdir(src_newdir)
def tst_truncate_path(mnt_dir):
assert len(TEST_DATA) > 1024
filename = pjoin(mnt_dir, name_generator())
with open(filename, 'wb') as fh:
with open(filename, "wb") as fh:
fh.write(TEST_DATA)
fstat = os.stat(filename)
@@ -354,21 +408,22 @@ def tst_truncate_path(mnt_dir):
# Add zeros at the end
os.truncate(filename, size + 1024)
assert os.stat(filename).st_size == size + 1024
with open(filename, 'rb') as fh:
with open(filename, "rb") as fh:
assert fh.read(size) == TEST_DATA
assert fh.read(1025) == b'\0' * 1024
assert fh.read(1025) == b"\0" * 1024
# Truncate data
os.truncate(filename, size - 1024)
assert os.stat(filename).st_size == size - 1024
with open(filename, 'rb') as fh:
assert fh.read(size) == TEST_DATA[:size-1024]
with open(filename, "rb") as fh:
assert fh.read(size) == TEST_DATA[: size - 1024]
os.unlink(filename)
def tst_truncate_fd(mnt_dir):
assert len(TEST_DATA) > 1024
with NamedTemporaryFile('w+b', 0, dir=mnt_dir) as fh:
with NamedTemporaryFile("w+b", 0, dir=mnt_dir) as fh:
fd = fh.fileno()
fh.write(TEST_DATA)
fstat = os.fstat(fd)
@@ -380,13 +435,14 @@ def tst_truncate_fd(mnt_dir):
assert os.fstat(fd).st_size == size + 1024
fh.seek(0)
assert fh.read(size) == TEST_DATA
assert fh.read(1025) == b'\0' * 1024
assert fh.read(1025) == b"\0" * 1024
# Truncate data
os.ftruncate(fd, size - 1024)
assert os.fstat(fd).st_size == size - 1024
fh.seek(0)
assert fh.read(size) == TEST_DATA[:size-1024]
assert fh.read(size) == TEST_DATA[: size - 1024]
def tst_utimens(mnt_dir, tol=0):
filename = pjoin(mnt_dir, name_generator())
@@ -395,20 +451,21 @@ def tst_utimens(mnt_dir, tol=0):
atime = fstat.st_atime + 42.28
mtime = fstat.st_mtime - 42.23
if sys.version_info < (3,3):
if sys.version_info < (3, 3):
os.utime(filename, (atime, mtime))
else:
atime_ns = fstat.st_atime_ns + int(42.28*1e9)
mtime_ns = fstat.st_mtime_ns - int(42.23*1e9)
atime_ns = fstat.st_atime_ns + int(42.28 * 1e9)
mtime_ns = fstat.st_mtime_ns - int(42.23 * 1e9)
os.utime(filename, None, ns=(atime_ns, mtime_ns))
fstat = os.lstat(filename)
assert abs(fstat.st_atime - atime) < tol
assert abs(fstat.st_mtime - mtime) < tol
if sys.version_info >= (3,3):
assert abs(fstat.st_atime_ns - atime_ns) < tol*1e9
assert abs(fstat.st_mtime_ns - mtime_ns) < tol*1e9
if sys.version_info >= (3, 3):
assert abs(fstat.st_atime_ns - atime_ns) < tol * 1e9
assert abs(fstat.st_mtime_ns - mtime_ns) < tol * 1e9
def tst_utimens_now(mnt_dir):
fullname = pjoin(mnt_dir, name_generator())
@@ -422,17 +479,18 @@ def tst_utimens_now(mnt_dir):
assert fstat.st_atime != 0
assert fstat.st_mtime != 0
def tst_passthrough(src_dir, mnt_dir, cache_timeout):
name = name_generator()
src_name = pjoin(src_dir, name)
mnt_name = pjoin(src_dir, name)
assert name not in os.listdir(src_dir)
assert name not in os.listdir(mnt_dir)
with open(src_name, 'w') as fh:
fh.write('Hello, world')
with open(src_name, "w") as fh:
fh.write("Hello, world")
assert name in os.listdir(src_dir)
if cache_timeout:
safe_sleep(cache_timeout+1)
safe_sleep(cache_timeout + 1)
assert name in os.listdir(mnt_dir)
assert os.stat(src_name) == os.stat(mnt_name)
@@ -441,10 +499,10 @@ def tst_passthrough(src_dir, mnt_dir, cache_timeout):
mnt_name = pjoin(src_dir, name)
assert name not in os.listdir(src_dir)
assert name not in os.listdir(mnt_dir)
with open(mnt_name, 'w') as fh:
fh.write('Hello, world')
with open(mnt_name, "w") as fh:
fh.write("Hello, world")
assert name in os.listdir(src_dir)
if cache_timeout:
safe_sleep(cache_timeout+1)
safe_sleep(cache_timeout + 1)
assert name in os.listdir(mnt_dir)
assert os.stat(src_name) == os.stat(mnt_name)
+12 -8
View File
@@ -6,36 +6,40 @@ set -e
# that we still need to fix
export ASAN_OPTIONS="detect_leaks=0"
export LSAN_OPTIONS="suppressions=$(pwd)/test/lsan_suppress.txt"
export LSAN_OPTIONS="suppressions=${PWD}/test/lsan_suppress.txt"
export CC
TEST_CMD="python3 -m pytest --maxfail=99 test/"
# Standard build with Valgrind
for CC in gcc gcc-6 clang; do
mkdir build-${CC}; cd build-${CC}
if [ ${CC} == 'gcc-6' ]; then
for CC in gcc clang; do
(
mkdir "build-${CC}"; cd "build-${CC}"
if [ "${CC}" == 'gcc-6' ]; then
build_opts='-D b_lundef=false'
else
build_opts=''
fi
# shellcheck disable=SC2086
meson -D werror=true ${build_opts} ../
ninja
TEST_WITH_VALGRIND=true ${TEST_CMD}
cd ..
)
done
(cd build-$CC; sudo ninja install)
(cd "build-${CC}"; sudo ninja install)
# Sanitized build
CC=clang
for san in undefined address; do
mkdir build-${san}; cd build-${san}
(
mkdir "build-${san}"
cd "build-${san}"
# b_lundef=false is required to work around clang
# bug, cf. https://groups.google.com/forum/#!topic/mesonbuild/tgEdAXIIdC4
meson -D b_sanitize=${san} -D b_lundef=false -D werror=true ..
ninja
${TEST_CMD}
sudo ninja install
cd ..
)
done
+1 -8
View File
@@ -2,25 +2,18 @@
set -e
# Meson 0.45 requires Python 3.5 or newer
sudo python3 -m pip install pytest meson==0.44
valgrind --version
ninja --version
meson --version
# Install fuse
wget https://github.com/libfuse/libfuse/archive/master.zip
unzip master.zip
cd libfuse-master
mkdir build
cd build
export CC=gcc-6
meson ..
ninja
sudo ninja install
test -e /usr/local/lib/pkgconfig || sudo mkdir /usr/local/lib/pkgconfig
sudo mv /usr/local/lib/*/pkgconfig/* /usr/local/lib/pkgconfig/
ls -d1 /usr/local/lib/*-linux-gnu | sudo tee /etc/ld.so.conf.d/usrlocal.conf
printf '%s\n' /usr/local/lib/*-linux-gnu | sudo tee /etc/ld.so.conf.d/usrlocal.conf
sudo ldconfig
# Setup ssh
+50 -25
View File
@@ -5,34 +5,51 @@ import os
import stat
import time
from os.path import join as pjoin
from contextlib import contextmanager
basename = pjoin(os.path.dirname(__file__), '..')
basename = pjoin(os.path.dirname(__file__), "..")
def wait_for_mount(mount_process, mnt_dir,
test_fn=os.path.ismount):
def os_create(name):
os.close(os.open(name, os.O_CREAT | os.O_RDWR))
@contextmanager
def os_open(name, flags):
fd = os.open(name, flags)
try:
yield fd
finally:
os.close(fd)
def wait_for_mount(mount_process, mnt_dir, test_fn=os.path.ismount):
elapsed = 0
while elapsed < 30:
if test_fn(mnt_dir):
return True
if mount_process.poll() is not None:
pytest.fail('file system process terminated prematurely')
pytest.fail("file system process terminated prematurely")
time.sleep(0.1)
elapsed += 0.1
pytest.fail("mountpoint failed to come up")
def cleanup(mount_process, mnt_dir):
subprocess.call(['fusermount', '-z', '-u', mnt_dir],
stdout=subprocess.DEVNULL,
stderr=subprocess.STDOUT)
subprocess.call(
["fusermount", "-z", "-u", mnt_dir],
stdout=subprocess.DEVNULL,
stderr=subprocess.STDOUT,
)
mount_process.terminate()
try:
mount_process.wait(1)
except subprocess.TimeoutExpired:
mount_process.kill()
def umount(mount_process, mnt_dir):
subprocess.check_call(['fusermount3', '-z', '-u', mnt_dir ])
subprocess.check_call(["fusermount3", "-z", "-u", mnt_dir])
assert not os.path.ismount(mnt_dir)
# Give mount process a little while to terminate. Popen.wait(timeout)
@@ -43,18 +60,19 @@ def umount(mount_process, mnt_dir):
if code is not None:
if code == 0:
return
pytest.fail('file system process terminated with code %s' % (code,))
pytest.fail(f"file system process terminated with code {code}")
time.sleep(0.1)
elapsed += 0.1
pytest.fail('mount process did not terminate')
pytest.fail("mount process did not terminate")
def safe_sleep(secs):
'''Like time.sleep(), but sleep for at least *secs*
"""Like time.sleep(), but sleep for at least *secs*
`time.sleep` may sleep less than the given period if a signal is
received. This function ensures that we sleep for at least the
desired time.
'''
"""
now = time.time()
end = now + secs
@@ -62,24 +80,27 @@ def safe_sleep(secs):
time.sleep(end - now)
now = time.time()
def fuse_test_marker():
'''Return a pytest.marker that indicates FUSE availability
"""Return a pytest.marker that indicates FUSE availability
If system/user/environment does not support FUSE, return
a `pytest.mark.skip` object with more details. If FUSE is
supported, return `pytest.mark.uses_fuse()`.
'''
"""
skip = lambda x: pytest.mark.skip(reason=x)
def skip(reason: str):
return pytest.mark.skip(reason=reason)
with subprocess.Popen(['which', 'fusermount'], stdout=subprocess.PIPE,
universal_newlines=True) as which:
with subprocess.Popen(
["which", "fusermount"], stdout=subprocess.PIPE, universal_newlines=True
) as which:
fusermount_path = which.communicate()[0].strip()
if not fusermount_path or which.returncode != 0:
return skip("Can't find fusermount executable")
if not os.path.exists('/dev/fuse'):
if not os.path.exists("/dev/fuse"):
return skip("FUSE kernel module does not seem to be loaded")
if os.getuid() == 0:
@@ -87,20 +108,24 @@ def fuse_test_marker():
mode = os.stat(fusermount_path).st_mode
if mode & stat.S_ISUID == 0:
return skip('fusermount executable not setuid, and we are not root.')
return skip("fusermount executable not setuid, and we are not root.")
try:
fd = os.open('/dev/fuse', os.O_RDWR)
fd = os.open("/dev/fuse", os.O_RDWR)
except OSError as exc:
return skip('Unable to open /dev/fuse: %s' % exc.strerror)
return skip(f"Unable to open /dev/fuse: {exc.strerror}")
else:
os.close(fd)
return pytest.mark.uses_fuse()
# Use valgrind if requested
if os.environ.get('TEST_WITH_VALGRIND', 'no').lower().strip() \
not in ('no', 'false', '0'):
base_cmdline = [ 'valgrind', '-q', '--' ]
if os.environ.get("TEST_WITH_VALGRIND", "no").lower().strip() not in (
"no",
"false",
"0",
):
base_cmdline = ["valgrind", "-q", "--"]
else:
base_cmdline = []