Compare commits

..

23 Commits

Author SHA1 Message Date
Nikolaus Rath a7e1038203 Released 3.7.0 2020-01-03 11:04:02 +00:00
Dominique Martinet 8340a67b31 sftp_init_reply_ok: fix small memory leak (#198)
The leak was identified with ASAN: configure the project with
meson -Db_sanitize=address to reproduce.
2019-11-30 11:42:40 +00:00
Nikolaus Rath ab0e339e80 Protect changes to conn->req_count with mutex. 2019-11-27 20:35:38 +00:00
Nikolaus Rath b19e3b8001 Fix memory leak in conntab
References need to be counted per-path, rather than per connection.
2019-11-27 20:35:38 +00:00
Nikolaus Rath e910453156 Disable buflimit workaround by default.
The corresponding bug in OpenSSH has been fixed in
2007 (cf. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=365541#37), so this shouldn't
be needed anymore.
2019-11-27 10:30:35 +00:00
Nikolaus Rath 28c22270f8 Add missing mutex lock/unlocks when accessing conntab
(problem introduced in parent commit).
2019-11-27 09:42:30 +00:00
Timo Savola 8822b60d9d Add support for using multiple connections
The -o max_conns=N option causes multiple SSH processes and response processing threads to
be created. This means that e.g. reading a large file no longer blocks all access to the
filesystem.

The connection is chosen by checking the per-connection statistics:

  1. Choose connection with least outstanding requests; if it's a tie,
  2. choose connection with least directory handles; if it's a tie,
  3. choose connection with least file handles; if it's a tie,
  4. choose connection which has already been established.

The implementation assumes that the max_conns values will be small; it
uses linear search.

Example benchmark:

With single connection:

$ sshfs -o max_conns=1,workaround=nobuflimit ebox: mnt
$ cat mnt/tmp/bigfile > /dev/null &
$ time find mnt > /dev/null

real	1m50.432s
user	0m0.133s
sys	0m0.467s

With multiple connections:

$ ~/in-progress/sshfs/build/sshfs -o max_conns=5,workaround=nobuflimit ebox: mnt
$ cat mnt/tmp/bigfile > /dev/null &
$ time find mnt > /dev/null

real	1m15.338s
user	0m0.142s
sys	0m0.491s

This feature was implemented to large extend by Timo Savola <timo.savola@iki.fi>. Thanks
to CEA.fr for sponsoring the remaining work to complete this feature and integrate it into
SSHFS!
2019-11-24 12:01:01 +00:00
Nikolaus Rath 0f3ab4fd4f Drop reference counter for struct sshfs_file
Variables of this kind are created in sshfs_open_common() and freed
in sshfs_release(). Since sshfs_release() calls sshfs_flush(), there
can be no pending write requests before at the time of freeing, so
there is no need for reference counting.
2019-11-24 10:23:42 +00:00
Nikolaus Rath ddd968b025 Do not fail tests if connecting to localhost for first time. 2019-11-24 10:19:13 +00:00
Nikolaus Rath 2421675f3f Added build/ to gitignore.
This is the default build directory.
2019-11-23 20:58:41 +00:00
jeg139 eade8feb05 fix some whitespace and indentation (#194) 2019-11-23 11:07:19 +00:00
Michael Forney 4d866526dc Fix some inconsistent whitespace (#192) 2019-11-23 11:06:05 +00:00
Nikolaus Rath 6935b49eea Add documentation of permission handling to manpage.
Fixes: #182
2019-11-03 13:39:23 +00:00
Nikolaus Rath a548abd1f3 Fix markup. 2019-11-03 09:38:31 +00:00
Nikolaus Rath a1e5f12c53 Released 3.6.0 2019-11-03 09:34:29 +00:00
kalvdans 011986306b Remedy gcc 7.4.0 warnings (#187)
* Use logical not for booleans

Fixes a gcc 7.4.0 complaint:

sshfs.c:1743:28: warning: ‘~’ on a boolean expression [-Wbool-operation]
sshfs.c:1743:28: note: did you mean to use logical not?

* strdup argument to keep it alive after function return

Fixes gcc 7.4.0 complaint:

sshfs.c: In function ‘sshfs_opt_proc’:
sshfs.c:3488:50: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
2019-11-03 09:28:36 +00:00
sunwire e1a9050c61 Added direct_io option (#173) 2019-10-19 20:26:10 +01:00
kalvdans af63f35ac1 Free addrinfo and close socket when returning error (#186) 2019-10-19 20:23:45 +01:00
Viktor Szakats 667cf34622 sshfs: fix another instance preventing use of global I/O size on macOS (#185)
Following-up on [1], there was another instance where blksize
was set to a non-zero value, thus making it impossible to
configure global I/O size on macOS, and using [2] the hard-wired
value of 4096 bytes instead, resulting in uniformly poor
performance [3].

With this patch, setting I/O size to a reasonable large value,
will result in much improved performance, e.g.:
  -o iosize=1048576

[1] https://github.com/osxfuse/sshfs/commit/5c0dbfe3eb40100f9277e863926f2e7d7c9a5a4c
[2] https://github.com/libfuse/sshfs/blob/4c21d696e9d46bebae0a936e2aec72326c5954ea/sshfs.c#L812
[3] https://github.com/libfuse/sshfs/issues/11#issuecomment-339407557
2019-09-12 05:33:32 -07:00
Michael Forney 4c21d696e9 Include poll.h instead of sys/poll.h (#178)
The standard header for the poll(3) interface is poll.h[0]. This prevents
a warning when building with musl libc:

    In file included from sshfs.c:44:
    /usr/include/sys/poll.h:1:2: warning: #warning redirecting incorrect #include <sys/poll.h> to <poll.h> [-Wcpp]
     #warning redirecting incorrect #include <sys/poll.h> to <poll.h>
      ^~~~~~~

[0] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/poll.h.html
2019-06-30 10:12:19 +01:00
Galen Getsov 469c96b6d2 Add --verbose option 2019-06-30 10:10:41 +01:00
Nikolaus Rath c4485188c9 Update issue templates 2019-04-30 06:06:39 -07:00
Nikolaus Rath 4dc9b612ec Documented development status. 2019-04-29 11:58:52 -07:00
10 changed files with 583 additions and 216 deletions
+17
View File
@@ -0,0 +1,17 @@
---
name: Issue report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
PLEASE READ BEFORE REPORTING AN ISSUE
SSHFS does not have any active, regular contributors or developers. The current maintainer continues to apply pull requests and tries to make regular releases, but unfortunately has no capacity to do any development beyond addressing high-impact issues. When reporting bugs, please understand that unless you are including a pull request or are reporting a critical issue, you will probably not get a response.
To prevent the issue tracker from being flooded with issues that no-one is intending to work on, and to give more visibilty to critical issues that users should be aware of and that most urgently need attention, I will also close most bug reports once they've been inactive for a while.
Please note that this isn't meant to imply that you haven't found a bug - you most likely have and I'm grateful that you took the time to report it. Unfortunately, SSHFS is a purely volunteer driven project,
and at the moment there simply aren't any volunteers.
+1
View File
@@ -34,3 +34,4 @@ sshfs.1
/patches
/m4
.deps/
/build
+8
View File
@@ -23,16 +23,22 @@ Brandon Carter <b-carter@users.noreply.github.com>
Chris Wolfe <cwolfe@chromium.org>
Clayton G. Hobbs <clay@lakeserv.net>
Daniel Lublin <daniel@lublin.se>
Dominique Martinet <asmadeus@codewreck.org>
DrDaveD <2129743+DrDaveD@users.noreply.github.com>
gala <gala132@users.noreply.github.com>
Galen Getsov <4815620+ggetsov@users.noreply.github.com>
George Vlahavas <vlahavas@gmail.com>
G.raud Meyer <graud@gmx.com>
harrim4n <git@harrim4n.com>
Jakub Jelen <jjelen@redhat.com>
jeg139 <54814784+jeg139@users.noreply.github.com>
Josh Triplett <josh@joshtriplett.org>
Julio Merino <jmmv@google.com>
Julio Merino <jmmv@meroh.net>
kalvdans <github@kalvdans.no-ip.org>
Michael Forney <mforney@mforney.org>
Mike Kelly <mike@pair.com>
Mike Salvatore <mike.s.salvatore@gmail.com>
Miklos Szeredi <miklos@szeredi.hu>
Miklos Szeredi <mszeredi@suse.cz>
mssalvatore <mike.s.salvatore@gmail.com>
@@ -46,7 +52,9 @@ Rian Hunter <rianhunter@users.noreply.github.com>
Samuel Murray <samuel.murray@outlook.com>
S. D. Cloudt <s.d.cloudt@student.tue.nl>
smheidrich <smheidrich@weltenfunktion.de>
sunwire <50745572+sunwire@users.noreply.github.com>
Tim Harder <radhermit@gmail.com>
Timo Savola <timo.savola@iki.fi>
tpoindessous <thomas@poindessous.com>
Viktor Szakats <vszakats@users.noreply.github.com>
Zoltan Kuscsik <zoltan.kuscsik@linaro.org>
+27
View File
@@ -1,3 +1,30 @@
Release 3.7.0 (2020-01-03)
--------------------------
* New max_conns option enables the use of multiple connections to improve responsiveness
during large file transfers. Thanks to Timo Savola for doing most of the implementation
work, and thanks to CEA.fr for sponsoring remaining bugfixes and cleanups!
* The `buflimit` workaround is now disabled by default. The corresponding bug in OpenSSH
has been fixed in 2007
(cf. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=365541#37), so this shouldn't be
needed anymore. If you depend on this workaround, please let the SSHFS maintainers know,
otherwise support for the workaround will be removed completely in a future version.
Release 3.6.0 (2019-11-03)
--------------------------
* Added "-o direct_io" option.
This option disables the use of page cache in kernel.
This is useful for example if the file size is not known before reading it.
For example if you mount /proc dir from a remote host without the direct_io
option, the read always will return zero bytes instead of actual data.
* Added --verbose option.
* Fixed a number of compiler warnings.
* Improved performance under OS X.
Release 3.5.2 (2019-04-13)
--------------------------
+14
View File
@@ -10,6 +10,20 @@ servers support and enable this SFTP access by default, so SSHFS is
very simple to use - there's nothing to do on the server-side.
Development Status
------------------
SSHFS is shipped by all major Linux distributions and has been in
production use across a wide range of systems for many years. However,
at present SSHFS does not have any active, regular contributors, and
there are a number of known issues (see the bugtracker). The current
maintainer continues to apply pull requests and makes regular
releases, but unfortunately has no capacity to do any development
beyond addressing high-impact issues. When reporting bugs, please
understand that unless you are including a pull request or are
reporting a critical issue, you will probably not get a response.
How to use
----------
+1 -1
View File
@@ -258,7 +258,7 @@ static void *cache_init(struct fuse_conn_info *conn,
res = cache.next_oper->init(conn, cfg);
// Cache requires a path for each request
cfg->nullpath_ok = 0;
cfg->nullpath_ok = 0;
return res;
}
+1 -1
View File
@@ -1,4 +1,4 @@
project('sshfs', 'c', version: '3.5.2',
project('sshfs', 'c', version: '3.7.0',
meson_version: '>= 0.40',
default_options: [ 'buildtype=debugoptimized' ])
+470 -200
View File
File diff suppressed because it is too large Load Diff
+37 -13
View File
@@ -27,21 +27,24 @@ To unmount it::
Description
===========
SSHFS allows you to mount a remote filesystem using SSH (more
precisely, the SFTP subsystem). Most SSH servers support and enable
this SFTP access by default, so SSHFS is very simple to use - there's
nothing to do on the server-side.
SSHFS allows you to mount a remote filesystem using SSH (more precisely, the SFTP
subsystem). Most SSH servers support and enable this SFTP access by default, so SSHFS is
very simple to use - there's nothing to do on the server-side.
SSHFS uses FUSE (Filesystem in Userspace) and should work on any
operating system that provides a FUSE implementation. Currently,
this includes Linux, FreeBSD and Mac OS X.
By default, file permissions are ignored by SSHFS. Any user that can access the filesystem
will be able to perform any operation that the remote server permits - based on the
credentials that were used to connect to the server. If this is undesired, local
permission checking can be enabled with ``-o default_permissions``.
It is recommended to run SSHFS as regular user (not as root). For
this to work the mountpoint must be owned by the user. If username is
omitted SSHFS will use the local username. If the directory is
omitted, SSHFS will mount the (remote) home directory. If you need to
enter a password sshfs will ask for it (actually it just runs ssh
which ask for the password if needed).
By default, only the mounting user will be able to access the filesystem. Access for other
users can be enabled by passing ``-o allow_others``. In this case you most likely also
want to use ``-o default_permissions``.
It is recommended to run SSHFS as regular user (not as root). For this to work the
mountpoint must be owned by the user. If username is omitted SSHFS will use the local
username. If the directory is omitted, SSHFS will mount the (remote) home directory. If
you need to enter a password sshfs will ask for it (actually it just runs ssh which ask
for the password if needed).
Options
@@ -202,6 +205,27 @@ Options
sets the interval for forced cleaning of the directory cache
when full.
-o direct_io
This option disables the use of page cache (file content cache) in
the kernel for this filesystem.
This has several affects:
1. Each read() or write() system call will initiate one or more read or
write operations, data will not be cached in the kernel.
2. The return value of the read() and write() system calls will correspond
to the return values of the read and write operations. This is useful
for example if the file size is not known in advance (before reading it).
e.g. /proc filesystem
-o max_conns=N
sets the maximum number of simultaneous SSH connections
to use. Each connection is established with a separate SSH process.
The primary purpose of this feature is to improve the responsiveness of the
file system during large file transfers. When using more than once
connection, the *password_stdin* and *slave* options can not be
used, and the *buflimit* workaround is not supported/
In addition, SSHFS accepts several options common to all FUSE file
systems. These are described in the `mount.fuse` manpage (look
for "general", "libfuse specific", and "high-level API" options).
+7 -1
View File
@@ -33,13 +33,17 @@ def name_generator(__ctr=[0]):
@pytest.mark.parametrize("debug", (False, True))
@pytest.mark.parametrize("cache_timeout", (0,1))
@pytest.mark.parametrize("sync_rd", (True, False))
def test_sshfs(tmpdir, debug, cache_timeout, sync_rd, capfd):
@pytest.mark.parametrize("multiconn", (True,False))
def test_sshfs(tmpdir, debug, cache_timeout, sync_rd, multiconn, capfd):
# Avoid false positives from debug messages
#if debug:
# capfd.register_output(r'^ unique: [0-9]+, error: -[0-9]+ .+$',
# count=0)
# Avoid false positives from storing key for localhost
capfd.register_output(r"^Warning: Permanently added 'localhost' .+", count=0)
# Test if we can ssh into localhost without password
try:
res = subprocess.call(['ssh', '-o', 'KbdInteractiveAuthentication=no',
@@ -74,6 +78,8 @@ def test_sshfs(tmpdir, debug, cache_timeout, sync_rd, capfd):
cmdline += [ '-o', 'entry_timeout=0',
'-o', 'attr_timeout=0' ]
if multiconn:
cmdline += [ '-o', 'max_conns=3' ]
new_env = dict(os.environ) # copy, don't modify