[SCM] Samba Shared Repository - branch master updated
Jeremy Allison
jra at samba.org
Fri May 25 23:44:04 UTC 2018
The branch, master has been updated
via 1531fcd vfs_gpfs: Use full_path_tos instead of talloc_asprintf
via 1843a2d vfs_gpfs: Remove get_full_smb_filename from is_offline check
via 4a29a94 vfs_gpfs: Remove wrong get_full_smb_filename from ntimes function
via 01a6aef vfs_gpfs: Adjust debug level when get_winattrs returns EBADF
via 9a79a61 smbd: Move dfree_info struct
via e30d0c0 smbd: Flush dfree memcache on service reload
via 8f12174 smbd: Cache dfree information based on query path
via f5d0556 memcache: Add new cache type for dfree information
via a55b3d2 selftest: Add test for 'dfree cache'
via 7ffcbd5 selftest: Add dfq_cache share with 'dfree cache time' set
from 52778af s3: vfs_fake_acls: Correctly implement the chmod/fchmod algorithm on fake acls.
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 1531fcde8252efcb2f69dd5462beb83786bb1135
Author: Christof Schmitt <cs at samba.org>
Date: Tue May 22 12:52:58 2018 -0700
vfs_gpfs: Use full_path_tos instead of talloc_asprintf
full_path_tos avoids the talloc call for most cases; use that instead of
talloc_asprintf.
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat May 26 01:43:09 CEST 2018 on sn-devel-144
commit 1843a2d63259c819eb7a6eefa7f4ab8c46efec37
Author: Christof Schmitt <cs at samba.org>
Date: Tue May 22 12:25:42 2018 -0700
vfs_gpfs: Remove get_full_smb_filename from is_offline check
No stream information is required here.
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 4a29a949ac45fab55b8c41dcbe11a5e07fd62c05
Author: Christof Schmitt <cs at samba.org>
Date: Tue May 22 12:22:06 2018 -0700
vfs_gpfs: Remove wrong get_full_smb_filename from ntimes function
Updating the timestamps requires the path to the file, but no stream
information.
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 01a6aef52227e0f029fa107140e670d8af45ad59
Author: Christof Schmitt <cs at samba.org>
Date: Wed May 23 15:04:08 2018 -0700
vfs_gpfs: Adjust debug level when get_winattrs returns EBADF
This is returned for a get_winattrs call against a non-gpfs file system.
This can happen for the .. entry when listing a share on the file system
root.
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 9a79a61abb08719acde6aec8df9356be36d8dda2
Author: Christof Schmitt <cs at samba.org>
Date: Wed May 16 13:25:54 2018 -0700
smbd: Move dfree_info struct
As the struct is no longer used as part of connection_struct, move it to
dfree.c.
This is not backported, as it would change the VFS ABI.
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit e30d0c0e0d11f65b2d1886be3c0fe9e32eaf3926
Author: Christof Schmitt <cs at samba.org>
Date: Fri May 18 20:51:58 2018 -0700
smbd: Flush dfree memcache on service reload
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 8f121747b06ca78cf51801a3931b2ddd1a424c77
Author: Christof Schmitt <cs at samba.org>
Date: Wed May 16 13:17:52 2018 -0700
smbd: Cache dfree information based on query path
Sub directories in a SMB share can have different free space information
(e.g. when a different file system is mounted there). Caching the dfree
information per SMB share will return invalid data. Address this by
switching to memcache and store the cached data based on the query path.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit f5d05562679f6aa691b98b4a75952f7dda7ed343
Author: Christof Schmitt <cs at samba.org>
Date: Wed May 16 13:05:36 2018 -0700
memcache: Add new cache type for dfree information
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit a55b3d2fcc2f7737a2702bf908dcf1f80969bf21
Author: Christof Schmitt <cs at samba.org>
Date: Wed May 23 11:25:42 2018 -0700
selftest: Add test for 'dfree cache'
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 7ffcbd5ce1222971cb9879f78765d87cdc4102a8
Author: Christof Schmitt <cs at samba.org>
Date: Wed May 23 11:07:54 2018 -0700
selftest: Add dfq_cache share with 'dfree cache time' set
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446
Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
-----------------------------------------------------------------------
Summary of changes:
lib/util/memcache.h | 3 +-
selftest/target/Samba3.pm | 6 ++
source3/include/vfs.h | 11 +--
source3/modules/vfs_gpfs.c | 53 +++++++--------
source3/script/tests/test_dfree_quota.sh | 35 ++++++++++
source3/smbd/dfree.c | 112 +++++++++++++++++++++++++------
source3/smbd/proto.h | 1 +
source3/smbd/server_reload.c | 1 +
8 files changed, 164 insertions(+), 58 deletions(-)
Changeset truncated at 500 lines:
diff --git a/lib/util/memcache.h b/lib/util/memcache.h
index 670cbd6..4331c2f 100644
--- a/lib/util/memcache.h
+++ b/lib/util/memcache.h
@@ -45,7 +45,8 @@ enum memcache_number {
SINGLETON_CACHE,
SMB1_SEARCH_OFFSET_MAP,
SHARE_MODE_LOCK_CACHE, /* talloc */
- VIRUSFILTER_SCAN_RESULTS_CACHE_TALLOC /* talloc */
+ VIRUSFILTER_SCAN_RESULTS_CACHE_TALLOC, /* talloc */
+ DFREE_CACHE,
};
/*
diff --git a/selftest/target/Samba3.pm b/selftest/target/Samba3.pm
index 2b1752f..c6ae7ab 100755
--- a/selftest/target/Samba3.pm
+++ b/selftest/target/Samba3.pm
@@ -2116,6 +2116,12 @@ sub provision($$$$$$$$$)
vfs objects = acl_xattr fake_acls xattr_tdb fake_dfq
admin users = $unix_name
include = $dfqconffile
+[dfq_cache]
+ path = $shrdir/dfree
+ vfs objects = acl_xattr fake_acls xattr_tdb fake_dfq
+ admin users = $unix_name
+ include = $dfqconffile
+ dfree cache time = 60
[dfq_owner]
path = $shrdir/dfree
vfs objects = acl_xattr fake_acls xattr_tdb fake_dfq
diff --git a/source3/include/vfs.h b/source3/include/vfs.h
index 945e4b6..81fb6c1 100644
--- a/source3/include/vfs.h
+++ b/source3/include/vfs.h
@@ -253,6 +253,8 @@
All users are now pwrite or async versions. */
/* Version 39 - Remove SMB_VFS_CHMOD_ACL - no longer used. */
/* Version 39 - Remove SMB_VFS_FCHMOD_ACL - no longer used. */
+/* Version 39 - Remove struct dfree_cached_info pointer from
+ connection struct */
#define SMB_VFS_INTERFACE_VERSION 39
@@ -411,14 +413,6 @@ typedef struct {
bool is_wild;
} name_compare_entry;
-struct dfree_cached_info {
- time_t last_dfree_time;
- uint64_t dfree_ret;
- uint64_t bsize;
- uint64_t dfree;
- uint64_t dsize;
-};
-
struct share_params {
int service;
};
@@ -481,7 +475,6 @@ typedef struct connection_struct {
name_compare_entry *veto_list; /* Per-share list of files to veto (never show). */
name_compare_entry *veto_oplock_list; /* Per-share list of files to refuse oplocks on. */
name_compare_entry *aio_write_behind_list; /* Per-share list of files to use aio write behind on. */
- struct dfree_cached_info *dfree_info;
struct trans_state *pending_trans;
struct rpc_pipe_client *spoolss_pipe;
diff --git a/source3/modules/vfs_gpfs.c b/source3/modules/vfs_gpfs.c
index a0fd48f..d733df0 100644
--- a/source3/modules/vfs_gpfs.c
+++ b/source3/modules/vfs_gpfs.c
@@ -243,8 +243,10 @@ static int vfs_gpfs_get_real_filename(struct vfs_handle_struct *handle,
char **found_name)
{
int result;
- char *full_path;
- char real_pathname[PATH_MAX+1];
+ char *full_path = NULL;
+ char *to_free = NULL;
+ char real_pathname[PATH_MAX+1], tmpbuf[PATH_MAX];
+ size_t full_path_len;
int buflen;
bool mangled;
struct gpfs_config_data *config;
@@ -264,8 +266,9 @@ static int vfs_gpfs_get_real_filename(struct vfs_handle_struct *handle,
mem_ctx, found_name);
}
- full_path = talloc_asprintf(talloc_tos(), "%s/%s", path, name);
- if (full_path == NULL) {
+ full_path_len = full_path_tos(path, name, tmpbuf, sizeof(tmpbuf),
+ &full_path, &to_free);
+ if (full_path_len == -1) {
errno = ENOMEM;
return -1;
}
@@ -275,7 +278,7 @@ static int vfs_gpfs_get_real_filename(struct vfs_handle_struct *handle,
result = gpfswrap_get_realfilename_path(full_path, real_pathname,
&buflen);
- TALLOC_FREE(full_path);
+ TALLOC_FREE(to_free);
if ((result == -1) && (errno == ENOSYS)) {
return SMB_VFS_NEXT_GET_REAL_FILENAME(
@@ -1612,7 +1615,16 @@ static NTSTATUS vfs_gpfs_get_dos_attributes(struct vfs_handle_struct *handle,
if (ret == -1 && errno == EACCES) {
ret = get_dos_attr_with_capability(smb_fname, &attrs);
}
- if (ret == -1) {
+
+ if (ret == -1 && errno == EBADF) {
+ /*
+ * Returned for directory listings in gpfs root for
+ * .. entry which steps out of gpfs.
+ */
+ DBG_DEBUG("Getting winattrs for %s returned EBADF.\n",
+ smb_fname->base_name);
+ return map_nt_error_from_unix(errno);
+ } else if (ret == -1) {
DBG_WARNING("Getting winattrs failed for %s: %s\n",
smb_fname->base_name, strerror(errno));
return map_nt_error_from_unix(errno);
@@ -1879,23 +1891,15 @@ static int vfs_gpfs_ntimes(struct vfs_handle_struct *handle,
struct gpfs_winattr attrs;
int ret;
- char *path = NULL;
- NTSTATUS status;
struct gpfs_config_data *config;
SMB_VFS_HANDLE_GET_DATA(handle, config,
struct gpfs_config_data,
return -1);
- status = get_full_smb_filename(talloc_tos(), smb_fname, &path);
- if (!NT_STATUS_IS_OK(status)) {
- errno = map_errno_from_nt_status(status);
- return -1;
- }
-
/* Try to use gpfs_set_times if it is enabled and available */
if (config->settimes) {
- ret = smbd_gpfs_set_times_path(path, ft);
+ ret = smbd_gpfs_set_times_path(smb_fname->base_name, ft);
if (ret == 0 || (ret == -1 && errno != ENOSYS)) {
return ret;
@@ -1928,7 +1932,7 @@ static int vfs_gpfs_ntimes(struct vfs_handle_struct *handle,
attrs.creationTime.tv_sec = ft->create_time.tv_sec;
attrs.creationTime.tv_nsec = ft->create_time.tv_nsec;
- ret = gpfswrap_set_winattrs_path(discard_const_p(char, path),
+ ret = gpfswrap_set_winattrs_path(smb_fname->base_name,
GPFS_WINATTR_SET_CREATION_TIME,
&attrs);
if(ret == -1 && errno != ENOSYS){
@@ -2002,8 +2006,6 @@ static bool vfs_gpfs_is_offline(struct vfs_handle_struct *handle,
SMB_STRUCT_STAT *sbuf)
{
struct gpfs_winattr attrs;
- char *path = NULL;
- NTSTATUS status;
struct gpfs_config_data *config;
int ret;
@@ -2015,24 +2017,17 @@ static bool vfs_gpfs_is_offline(struct vfs_handle_struct *handle,
return false;
}
- status = get_full_smb_filename(talloc_tos(), fname, &path);
- if (!NT_STATUS_IS_OK(status)) {
- return false;
- }
-
- ret = gpfswrap_get_winattrs_path(path, &attrs);
+ ret = gpfswrap_get_winattrs_path(fname->base_name, &attrs);
if (ret == -1) {
- TALLOC_FREE(path);
return false;
}
if ((attrs.winAttrs & GPFS_WINATTR_OFFLINE) != 0) {
- DEBUG(10, ("%s is offline\n", path));
- TALLOC_FREE(path);
+ DBG_DEBUG("%s is offline\n", fname->base_name);
return true;
}
- DEBUG(10, ("%s is online\n", path));
- TALLOC_FREE(path);
+
+ DBG_DEBUG("%s is online\n", fname->base_name);
return false;
}
diff --git a/source3/script/tests/test_dfree_quota.sh b/source3/script/tests/test_dfree_quota.sh
index 6e227c4..abd82b4 100755
--- a/source3/script/tests/test_dfree_quota.sh
+++ b/source3/script/tests/test_dfree_quota.sh
@@ -130,6 +130,35 @@ test_smbclient_dfree() {
return $status
}
+# Issue two queries to different directories in one session to test
+# caching effects
+test_smbclient_dfree_2() {
+ name="$1"
+ share="$2"
+ dir1="$3"
+ dir2="$4"
+ confs="$5"
+ expected="$6"
+ subunit_start_test "$name"
+ setup_conf $confs
+ output=$($VALGRIND $smbclient //$SERVER/$share \
+ -c "cd $dir1; du; cd ..; cd $dir2 ; du" $@ 2>&1)
+ status=$?
+ if [ "$status" = "0" ]; then
+ received=$(echo "$output" | \
+ awk '/blocks of size/ {print $1, $5, $6}' | \
+ tr '\n' ' ')
+ if [ "$expected" = "$received" ]; then
+ subunit_pass_test "$name"
+ else
+ echo "$output" | subunit_fail_test "$name"
+ fi
+ else
+ echo "$output" | subunit_fail_test "$name"
+ fi
+ return $status
+}
+
test_smbcquotas() {
name="$1"
conf="$2"
@@ -164,6 +193,12 @@ test_smbclient_dfree "Test large disk" dfq "." "conf3 ." "1125899906842624 1024.
#basic quota test (SMB1 only)
test_smbcquotas "Test user quota" confq1 $USERNAME "40960/4096000/3072000" -U$USERNAME%$PASSWORD --option=clientmaxprotocol=NT1 || failed=`expr $failed + 1`
+# Test dfree cache through queries in two different directories
+test_smbclient_dfree_2 "Test dfree cache" dfq_cache "." "subdir1" \
+ "conf1 . conf2 subdir1" "10 1024. 5 20 1024. 10 " \
+ -U$USERNAME%$PASSWORD --option=clientmaxprotocol=SMB3 \
+ || failed=`expr $failed + 1`
+
#quota limit > disk size, remaining quota > disk free
test_smbclient_dfree "Test dfree share root df vs quota case 1" dfq "." "confdfq1 ." "80 1024. 40" -U$USERNAME%$PASSWORD --option=clientmaxprotocol=SMB3 || failed=`expr $failed + 1`
#quota limit > disk size, remaining quota < disk free
diff --git a/source3/smbd/dfree.c b/source3/smbd/dfree.c
index a702d08..05f6d69 100644
--- a/source3/smbd/dfree.c
+++ b/source3/smbd/dfree.c
@@ -21,6 +21,7 @@
#include "smbd/smbd.h"
#include "smbd/globals.h"
#include "lib/util_file.h"
+#include "lib/util/memcache.h"
/****************************************************************************
Normalise for DOS usage.
@@ -167,48 +168,121 @@ dfree_done:
/****************************************************************************
Potentially returned cached dfree info.
+
+ Depending on the file system layout and file system features, the free space
+ information can be different for different sub directories underneath a SMB
+ share. Store the cache information in memcache using the query path as the
+ key to accomodate this.
****************************************************************************/
+struct dfree_cached_info {
+ time_t last_dfree_time;
+ uint64_t dfree_ret;
+ uint64_t bsize;
+ uint64_t dfree;
+ uint64_t dsize;
+};
+
uint64_t get_dfree_info(connection_struct *conn, struct smb_filename *fname,
uint64_t *bsize, uint64_t *dfree, uint64_t *dsize)
{
int dfree_cache_time = lp_dfree_cache_time(SNUM(conn));
- struct dfree_cached_info *dfc = conn->dfree_info;
+ struct dfree_cached_info *dfc = NULL;
+ struct dfree_cached_info dfc_new = { 0 };
uint64_t dfree_ret;
+ char tmpbuf[PATH_MAX];
+ char *full_path = NULL;
+ char *to_free = NULL;
+ char *key_path = NULL;
+ size_t len;
+ DATA_BLOB key, value;
+ bool found;
if (!dfree_cache_time) {
return sys_disk_free(conn, fname, bsize, dfree, dsize);
}
+ len = full_path_tos(conn->connectpath,
+ fname->base_name,
+ tmpbuf,
+ sizeof(tmpbuf),
+ &full_path,
+ &to_free);
+ if (len == -1) {
+ errno = ENOMEM;
+ return -1;
+ }
+
+ if (VALID_STAT(fname->st) && S_ISREG(fname->st.st_ex_mode)) {
+ /*
+ * In case of a file use the parent directory to reduce number
+ * of cache entries.
+ */
+ bool ok;
+
+ ok = parent_dirname(talloc_tos(),
+ full_path,
+ &key_path,
+ NULL);
+ TALLOC_FREE(to_free); /* We're done with full_path */
+
+ if (!ok) {
+ errno = ENOMEM;
+ return -1;
+ }
+
+ /*
+ * key_path is always a talloced object.
+ */
+ to_free = key_path;
+ } else {
+ /*
+ * key_path might not be a talloced object; rely on
+ * to_free set from full_path_tos.
+ */
+ key_path = full_path;
+ }
+
+ key = data_blob_const(key_path, strlen(key_path));
+ found = memcache_lookup(smbd_memcache(),
+ DFREE_CACHE,
+ key,
+ &value);
+ dfc = found ? (struct dfree_cached_info *)value.data : NULL;
+
if (dfc && (conn->lastused - dfc->last_dfree_time < dfree_cache_time)) {
- /* Return cached info. */
+ DBG_DEBUG("Returning dfree cache entry for %s\n", key_path);
*bsize = dfc->bsize;
*dfree = dfc->dfree;
*dsize = dfc->dsize;
- return dfc->dfree_ret;
+ dfree_ret = dfc->dfree_ret;
+ goto out;
}
dfree_ret = sys_disk_free(conn, fname, bsize, dfree, dsize);
if (dfree_ret == (uint64_t)-1) {
/* Don't cache bad data. */
- return dfree_ret;
- }
-
- /* No cached info or time to refresh. */
- if (!dfc) {
- dfc = talloc(conn, struct dfree_cached_info);
- if (!dfc) {
- return dfree_ret;
- }
- conn->dfree_info = dfc;
+ goto out;
}
- dfc->bsize = *bsize;
- dfc->dfree = *dfree;
- dfc->dsize = *dsize;
- dfc->dfree_ret = dfree_ret;
- dfc->last_dfree_time = conn->lastused;
-
+ DBG_DEBUG("Creating dfree cache entry for %s\n", key_path);
+ dfc_new.bsize = *bsize;
+ dfc_new.dfree = *dfree;
+ dfc_new.dsize = *dsize;
+ dfc_new.dfree_ret = dfree_ret;
+ dfc_new.last_dfree_time = conn->lastused;
+ memcache_add(smbd_memcache(),
+ DFREE_CACHE,
+ key,
+ data_blob_const(&dfc_new, sizeof(dfc_new)));
+
+out:
+ TALLOC_FREE(to_free);
return dfree_ret;
}
+
+void flush_dfree_cache(void)
+{
+ memcache_flush(smbd_memcache(), DFREE_CACHE);
+}
diff --git a/source3/smbd/proto.h b/source3/smbd/proto.h
index 86b6551..7a5ac4c 100644
--- a/source3/smbd/proto.h
+++ b/source3/smbd/proto.h
@@ -173,6 +173,7 @@ uint64_t sys_disk_free(connection_struct *conn, struct smb_filename *fname,
uint64_t *bsize, uint64_t *dfree, uint64_t *dsize);
uint64_t get_dfree_info(connection_struct *conn, struct smb_filename *fname,
uint64_t *bsize, uint64_t *dfree, uint64_t *dsize);
+void flush_dfree_cache(void);
/* The following definitions come from smbd/dir.c */
diff --git a/source3/smbd/server_reload.c b/source3/smbd/server_reload.c
index c93c077..9b62096 100644
--- a/source3/smbd/server_reload.c
+++ b/source3/smbd/server_reload.c
@@ -164,6 +164,7 @@ bool reload_services(struct smbd_server_connection *sconn,
mangle_reset_cache();
reset_stat_cache();
+ flush_dfree_cache();
/* this forces service parameters to be flushed */
set_current_service(NULL,0,True);
--
Samba Shared Repository
More information about the samba-cvs
mailing list