[SCM] Samba Shared Repository - branch master updated
Andreas Schneider
asn at samba.org
Mon May 7 13:14:03 MDT 2012
The branch, master has been updated
via 5909188 talloc: Update doxygen config.
via 2040828 doc: Remove latex to doxygen conversion leftovers in talloc.
via 6952699 doc: Fixes for the talloc best practices tutorial.
via c1c9ab1 doc: Fixes for the talloc debugging tutorial.
via 5a758f4 doc: Fixes for the talloc pool tutorial.
via 79efc9d doc: Fixes for the talloc destructor tutorial.
via 7d5565e doc: Fixes for the talloc dynamic type system tutorial.
via dc112dc doc: Fixes for the talloc stealing tutorial.
via 9423ac0 doc: Fixes for the talloc context tutorial.
via d99b7d0 doc: Add talloc tutorial.
from 890485b heimdal: Cope with newer Heimdal versions accepting a keyset argument to hdb_enctype2key.
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 59091884922eda5a1524ad97fa1a5d7477cb5b96
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:57:34 2012 +0200
talloc: Update doxygen config.
Autobuild-User: Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date: Mon May 7 21:13:15 CEST 2012 on sn-devel-104
commit 20408286e2845ebca64b28e7e8ec0ed8dc9130c2
Author: Pavel Březina <pbrezina at redhat.com>
Date: Mon May 7 12:30:44 2012 +0200
doc: Remove latex to doxygen conversion leftovers in talloc.
Signed-off-by: Andreas Schneider <asn at samba.org>
commit 69526997e5636211824f8c041a9e57d039cc62f4
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:56:39 2012 +0200
doc: Fixes for the talloc best practices tutorial.
commit c1c9ab1c79571dfa1ae0a9f92f401ea89a50ca0e
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:42:44 2012 +0200
doc: Fixes for the talloc debugging tutorial.
commit 5a758f448d61f327aa510b9b4506a509e346c5df
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:36:37 2012 +0200
doc: Fixes for the talloc pool tutorial.
commit 79efc9d6e2c57c6acd8216be4b3387180032addd
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:30:06 2012 +0200
doc: Fixes for the talloc destructor tutorial.
commit 7d5565e22d0ad8994225da18ef025bfdfa56c245
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:25:50 2012 +0200
doc: Fixes for the talloc dynamic type system tutorial.
commit dc112dcee058f2f1a211a50790f659a629ab2978
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:18:26 2012 +0200
doc: Fixes for the talloc stealing tutorial.
commit 9423ac06aa2a88a1cf64d5256e948bbec33ecb91
Author: Andreas Schneider <asn at samba.org>
Date: Mon May 7 11:09:56 2012 +0200
doc: Fixes for the talloc context tutorial.
commit d99b7d0220d8bd694c0d997622a7a87eb09c5570
Author: Pavel Březina <pbrezina at redhat.com>
Date: Sun May 6 14:34:48 2012 +0200
doc: Add talloc tutorial.
Signed-off-by: Andreas Schneider <asn at samba.org>
-----------------------------------------------------------------------
Summary of changes:
lib/talloc/doc/context.png | Bin 0 -> 4715 bytes
lib/talloc/doc/context_tree.png | Bin 0 -> 6158 bytes
lib/talloc/doc/mainpage.dox | 5 +
lib/talloc/doc/stealing.png | Bin 0 -> 6994 bytes
lib/talloc/doc/tutorial_bestpractices.dox | 192 +++++++++++
lib/talloc/doc/tutorial_context.dox | 198 +++++++++++
lib/talloc/doc/tutorial_debugging.dox | 116 +++++++
lib/talloc/doc/tutorial_destructors.dox | 82 +++++
lib/talloc/doc/tutorial_dts.dox | 109 ++++++
lib/talloc/doc/tutorial_introduction.dox | 43 +++
lib/talloc/doc/tutorial_pools.dox | 93 ++++++
lib/talloc/doc/tutorial_stealing.dox | 55 +++
lib/talloc/doxy.config | 512 ++++++++++++++++++++++-------
13 files changed, 1285 insertions(+), 120 deletions(-)
create mode 100644 lib/talloc/doc/context.png
create mode 100644 lib/talloc/doc/context_tree.png
create mode 100644 lib/talloc/doc/stealing.png
create mode 100644 lib/talloc/doc/tutorial_bestpractices.dox
create mode 100644 lib/talloc/doc/tutorial_context.dox
create mode 100644 lib/talloc/doc/tutorial_debugging.dox
create mode 100644 lib/talloc/doc/tutorial_destructors.dox
create mode 100644 lib/talloc/doc/tutorial_dts.dox
create mode 100644 lib/talloc/doc/tutorial_introduction.dox
create mode 100644 lib/talloc/doc/tutorial_pools.dox
create mode 100644 lib/talloc/doc/tutorial_stealing.dox
Changeset truncated at 500 lines:
diff --git a/lib/talloc/doc/context.png b/lib/talloc/doc/context.png
new file mode 100644
index 0000000..48a6ca0
Binary files /dev/null and b/lib/talloc/doc/context.png differ
diff --git a/lib/talloc/doc/context_tree.png b/lib/talloc/doc/context_tree.png
new file mode 100644
index 0000000..9723459
Binary files /dev/null and b/lib/talloc/doc/context_tree.png differ
diff --git a/lib/talloc/doc/mainpage.dox b/lib/talloc/doc/mainpage.dox
index 3204e8a..3b56898 100644
--- a/lib/talloc/doc/mainpage.dox
+++ b/lib/talloc/doc/mainpage.dox
@@ -10,6 +10,11 @@
* <a href="http://samba.org/ftp/talloc" target="_blank">talloc directory</a>
* on the samba public source archive.
*
+ * @section main-tutorial Tutorial
+ *
+ * You should start by reading @subpage libtalloc_tutorial, then reading the documentation of
+ * the interesting functions as you go.
+
* @section talloc_bugs Discussion and bug reports
*
* talloc does not currently have its own mailing list or bug tracking system.
diff --git a/lib/talloc/doc/stealing.png b/lib/talloc/doc/stealing.png
new file mode 100644
index 0000000..8833e06
Binary files /dev/null and b/lib/talloc/doc/stealing.png differ
diff --git a/lib/talloc/doc/tutorial_bestpractices.dox b/lib/talloc/doc/tutorial_bestpractices.dox
new file mode 100644
index 0000000..3634446
--- /dev/null
+++ b/lib/talloc/doc/tutorial_bestpractices.dox
@@ -0,0 +1,192 @@
+/**
+ at page libtalloc_bestpractices Chapter 7: Best practises
+
+The following sections contain several best practices and good manners that were
+found by the <a href="http://www.samba.org">Samba</a> and
+<a href="https://fedorahosted.org/sssd">SSSD</a> developers over the years.
+These will help you to write code which is better, easier to debug and with as
+few (hopefully none) memory leaks as possible.
+
+ at section bp-hierarchy Keep the context hierarchy steady
+
+The talloc is a hierarchy memory allocator. The hierarchy nature is what makes
+the programming more error proof. It makes the memory easier to manage and to
+free. Therefore, the first thing we should have on our mind is: always project
+your data structures into the talloc context hierarchy.
+
+That means if we have a structure, we should always use it as a parent context
+for its elements. This way we will not encounter any troubles when freeing the
+structure or when changing its parent. The same rule applies for arrays.
+
+For example, the structure <code>user</code> from section @ref context-hierarchy
+should be created with the context hierarchy illustrated on the next image.
+
+ at image html context_tree.png
+
+ at section bp-tmpctx Every function should use its own context
+
+It is a good practice to create a temporary talloc context at the function
+beginning and free the context just before the return statement. All the data
+must be allocated on this context or on its children. This ensures that no
+memory leaks are created as long as we do not forget to free the temporary
+context.
+
+This pattern applies to both situations - when a function does not return any
+dynamically allocated value and when it does. However, it needs a little
+extension for the latter case.
+
+ at subsection bp-tmpctx-1 Functions that do not return any dynamically allocated
+value
+
+If the function does not return any value created on the heap, we will just obey
+the aforementioned pattern.
+
+ at code
+int bar()
+{
+ int ret;
+ TALLOC_CTX *tmp_ctx = talloc_new(NULL);
+ if (tmp_ctx == NULL) {
+ ret = ENOMEM;
+ goto done;
+ }
+ /* allocate data on tmp_ctx or on its descendants */
+ ret = EOK;
+done:
+ talloc_free(tmp_ctx);
+ return ret;
+}
+ at endcode
+
+ at subsection bp-tmpctx-2 Functions returning dynamically allocated values
+
+If our function returns any dynamically allocated data, its first parameter
+should always be the destination talloc context. This context serves as a parent
+for the output values. But again, we will create the output values as the
+descendants of the temporary context. If everything goes well, we will change
+the parent of the output values from the temporary to the destination talloc
+context.
+
+This pattern ensures that if an error occurs (e.g. I/O error or insufficient
+amount of the memory), all allocated data is freed and no garbage appears on
+the destination context.
+
+ at code
+int struct_foo_init(TALLOC_CTX *mem_ctx, struct foo **_foo)
+{
+ int ret;
+ struct foo *foo = NULL;
+ TALLOC_CTX *tmp_ctx = talloc_new(NULL);
+ if (tmp_ctx == NULL) {
+ ret = ENOMEM;
+ goto done;
+ }
+ foo = talloc_zero(tmp_ctx, struct foo);
+ /* ... */
+ *_foo = talloc_steal(mem_ctx, foo);
+ ret = EOK;
+done:
+ talloc_free(tmp_ctx);
+ return ret;
+}
+ at endcode
+
+ at section bp-null Allocate temporary contexts on NULL
+
+As it can be seen on the previous listing, instead of allocating the temporary
+context directly on <code>mem_ctx</code>, we created a new top level context
+using <code>NULL</code> as the parameter for <code>talloc_new()</code> function.
+Take a look at the following example:
+
+ at code
+char *create_user_filter(TALLOC_CTX *mem_ctx,
+ uid_t uid, const char *username)
+{
+ char *filter = NULL;
+ char *sanitized_username = NULL;
+ /* tmp_ctx is a child of mem_ctx */
+ TALLOC_CTX *tmp_ctx = talloc_new(mem_ctx);
+ if (tmp_ctx == NULL) {
+ return NULL;
+ }
+
+ sanitized_username = sanitize_string(tmp_ctx, username);
+ if (sanitized_username == NULL) {
+ talloc_free(tmp_ctx);
+ return NULL;
+ }
+
+ filter = talloc_aprintf(tmp_ctx,"(|(uid=%llu)(uname=%s))",
+ uid, sanitized_username);
+ if (filter == NULL) {
+ return NULL; /* tmp_ctx is not freed */ (*@\label{lst:tmp-ctx-3:leak}@*)
+ }
+
+ /* filter becomes a child of mem_ctx */
+ filter = talloc_steal(mem_ctx, filter);
+ talloc_free(tmp_ctx);
+ return filter;
+}
+ at endcode
+
+We forgot to free <code>tmp_ctx</code> before the <code>return</code> statement
+in the <code>filter == NULL</code> condition. However, it is created as a child
+of <code>mem_ctx</code> context and as such it will be freed as soon as the
+<code>mem_ctx</code> is freed. Therefore, no detectable memory leak is created.
+
+On the other hand, we do not have any way to access the allocated data
+and for all we know <code>mem_ctx</code> may exist for the lifetime of our
+application. For these reasons this should be considered as a memory leak. How
+can we detect if it is unreferenced but still attached to its parent context?
+The only way is to notice the mistake in the source code.
+
+But if we create the temporary context as a top level context, it will not be
+freed and memory diagnostic tools
+(e.g. <a href="http://valgrind.org">valgrind</a>) are able to do their job.
+
+ at section bp-pool Temporary contexts and the talloc pool
+
+If we want to take the advantage of the talloc pool but also keep to the
+pattern introduced in the previous section, we are unable to do it directly. The
+best thing to do is to create a conditional build where we can decide how do we
+want to create the temporary context. For example, we can create the following
+macros:
+
+ at code
+#ifdef USE_POOL_CONTEXT
+ #define CREATE_POOL_CTX(ctx, size) talloc_pool(ctx, size)
+ #define CREATE_TMP_CTX(ctx) talloc_new(ctx)
+#else
+ #define CREATE_POOL_CTX(ctx, size) talloc_new(ctx)
+ #define CREATE_TMP_CTX(ctx) talloc_new(NULL)
+#endif
+ at endcode
+
+Now if our application is under development, we will build it with macro
+<code>USE_POOL_CONTEXT</code> undefined. This way, we can use memory diagnostic
+utilities to detect memory leaks.
+
+The release version will be compiled with the macro defined. This will enable
+pool contexts and therefore reduce the <code>malloc()</code> calls, which will
+end up in a little bit faster processing.
+
+ at code
+int struct_foo_init(TALLOC_CTX *mem_ctx, struct foo **_foo)
+{
+ int ret;
+ struct foo *foo = NULL;
+ TALLOC_CTX *tmp_ctx = CREATE_TMP_CTX(mem_ctx);
+ /* ... */
+}
+
+errno_t handle_request(TALLOC_CTX mem_ctx)
+{
+ int ret;
+ struct foo *foo = NULL;
+ TALLOC_CTX *pool_ctx = CREATE_POOL_CTX(NULL, 1024);
+ ret = struct_foo_init(mem_ctx, &foo);
+ /* ... */
+}
+ at endcode
+
+*/
diff --git a/lib/talloc/doc/tutorial_context.dox b/lib/talloc/doc/tutorial_context.dox
new file mode 100644
index 0000000..b8bfe26
--- /dev/null
+++ b/lib/talloc/doc/tutorial_context.dox
@@ -0,0 +1,198 @@
+/**
+ at page libtalloc_context Chapter 1: Talloc context
+ at section context Talloc context
+
+The talloc context is the most important part of this library and is
+responsible for every single feature of this memory allocator. It is a logical
+unit which represents a memory space managed by talloc.
+
+From the programmer's point of view, the talloc context is completely
+equivalent to a pointer that would be returned by the memory routines from the
+C standard library. This means that every context that is returned from the
+talloc library can be used directly in functions that do not use talloc
+internally. For example we can do the following:
+
+ at code
+char *str1 = strdup("I am NOT a talloc context");
+char *str2 = talloc_strdup(NULL, "I AM a talloc context");
+
+printf("%d\n", strcmp(str1, str2) == 0);
+
+free(str1);
+talloc_free(str2); /* we can not use free() on str2 */
+ at endcode
+
+This is possible because the context is internally handled as a special
+fixed-length structure called talloc chunk. Each chunk stores context metadata
+followed by the memory space requested by the programmer. When a talloc
+function returns a context (pointer), it will in fact return a pointer to the user
+space portion of the talloc chunk. If we to manipulate this context using
+talloc functions, the talloc library transforms the user-space pointer back to
+the starting address of the chunk. This is also the reason why we were unable
+to use <code>free(str2)</code> in the previous example - because
+<code>str2</code> does not point at the beginning of the allocated block of
+memory. This is illustrated on the next image:
+
+ at image html context.png
+
+The type TALLOC_CTX is defined in talloc.h to identify a talloc context in
+function parameters. However, this type is just an alias for <code>void</code>
+and exists only for semantical reasons - thus we can differentiate between
+<code>void *</code> (arbitrary data) and <code>TALLOC_CTX *</code> (talloc
+context).
+
+ at subsection metadata Context meta data
+
+Every talloc context carries several pieces of internal information along with
+the allocated memory:
+
+ - name - which is used in reports of context hierarchy and to simulate
+ a dynamic type system,
+ - size of the requested memory in bytes - this can be used to determine
+ the number of elements in arrays,
+ - attached destructor - which is executed just before the memory block is
+ about to be freed,
+ - references to the context
+ - children and parent contexts - create the hierarchical view on the
+ memory.
+
+ at section context-hierarchy Hierarchy of talloc context
+
+Every talloc context contains information about its parent and children. Talloc
+uses this information to create a hierarchical model of memory or to be more
+precise, it creates an n-ary tree where each node represents a single talloc
+context. The root node of the tree is referred to as a top level context - a
+context without any parent.
+
+This approach has several advantages:
+
+ - as a consequence of freeing a talloc context, all of its children
+ will be properly deallocated as well,
+ - the parent of a context can be changed at any time, which
+ results in moving the whole subtree under another node,
+ - it creates a more natural way of managing data structures.
+
+ at subsection Example
+
+We have a structure that stores basic information about a user - his/her name,
+identification number and groups he/she is a member of:
+
+ at code
+struct user {
+ uid_t uid;
+ char *username;
+ size_t num_groups;
+ char **groups;
+};
+ at endcode
+
+We will allocate this structure using talloc. The result will be the following
+context tree:
+
+ at image html context_tree.png
+
+ at code
+/* create new top level context */
+struct user *user = talloc(NULL, struct user);
+
+user->uid = 1000;
+user->num_groups = N;
+
+/* make user the parent of following contexts */
+user->username = talloc_strdup(user, "Test user");
+user->groups = talloc_array(user, char*, user->num_groups);
+
+for (i = 0; i < user->num_groups; i++) {
+ /* make user->groups the parent of following context */
+ user->groups[i] = talloc_asprintf(user->groups,
+ "Test group %d", i);
+}
+ at endcode
+
+This way, we have gained a lot of additional capabilities, one of which is
+very simple deallocation of the structure and all of its elements.
+
+With the C standard library we need first to iterate over the array of groups
+and free every element separately. Then we must deallocate the array that stores
+them. Next we deallocate the username and as the last step free the structure
+itself. But with talloc, the only operation we need to execute is freeing the
+structure context. Its descendants will be freed automatically.
+
+ at code
+talloc_free(user);
+ at endcode
+
+ at section keep-hierarchy Always keep the hieararchy steady!
+
+The talloc is a hierarchy memory allocator. The hierarchy nature is what makes
+the programming more error proof. It makes the memory easier to manage and to
+free. Therefore, the first thing we should have on our mind is: <strong>always
+project our data structures into the talloc context hierarchy</strong>.
+
+That means if we have a structure, we should always use it as a parent context
+for its elements. This way we will not encounter any troubles when freeing this
+structure or when changing its parent. The same rule applies for arrays.
+
+ at section creating-context Creating a talloc context
+
+Here are the most important functions that create a new talloc context.
+
+ at subsection type-safe Type-safe functions
+
+It allocates the size that is necessary for the given type and returns a new,
+properly-casted pointer. This is the preferred way to create a new context as
+we can rely on the compiler to detect type mismatches.
+
+The name of the context is automatically set to the name of the data type which
+is used to simulate a dynamic type system.
+
+ at code
+struct user *user = talloc(ctx, struct user);
+
+/* initialize to default values */
+user->uid = 0;
+user->name = NULL;
+user->num_groups = 0;
+user->groups = NULL;
+
+/* or we can achieve the same result with */
+struct user *user_zero = talloc_zero(ctx, struct user);
+ at endcode
+
+ at subsection zero-length Zero-length contexts
+
+The zero-length context is basically a context without any special semantical
+meaning. We can use it the same way as any other context. The only difference
+is that it consists only of the meta data about the context. Therefore, it is
+strictly of type <code>TALLOC_CTX*</code>. It is often used in cases where we
+want to aggregate several data structures under one parent (zero-length)
+context, such as a temporary context to contain memory needed within a single
+function that is not interesting to the caller. Allocating on a zero-length
+temporary context will make clean-up of the function simpler.
+
+ at code
+TALLOC_CTX *tmp_ctx = NULL;
+struct foo *foo = NULL;
+struct bar *bar = NULL;
+
+/* new zero-length top level context */
+tmp_ctx = talloc_new(NULL);
+if (tmp_ctx == NULL) {
+ return ENOMEM;
+}
+
+foo = talloc(tmp_ctx, struct foo);
+bar = talloc(tmp_ctx, struct bar);
+
+/* free everything at once */
+talloc_free(tmp_ctx);
+ at endcode
+
+ at subsection context-see-also See also
+
+- talloc_size()
+- talloc_named()
+- @ref talloc_array
+- @ref talloc_string
+
+*/
diff --git a/lib/talloc/doc/tutorial_debugging.dox b/lib/talloc/doc/tutorial_debugging.dox
new file mode 100644
index 0000000..aadbb0d
--- /dev/null
+++ b/lib/talloc/doc/tutorial_debugging.dox
@@ -0,0 +1,116 @@
+/**
+ at page libtalloc_debugging Chapter 6: Debugging
+
+Although talloc makes memory management significantly easier than the C standard
+library, developers are still only humans and can make mistakes. Therefore, it
+can be handy to know some tools for the inspection of talloc memory usage.
+
+ at section log-abort Talloc log and abort
+
+We have already encountered the abort function in section @ref dts.
+In that case it was used when a type mismatch was detected. However, talloc
+calls this abort function in several more situations:
+
+- when the provided pointer is not a valid talloc context,
+- when the meta data is invalid - probably due to memory corruption,
+- and when an access after free is detected.
+
+The third one is probably the most interesting. It can help us with detecting
+an attempt to double-free a context or any other manipulation with it via
+talloc functions (using it as a parent, stealing it, etc.).
+
+Before the context is freed talloc sets a flag in the meta data. This is then
+used to detect the access after free. It basically works on the assumption that
+the memory stays unchanged (at least for a while) even when it is properly
+deallocated. This will work even if the memory is filled with the value
+specified in <code>TALLOC_FREE_FILL</code> environment variable, because it
+fills only the data part and leaves the meta data intact.
+
+Apart from the abort function, talloc uses a log function to provide additional
+information to the aforementioned violations. To enable logging we shall set the
+log function with one of:
+
+- talloc_set_log_fn()
+- talloc_set_log_stderr()
+
+The following code is a sample output of accessing a context after it has been
+freed:
+
+ at code
+talloc_set_log_stderr();
+TALLOC_CTX *ctx = talloc_new(NULL);
+
+talloc_free(ctx);
+talloc_free(ctx);
+
+results in:
+talloc: access after free error - first free may be at ../src/main.c:55
+Bad talloc magic value - access after free
+ at endcode
+
+Another example is an invalid context:
+
+ at code
+talloc_set_log_stderr();
+TALLOC_CTX *ctx = talloc_new(NULL);
+char *str = strdup("not a talloc context");
+talloc_steal(ctx, str);
+
+results in:
+Bad talloc magic value - unknown value
+ at endcode
+
+ at section reports Memory usage reports
+
--
Samba Shared Repository
More information about the samba-cvs
mailing list