pseudobacklink / one_way_link: corruption ahead

Mon Oct 10 15:46:46 MDT 2011

Hi Tridge,

I tried your patches today, it seems to help quite a lot but there is 
still some corner cases, the main one seems to me the fact that the link 
is not reported as removed (RMD_FLAGS=0) when pointing to removed objects.

One good example is that after removing a windows DC, the following search:
./bin/ldbsearch -H ~/workspace/samba/home.matws.net/private/sam.ldb -k 1 
--cross-ncs '(fromserver=*)'
didn't return any object which is what I expect but I have this in the 
logs even after restarting samba:
Failed to connect host 172.16.100.27 on port 135 - 
NT_STATUS_HOST_UNREACHABLE
Failed to connect host 172.16.100.27 
(241a5b86-38d5-4ec9-9106-645bf55c98e7._msdcs.home.matws.net) on port 135 
- NT_STATUS_HOST_UNREACHABLE.

Where 172.16.100.27 was the address of the Windows DC.

Matthieu.
  On 07/10/2011 04:40, tridge at samba.org wrote:
> Hi Matthieu,
>
>   >  In order to reproduce them: just create a site and put it in the
>   >  DEFAULTIPSITELINK, then remove the site.
>   >  In theory the site should be removed from the siteList attribute, but
>   >  it's not !
>
> thanks for that example. Andrew and I spent some time on it today and
> we think we have it fixed up.
>
>   >  I found first a bug, attribute that have only forward link were not
>   >  covered correctly by your patches, I made this fix: http://bit.ly/oJnYo0.
>   >  That was an easy one.
>
> it turns out that while the fix was easy, it wasn't actually correct,
> although it is harmless :-)
>
> we had a bug in our interpretation of the isDeleted and isRecycled
> attributes for the different cases of when the recycled bin is or is
> not enabled. When the recycled bin is not enabled then isRecycled
> should be ignored (the value may be TRUE or not present, but it is
> never acted on). When the recycle bin is enabled we need to look at
> isRecycled.
>
> We've now fixed the show_deleted module to correctly honor the recycle
> bin setting, and this now means that the show_recycled control will
> always show all objects, regardless of what mode we are in. The
> show_deleted control will not show recycled objects if the recycle bin
> feature is enabled.
>
>   >  But if you look at the attribute, you will see an attribute that is
>   >  pointing to a deleted object
>
> minor correction - its pointing at a tombstoned object, not a deleted
> object. You can only actually have 'deleted objects' when the recycle
> bin is enabled.
>
>   >  in most cases you expect in fact the
>   >  object not to be here, your implementation of backlink handle well the
>   >  renaming but not so well the deleting.
>   >  I think it should be quite easy to mask the attribute if the pointed dn
>   >  is the DN of a deleted object.
>
> yes, and that is what we've added. It turns out we need that search
> anyway, as we need it to honor the show_deactivated_link control. See
> the new logic in the fix_one_way_link() function.
>
> We also had a simple bug in the schema code that sets up the
> one_way_link boolean on attributes. We were not setting it true for
> the sitelist attribute.
>
>   >  But then what would happen when replicating, I think a search with the
>   >  --reveal show it quite clearly:
>   >
>   >  sudo ./bin/ldbsearch -H ~/workspace/samba/home.matws.net/private/sam.ldb
>   >  -k 1 --cross-ncs '(sitelist=*)' sitelist  --reveal --extended-dn
>   >  # record 1
>   >  dn:
>   >  <GUID=f872b9fc-cedc-4949-bd51-fc8ecfb9d211>;CN=DEFAULTIPSITELINK,CN=IP,CN=Inter-Site
>   >  Transports,CN=Sites,CN=Configuration,DC=home,DC=matws,DC=net
>   >  siteList:
>   >  <GUID=c9b85202-4f5a-43cb-ac7b-a6f6cd9a8e30>;<RMD_ADDTIME=12962226990
>   >    0000000>;<RMD_CHANGETIME=129622269900000000>;<RMD_FLAGS=0>;<RMD_INVOCID=00b57
>   >    015-57c4-4baf-828a-c93c12ac7e3a>;<RMD_LOCAL_USN=1983>;<RMD_ORIGINATING_USN=19
>   >    83>;<RMD_VERSION=0>;CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=h
>   >    ome,DC=matws,DC=net
>   >  siteList:
>   >  <GUID=c75d75fc-2521-475f-ade4-5c233a286338>;<RMD_ADDTIME=12962304242
>   >    0000000>;<RMD_CHANGETIME=129623042420000000>;<RMD_FLAGS=0>;<RMD_INVOCID=00b57
>   >    015-57c4-4baf-828a-c93c12ac7e3a>;<RMD_LOCAL_USN=8531>;<RMD_ORIGINATING_USN=85
>   >    31>;<RMD_VERSION=0>;CN=test\0ADEL:c75d75fc-2521-475f-ade4-5c233a286338,CN=Sit
>   >    es,CN=Configuration,DC=home,DC=matws,DC=net
>   >
>   >  The attribute is not removed, if we are running a samba only domain it's
>   >  mostly ok but in mixed domain it must be fun !
>
> We think this is OK now. If the link is to a tombstoned or recycled
> object then the link is not displayed when queried, and this also
> applies to DRS replication searches.
>
>   >  Please tell me how you think this could be solved, the best I can see is
>   >  a kind of garbage collector.
>
> yes, we will eventually need a garbage collector, much like windows
> has the infrastructure master for cleaning up phantom objects. When we
> changed extended_dn_out we added a check for the reveal control to
> allow us to add this garbage collection in the future (the garbage
> collection would need to search for one way links with reveal enabled,
> and remove the dead ones).
>
> Meanwhile I think our external behaviour is now much better than it
> was. There are probably still cases we get wrong, but its closer than
> it was :-)
>
> Cheers, Tridge
*

-- 
Matthieu Patou
Samba Team
http://samba.org