[Report]Automated testing of Samba with Windows Protocol Test Suites

Jeremy Allison jra at samba.org
Tue Jan 2 23:34:39 UTC 2018


On Wed, Jan 03, 2018 at 10:05:43AM +1300, joeg--- via samba-technical wrote:
> Hi everyone,
> 
> Happy new year!
> 
> I've done some job to use Ansible and OpenStack Cloud to automate the
> testing of Samba with Windows Protocol Test Suites
> <https://github.com/Microsoft/WindowsProtocolTestSuites>.
> 
> So far, I've only tried KDC category, but the other ones are easy to
> support.
> 
> This is the current KDC test result:
> 
> total: 85
> 
> inconclusive: 3
> 
> passed: 32
> 
> failed: 50
> 
> And this is the 50 failed tests:
> 
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CanonicalizeSpnInReferralTgt
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.UserInAnotherDomain
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.ComputerAccount_DES_PreAuthentication_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.DesDowngradeProtection
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.HwAuthentFlag
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.Krbtgt_DES_PreAuthentication_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.OkAsDelegateFlag
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccount_DES_PreAuthentication_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountDisabled
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountExpired
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountLocked
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangePast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangeZero
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserDelegationNotAllowed
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserPreAuthenticationNotRequired
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithDES
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadMatch
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadOptions
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadProtocolVersionNum
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorCannotPostdate
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorFieldTooLong
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorGeneric
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorMsgType
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorRepeat
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTicketNeverValid
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktExpired
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktNotYetValid
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastArmorInAuthenticator
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastUsedInAuthenticator
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastKrb_Error
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastOptions_HideClientNames
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedAsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedTgsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.PaFxFastAdvertise
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.StrengthenKey
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.UnsupportedFastArmorType
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Succeed
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseExplicitFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.APDS_KERBEROS_PAC_VALIDATION
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.ServerSignature
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_AdFxFastUsedInAuthenticator
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_FastKrb_Error
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseExplicitFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedAsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedTgsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_StrengthenKey
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UnsupportedFastArmorType
> 
> 
> It's now quite easy to set up the testing network and run it, and we are
> going to fix the failed ones gradually in future.

Great work Joe - thanks for the report !

Jeremy.



More information about the samba-technical mailing list