[jcifs] Re: Re: Loadbalancing with NTLMServlet/Filter

Eric Glass eric.glass at gmail.com
Tue Dec 6 06:29:44 GMT 2005


This is an inherent issue if the HTTP session is used prior to
completing the authentication process; the reason being that cookies
are used to maintain the session, and can result in basically a race
condition as noted below.

We had some similar issues way back in 0.7.0b5 around when the filter
was first introduced; as maintaining state based on the physical
socket isn't directly available via the servlet API, the solution was
to do the authentication handshake statelessly.  This precludes the
tracking that is needed to support the new loadbalancing functionality
however (that appears to be what the HTTP session storage is used
for).

The servlet 2.4 spec has an HttpServletRequest.getLocalPort() call;
while it doesn't provide access to the actual socket, it might be
sufficient to match transactions to connections appropriately without
relying on the HTTP session.


Eric

>
> There's even more to this issue than has been described.  If you have a
> multi-process (not multi-threaded, but truly multi-process),
> load-balanced app server, try creating a simple .html file like this:
>
> <html>
> <head>
>         <title>Test Page</title>
> </head>
> <body>
>         <img
> src="http://load.balanced.server.com/content/images/image1.gif"/>
>         <img
> src="http://load.balanced.server.com/content/images/image2.jpg"/>
>         <img
> src="http://load.balanced.server.com/content/images/image3.jpg"/>
> </body>
> </html>
>
> Save this file locally on your computer and then load it into your
> browser using a file: URL.  What ends up happening here is that the
> browser makes multiple simultaneous requests to the app servers.  These
> requests initially get routed per the load balancer policy to different
> app servers (random, round-robin, etc).  The app servers all
> independently respond with HTTP 401 errors requesting NTLM
> authentication, and each app server supplies a different JSESSIONID
> cookie.
>
> At some point during (or after) the authentication process, the browser
> will choose to "standardize" on a single JSESSIONID cookie for the
> server.  Depending on when the browser chooses a single JSESSIONID
> cookie, there are 2 different cases to describe.
>
> CASE A:
> The browser _sometimes_ selects a single JSESSIONID cookie to
> "standardize" on and use to submit the NTLM type 1 requests.  If it
> selects a single JSESSIONID cookie to use for the NTLM type 1 requests,
> then a problem arises in that once the first NTLM type 3 request is
> processed by the app server, the NtlmHttpChal attribute is removed from
> the session (ssn.removeAttribute("NtlmHttpChal") in NtlmHttpFilter).
> The next NTLM type 3 requests will be required to obtain a new NTLM
> challenge, and _may_ not get the same transport/challenge that was
> originally used for the NTLM type 2 response sent by the app server.
>
> CASE B:
> We have also seen, however, that the browser could elect to use the
> different distinct JSESSIONID cookies to submit the NTLM type 1
> requests, and not "standardize" on a JSESSIONID cookie until it submits
> the NTLM type 3 requests.  In this case, you end up with multiple
> independent app servers creating SMB transports and generating NTLM
> challenges and saving those challenges in different sessions on the
> different app servers.  When the browser chooses to "standardize" on a
> single JSESSIONID cookie, the NTLM type 3 requests then all get
> submitted to a single app server.  This results in NTLM type 3 requests
> with the NTLM responses being sent to an app server that will be unable
> to properly verify the responses, because the app server that the NTLM
> type 3 requests are submitted to does not have the corresponding SMB
> transport with the proper transport encryption key (challenge) to verify
> the NTLM responses.
>
> We have seen with the sample .html file above, that we usually get a mix
> of case A and case B as described above.  There is no predictable
> pattern to how/when the browser will choose to "standardize" on a
> JSESSIONID cookie.
>
> The only way that I have thought of to completely resolve this issue is
> to create a cluster-wide map to save the NTLM challenges used when the
> NTLM type 2 response is generated.  This map would require that the key
> be completely independent of the session ID, as the browser can (and
> does) choose to change sessions during the middle of the NTLM
> negotiation process.  The map key could be based on remote IP address,
> request URI and query string (but even this poses a potential problem if
> you have multiple users on a shared machine who happen to require NTLM
> authentication simultaneously for the same URI).  (This is highly
> unlikely, but in my experience with concurrent programming, if you leave
> the window open, bugs will fly in.)
>
> This is why Microsoft stipulates that connection keep-alives MUST be
> used for NTLM authentication to succeed.  If connection keep-alives are
> used, then you are guaranteed to return to the same server to continue
> the NTLM negotiation process.  I'm not sure how this could be used in a
> multi-tiered environment where there is a web server tier (Apache/IBM
> HttpServer) in front of the app server tier where the jcifs code
> actually executes.
>
>
>
>
> >From: Smyth, Jim <Jim.Smyth <at> broadvision.com>
> >Subject: Re: Re: Loadbalancing with NTLMServlet/Filter
> <http://news.gmane.org/find-root.php?message_id=%3cAA8C151FBFFAF045BC28C
> 074A4483FBB03AE43%40uk%2dmsg%2d01.broadvision.com%3e>
> >Newsgroups: gmane.network.samba.java
> <http://news.gmane.org/gmane.network.samba.java>
> >Date: 2005-11-23 01:10:51 GMT (1 week, 5 days, 15 hours and 13 minutes
> ago)
> >Hi,
> >
> >I experienced an issue with using jcifs on applications running across
> multiple app servers which I found
> >reported some time ago (see below).  I believe the end solution may not
> be correct.  To recap (and fully
> >explain) the issue, the 401 Authenticate response created by JCIFS
> filter tells the browser to resend the
> >same request with NTLM credentials.  When a user first hits an
> application a session is automatically
> >created for that user.  Normally all subsequent requests by that user
> will use a session identifier (via
> >URL-rewriting or a cookie) to maintain the state of the application.
> All quite simple so far.
> >
> >An issue arises when an application is running on multiple processes
> rather than just multiple threads,
> >because the 401 which makes the browser resend a request may end up at
> a different running process which
> >will invalidate the NTLM negociation mechanism.  Running an application
> over a number of machines is one
> >way in which this situation arises, but the big boys from weblogic and
> WAS also recognize that scaling
> >appservers on threads only goes so far, such that multi-process
> appservers on single (big) boxes also exist.
> >
> >Anyway, Im sure you know all that.  The point is that
> >
> >a)  the session is implicitly created at the outset, therefore, calling
> req.getSession() as originally
> >suggested will not have any effect
> >b)  to avoid this issue the servlet filter should start with something
> like the following so that negociate
> >will never commence until we can be sure the authorization will be sent
> back to the correct appserver process:
> >
> >                       if
> (Config.getProperty("jcifs.util.multiProcess")!=null &&
> >Config.getProperty("jcifs.util.multiProcess").equals("true")) {
> //check if defined as
> >multi-process appserver
> >                               //this code should only ever get
> executed once per user per session
> >                               if
> (session.getAttribute("NTLMNegociationInitiated")==null) {
> >
> session.setAttribute("NTLMNegociationInitiated","true");
> >
> >                                       //should be a block of code here
> responsible for setting a cookie if cookie based session
> >management is in use
> >                                       //but I havent bothered doing
> that yet
> >                                       //...
> >
> >                                       //the following code will take
> care of urlrewriting if required
> >                                       PrintWriter out = null;
> >                                               out = resp.getWriter();
> >                                               out.println("<html>");
> >                                               out.println("<head>");
> >
> out.println("<title>Resending request with Session id</title>");
> >                                               out.println("<META
> Http-Equiv=\"Cache-Control\" Content=\"no-cache\">");
> >                                               out.println("<META
> Http-Equiv=\"Pragma\" Content=\"no-cache\">");
> >                                               out.println("<META
> Http-Equiv=\"Expires\" Content=\"0\">");
> >                                               out.println("<meta
> http-equiv=\"refresh\" content=\"0; URL=" +
> >resp.encodeURL(req.getRequestURL().toString()) + "\">");
> >                                               out.println("</head>");
> >                                       out.println("</html>");
> >                                       return;
> >                               } //end if
> >                       } //end of check for multi-process appserver
> >
> >Anyways - thats the way I would (will) do it.  Hope its of
> use/interest.
> >
> >rgds
> >jim
> >
> >From: Eric <eglass1 <at> comcast.net>
> >Subject: Re: Re: Loadbalancing with NTLMServlet/Filter
> <http://news.gmane.org/find-root.php?message_id=%3c405B99A2.2090403%40co
> mcast.net%3e>
> >Date: 2004-03-20 01:08:50 GMT (1 year, 35 weeks, 2 days, 17 hours and
> 16 minutes ago)
> >
> >He's load balancing the servlet containers (application servers); he's
> >got a single web server with multiple WebSphere containers on the back
> >end.  The connector on the web server uses the session ID to determine
> >which application server the request gets routed to.  We don't
> currently
> >create a session until the NTLM handshake has completed (when we store
> >the username etc. in the session).  So what he's seeing is server A
> >generates the challenge and sends the Type 2 message, but server B ends
>
> >up getting the Type 3 message (the HTTP keepalive is between the client
>
> >and the web server, but not between the web server and a particular
> >application server).
> >
> >Creating the session at the outset would prevent this (and is probably
> >an appropriate measure).
> >
> >Eric
> >
> >Michael B Allen wrote:
> >> I don't see what this has to do with load balancing. Please send your
> >> precise suggested fix to the jcifs mailing list.
> >>
> >> Sylwester Lachiewicz said:
> >>
> >>>Hi,
> >>>
> >>>With current NtlmServletFilter it's not possible to create
> application
> >>>with loadbalancing
> >>>Our http server  (IBM HTTP with WebSphere plugin) use session cookie
> to
> >>>route requests to one of our Application Server. Becouse full NTLM
> Auth
> >>>requires 3 resuests/responses it's possible that one request will be
> >>>received by AppServer1 and second with AppServer2. This creates auth
> >>>failure.
> >>>
> >>>To fix this, in service method, i add line request.getSession() so
> session
> >>>will be created and JSESSIONID returned to browser and available to
> next
> >>>requests.
> >>>It's possible to add this to NtlmServlet and NtlmFilter?
> >>>
> >>>S.
> >>>
> >>
> >>
> >>
> >
> >> Jim Smyth
> >> BVGS
> >>
> >> Mobile +44 (0) 7720 352 014
> >>
> >> BroadVision
> >> Energizing e-Business
> >
> >> This message is intended only for the use of the Addressee and may
> contain information that is PRIVILEGED
> >and CONFIDENTIAL.  If you are not the intended recipient, dissemination
> of this communication is
> >prohibited.  If you have received this communication in error, please
> erase all copies of the message and
> >its attachments and notify us immediately.
>


More information about the jcifs mailing list