This project has moved. For the latest updates, please go here.

Microsoft.Owin.Security.Cookies NullReferenceException

Sep 4, 2014 at 11:18 PM
Edited Sep 4, 2014 at 11:18 PM
Ever since I moved up to the 3.0.0 release of the Owin packages, I've been seeing the following error in my ELMAH logs:
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Owin.Security.Cookies.CookieAuthenticationProvider.Exception(CookieExceptionContext context)
   at Microsoft.Owin.Security.Cookies.CookieAuthenticationHandler.<ApplyResponseGrantAsync>d__f.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Security.Infrastructure.AuthenticationHandler.<ApplyResponseCoreAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Security.Infrastructure.AuthenticationHandler.<TeardownAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.<DoFinalWork>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
So far I've had no luck reproducing this locally, and only seem to be seeing it in production.

Before upgrading to the 3.0.0 packages, I was seeing the error mentioned here:

https://katanaproject.codeplex.com/discussions/540202

but I'm now seeing this new error instead. Perhaps they are the same problem?
Sep 5, 2014 at 4:54 AM
I've spent a little bit of time taking a look at the code mentioned in the stack trace, and I think the NullReferenceException is coming from the new Exception() method and OnException Action added to CookieAuthenticationProvider in commit d36b4f64. All of the other Actions in that class are initialized in the constructor, but that one is not.

Now, it's always possible that is being set somewhere else in the code and I missed it, but it seems like it's being left null, which is causing the error I'm seeing above.

Looking at how Exception() and the CookieExceptionContext are being used, it does look like the NullReferenceException is just masking another problem, likely the one referenced in the other discussion I linked to above.
Coordinator
Sep 5, 2014 at 5:14 PM
I think you're right. I've opened a new issue for this: https://katanaproject.codeplex.com/workitem/346

Luckily the issue is easy to workaround by setting the OnException callback.
Sep 8, 2014 at 7:42 PM
Setting the OnException callback is a successful workaround, and like I thought, I'm once again seeing the "Server cannot append header after HTTP headers have been sent." errors from https://katanaproject.codeplex.com/discussions/540202
Sep 15, 2014 at 6:24 AM
Edited Sep 15, 2014 at 6:25 AM
Thanks for investigating the problem. Is there an ETA for the fix? This error is a show stopper for us since it is interrupting execution and we are unsure how to properly set the the OnException callback..
Coordinator
Sep 15, 2014 at 2:11 PM
It has been fixed in the nightly builds (https://www.myget.org/f/aspnetwebstacknightly/). No ETA yet for the next release.
Sep 16, 2014 at 2:09 AM
If you want to use the workaround mentioned above to hold you over until they release a version with the fix, you need to set the OnException property of the CookieAuthenticationProvider class to a non-null value.

Somewhere in your code, probably in a Startup class, you'll have a call to app.UseCookieAuthentication(), passing in a CookieAuthenticationOptions instance. That instance will have a Provider property.

So you want to do something like:
var provider = new CookieAuthenticationProvider();
provider.OnException = context => { };

app.UseCookieAuthentication(new CookieAuthenticationOptions
{
    Provider = provider
    //and all the other properties that need to be set will be here
});
Hope that helps!
Sep 22, 2014 at 12:22 AM
Thanks bording for the workaround code. I confirmed this works for my project as well.
Sep 22, 2014 at 3:58 PM
Excellent! I added
OnException = context => { }
in the app.UseCookieAuthentication() code that MVC made for me, and can now work again.

Thanks!
Nov 29, 2014 at 5:28 AM
Enabling this fix did not fix the problem for me on the Chrome for the iPad (iOS 8). Now I am getting the following exception:
System.FormatException: The chunked cookie is incomplete. Only 0 of the expected 2 chunks were found, totaling 0 characters. A client size limit may have been exceeded. 

Stack trace 
at Microsoft.Owin.Infrastructure.ChunkingCookieManager.GetRequestCookie(IOwinContext context, String key)
at Microsoft.Owin.Security.Cookies.CookieAuthenticationHandler.<AuthenticateCoreAsync>d__2.MoveNext()
I'm not seeing this error on any other browser, but I think a variant of this issue is also happening on Chrome for MacOS as well (as reported by one of our users).

Any thoughts on how I can fix this? The Kentor.OwinCookieSaver NuGet package is not fixing it either.

Thanks!
Coordinator
Nov 29, 2014 at 4:16 PM
advancedrei, that sounds like a client specific size issue. Browsers all have different per cookie and total size limits. The ChunkingCookieManager feature was done to help browsers that limit the size of individual cookies (4kb). However, a few browsers have very low total size limits as well (Safari, 4kb). To support them you need to provide an IAuthenticationSessionStore instead. See
https://katanaproject.codeplex.com/SourceControl/latest#src/Microsoft.Owin.Security.Cookies/CookieAuthenticationOptions.cs
https://katanaproject.codeplex.com/SourceControl/latest#tests/Katana.Sandbox.WebServer/InMemoryAuthSessionStore.cs

Or create smaller cookies...
Nov 29, 2014 at 4:55 PM
Maybe, but I did not have this problem with Katana 2.0. I upgrade to 3.0, and suddenly my cookie sizes are a problem. I'm using Auth0 + Claims for authentication, and AFAIK I can't control the size of whatever they send me.

It would seem like, if Katana detects that you're using a browser that has cookie size limits, it should use the In-Memory store for users of that browser only. That way the extra memory is only taken up by the clients that actually need it. Doesn't seem wise from a memory-management standpoint to make it an all or nothing proposition.
Coordinator
Nov 29, 2014 at 5:58 PM
The InMemory storage is for demonstration purposes. Actual implementations are more likely to use a database or other back-end store.

We opted not to attempt browser detection because there are such a wide variety of clients with varying capabilities and the list changes very often.
Nov 30, 2014 at 11:48 PM
Edited Nov 30, 2014 at 11:49 PM
Sorry, I replied with the wrong LiveID. My other username is AdvancedREI.

So I understand what you are saying, and I agree that the message would indicate the problem you are saying. But you said that Safari has a maximum total cookie size of 4kb, and the authentication cookies are working just fine in Safari. In fact, they worked just fine on the iPad in Chrome until this Katana 3.0 update. Which would lead me to believe that this is just a symptom of the actual problem, and that the actual problem lies somewhere else. The problem could be in Chrome itself, not Katana, but I think the problem probably lies in how the cookie is transmitted back to the browser after my Controller processes the Auth0 authentication response, and when the browser requests the new page after redirection and transmits said authentication cookie back to the server.
Coordinator
Dec 1, 2014 at 1:31 AM
Well, the best way to debug it is to capture the cookies for v2 and v3 in your scenario and compare them.

Note you can effectively disable the new cookie chunking feature by raising the threshold on the ChunkingCookieManager.
Dec 1, 2014 at 5:04 AM
Using Fiddler to decrypt the traffic between Chrome and Safari on the same iPad, both browsers say that .AspNet.ApplicationCookie is in 2 chunks. Chrome is missing the C1 cookie, where Safari is not. In addition, both browsers also report an .AspNet.ExternalCookie with two chunks. But Chrome has the C1 chunk where Safari does not. However, the missing .AspNet.ExternalCookieC1 does not cause the page to error out, whereas a missing .AspNet.ApplicationCookieC1 does.

I'm following the directions on Step 4 of this page. The question is, are both cookies actually needed? And why are only 3 of the 4 chunks making it to any browser on the iPad?

Thanks!

-Robert
Coordinator
Dec 1, 2014 at 7:56 PM
That's a little hard to follow. Can you post all of the Set-Cookie headers for each scenario (with the values abbreviated)?
Dec 2, 2014 at 3:25 PM
Cookie results on the iPad:

Chrome - FAIL
.AspNet.ApplicationCookie: chunks:2
.AspNet.ApplicationCookieC2: "blahblah..."
.AspNet.ExternalCookie: chunks:2
.AspNet.ExternalCookieC1: "blahblah..."
.AspNet.ExternalCookieC2: "blahblah..."

Safari - SUCCESS
.AspNet.ApplicationCookie: chunks:2
.AspNet.ApplicationCookieC1: "blahblah..."
.AspNet.ApplicationCookieC2: "blahblah..."
.AspNet.ExternalCookie: chunks:2
.AspNet.ExternalCookieC2: "blahblah..."

Both browsers are missing a cookie chunk. The one Chrome is missing causes the error I first mentioned, which is obfuscated by the error at the beginning of this thread. The one Safari is missing does not affect authentication. This is a new issue to Katana 3.0 and was not in Katana 2.0.

HTH
Coordinator
Dec 2, 2014 at 3:42 PM
I think the external cookie is passive by default, so you'd only get the error message if you did Authenticate("External"). I'd guess Safari is OK because ExternalCookieC1 was successfully transmitted and consumed, but bumped from memory on a subsequent request by the new ApplicationCookieC1. (That said, I thought the template was supposed to delete the external cookie after a successful login). Chrome may just have a different expiration policy.

What does this test show are your limits on each browser? http://www.ruslog.com/tools/cookies.html

The cookies are slightly larger in v3 because we put a little more information in them. The chunking feature is also new in v3. You can disable chunking via the cookie manager option, but I think you would hit your per-cookie size limits in the browser.
Dec 2, 2014 at 3:57 PM
Edited Dec 2, 2014 at 3:57 PM
Chrome 39 for iPad - Cookie Test:
Max Cookie Count per Domain: 1000
Max Cookie Size per Cookie: 4097 bytes
Max Cookie Size per Domain: 8180 bytes

Safari for iPad (iOS 8.1.1) - Cookie Test:
Max Cookie Count per Domain: 1000
Max Cookie Size per Cookie: 4097 bytes
Max Cookie Size per Domain: 8180 bytes

As you can see, each browser is identical.

I can disable the ChunkingCookieManager, but isn't that just a band-aid? It won't solve the actual code problem, correct? So wouldn't it be prudent to keep it enabled and help you guys fix the actual problem? Because from my standpoint, it seems like the ChunkingCookieManager is broken and needs more work ASAP. I can't be the only person running into this issue.
Coordinator
Dec 2, 2014 at 4:44 PM
Disabling the chunking cookie manager would have just been a debugging step. You are the only one to report this issue so far, but we do appreciate the effort you're putting into it.

Those size limits are about what I expected. ChunkingCookieManager defaults to 4090 char chunks, so you're probably within the per-cookie limits there. That said, between the ApplicationCookie and the ExternalCookie you could easily be exceeding the per-domain limit where the client starts discarding cookies. If you capture the actual Set-Cookie headers you'll see if the server is sending them correctly. If so then this is client-side issue and your best solution is to use the auth session store instead.

It might also be worth tracking down why the external cookie isn't being deleted when the application cookie is issued. The Set-Cookie headers should help show that too.
Dec 2, 2014 at 10:35 PM
OK, so I'm having a difficult time getting Telerik's Fiddler working with Chrome39 on the iPad. But I'm able to get it working with Safari, so I've produced those steps here.

For processing the login response from a 3rd party website, in this case a Microsoft Account, the OAuth request redirects from Auth0's servers to my Azure Website. The first request goes to the Auth0 OWIN Middleware on my site. It has the following response headers:

Set-Cookie: .AspNet.ExternalCookie=chunks:2; path=/; secure; HttpOnly
Set-Cookie: .AspNet.ExternalCookieC1=somestuff...; path=/; secure; HttpOnly
Set-Cookie: .AspNet.ExternalCookieC2=morestuff...; path=/; secure; HttpOnly

Then, that request redirects to another controller for additional processing. That request has the new cookies from the prior request in the request headers. Now, there is a different set of cookies in the response. They are:

Set-Cookie: .AspNet.ApplicationCookie=chunks:2; path=/; secure; HttpOnly
Set-Cookie: .AspNet.ApplicationCookieC1=somestuff...; path=/; secure; HttpOnly
Set-Cookie: .AspNet.ApplicationCookieC2=morestuff...; path=/; secure; HttpOnly

At this point, I should have 6 cookies in the next request. However, it only has 5. Those cookies are:

.AspNet.ApplicationCookie=chunks:2
.AspNet.ApplicationCookieC1=somestuff...
.AspNet.ApplicationCookieC2=morestuff...
.AspNet.ExternalCookie=chunks:2
.AspNet.ExternalCookieC2=morestuff...

Something in the authentication pipeline appears to be clearing the wrong set of cookies (you mentioned the External one should be deleted after the pipeline is finished processing, etc). I can attempt to manually delete the ExternalCookie after the middlware redirects to my controller and see if that helps.
Coordinator
Dec 2, 2014 at 11:14 PM
The server can only delete a cookie stored on the client by doing something like this:
Set-Cookie: CookieName= Expires=(in the past)

If you don't see any of those in the response then it's the client deleting its own cookie, most likely because it has hit it's per-domain limit.
Dec 3, 2014 at 12:08 AM
Yes, I believe (as you originally suggested) that I can confirm hitting the per-domain limit is what is happening. Because ApplicationCookieC1 and ExternalCookieC1 are both huge.

I can also confirm that manually setting the 3 ExternalCookies to expire yesterday solves the problem. More specifically:
            if (Request.Cookies[".AspNet.ExternalCookie"] != null)
            {
                Response.Cookies[".AspNet.ExternalCookie"].Expires = DateTime.Now.AddDays(-1);
            }
            if (Request.Cookies[".AspNet.ExternalCookieC1"] != null)
            {
                Response.Cookies[".AspNet.ExternalCookieC1"].Expires = DateTime.Now.AddDays(-1);
            }
            if (Request.Cookies[".AspNet.ExternalCookieC2"] != null)
            {
                Response.Cookies[".AspNet.ExternalCookieC2"].Expires = DateTime.Now.AddDays(-1);
            }
Now that we've confirmed the problem, isn't this something that calling IAuthenticationManager.SignIn() should handle on its own? Or, AuthenticationManager.SignOut(DefaultAuthenticationTypes.ExternalCookie)? Seems like the core Authentication middleware should be doing this, and not expecting the developer to know that this needs to be done. Also, since it is not obvious how to get the current instance of ChunkingCookieManager, the DeleteCookie method should probably be static.

Thanks for your help :)

-Robert