Security for a day

Like any normal day in the office, at some point I’ll get blamed for something not working.

More specifically, one of the security systems I manage gets blamed, but I cop the flack for it. Moaning at an inanimate firewall doesn’t elicit the sheer pleasure of ranting a human being, or me, it appears.

Anyway, over the weekend a financial web site had stopped working and they wanted it to work. It was throwing up authentication errors in a Java Applet screen they never seen before. The Web company’s helpdesk said nothing had changed, so it had to be our problem, or my problem, as normal. Since nothing changed on the network, client machines or firewall rule sets (gotta love change management) and I had a screen shot of the error with times of the problems, I called up the ISA logs, and filtered on the url of the site.

The screen shots filled me with apprehension, as the web site had a big ugly Java error and the pages were .jhtml and it was all running over SSL. After a chat with the staff member, he said the site looked like it had an update.

I’ve had problems before with Java programmers doing dubious things over http and the ISA correctly dropping the traffic, so wasn’t looking forward to getting in to a fight with a big financial web company over coding.

The ISA filter displayed a whole heap of these errors when connecting to the site:

Status: 995 The I/O operation has been aborted because of either a thread exit or an application request

Now this doesn’t tell you much, so after a quick bit of browsing the web I found a reply from Jim Harrison to someone with similar issues

“This is expected even for normal termination of SSL Tunnel traffic.

ISA can’t follow the HTTP conversation within the SSL session and so the session closure is always a surprise.

It does not indicate an error in ISA."

I trust Jim’s advice implicitly, and was sure it was the lovely web company’s fault, but his reply didn’t help nail what was wrong and SSL won’t let me analyse the traffic.

Help came from an unexpected source when the staffer mention he could access the web site at home, after taking a very long time to load to first time.

Did anything else happen while you accessed the site, like some piece of software update? I enquired.

It appears that his machine downloaded the latest version of Java first. Hmm, to the test lab machines!

I fired up a test machine, broke company standard build policy, ripped out the current package version of Java and installed the latest and greatest straight from the web site. This promptly broke the machine. I grabbed the next test machine and attempted to update Java. That broke it too. The third machine, a totally non-standard machine, installed Java without issues and could “magically” accessed the site. No need to touch the ISA rules.

Grinning like a Cheshire cat I promptly handed over the mess of updating everyones Java client to the packaging team with a couple of notes on how it had destroyed two perfectly good machines for no apparent reason.

Take away: if you see status message 995 being logged on ISA, a web app stops working and the site is Java based, then check the Java client and ask what version you should be using to access the site first.

Put money on it that the third party will say “Install at minimum version X”, which won’t be the version you’re running.