WLAN IDS and the bizarre world of security exploits

If you make security software (or any software, for that matter) sooner or later you will create what I technically refer to as a booboo. A security vulnerability in your software that raises the ire of your customers and make you feel foolish and sad. Not to worry, mateys, this happens to all software manufacturers. The important thing to remember here is how you handle it. Are you going to be a Pro or a shmuck? Recently, AirDefense (why no dot com?), a WLAN IDS manufacturer had just such and incident. Is this uncommon? Relatively so. Is it dire? Not really. Are you just sniping at your competitor? Kind of, but in the interest of disclosure, we had an incident a long time ago as well so, dear friends, I feel their pain.

Let's talk about what happened first. The vulnerability as explained here happens when you send a specially crafted HTTPS request, which will cause the HTTPS service on the system to crash. It appears from my quick glance as if you need to authenticate first and also be on the segment from which you can administer the system. So what is this? Granted it can bring down the sensor but actually it appears to be a "tempest in a teacup". You need to be the admin or snarf the admin login in order to cause a denial of service to one of probably many tens or hundereds of sensors. Unlikely at best.

So how was this handled? Professionally, in my humble opinion. AirDefense contacted the people who reported the exploit and directed them to a patch for it as reported here, "Solution: Update to the latest firmware version"

AirMagnet had a similar experience Last October. And we handled it the same way. Here is our official response to the problem from back then:

Re: Airmagnet management interfaces multiple vulnerabilities
AirMagnet vendor response below -

(1) The vulnerabilities are tested against an over-a-year old AirMagnet Enterprise product,
(2) Some of these vulnerabilities have been patched and fixed in AirMagnet Enterprise version 7.0.x,
(3) All vulnerabilities are now completely fixed by AirMagnet Enterprise version 7.5 build 6307 and later.
(4) AirMagnet customers can download patches from MyAirMagnet support web site (http://www.airmagnet.com/my_airmagnet/index.php)

So to summarize, there are a lot of security professionals out there who are trying to make a name for themselves and do it in an industry, like the WLAN industry, that is going places. They spend all their time looking for these exploits and I, for one, am glad they do. They keep us honest and ensure that we are doing our very best to protect our customers. Are their motives pure? Debatable but mostly. Do they sit down afterwards and talk amongst themselves about what l@m3rz those software guys are? You bet! Should I take it personally? Nah.

The Myth of the Self-Monitoring WLAN

Recently, as you all probably know by now, Duke University had a WLAN meltdown. The CIO, Tracy Futhey (Comment here) and the assistant IT director, Kevin Miller (Comment here) have put to rest the notion that the Apple iPhone caused it. Cisco has issued an advisory to the effect and Apple assisted in the effort.

I am not going to go into the details of what happened or why. Suffice it to say that mobile handhelds of all types, not just iPhones, send a lot of ARP traffic and the Cisco infrastructure was not ready for it. The quote at Network World explains that, "The advisory finally makes it clear that the iPhone simply triggered the ARP storms that were made possible by the controller vulnerabilities. Any other wireless client device, moving from one subnet to another apparently could have done the same thing."

What I will point out, however, is the problem we in the Wi-Fi community have today with the following simple delusion, "Your WLAN infrastructure as a cohesive, integrated, single-vendor solution is all anybody needs. It is self monitoring and self healing." I talk to a lot of people about which WLAN solution they are going to purchase and implement and I am always surprised by how many believe that the AP and controller vendor has all the answers. Don't get me wrong, I am a huge fan of this type of solution. Central management is critical for even medium sized organizations of 50 or more APs, much less larger ones that may a few hundred or even thousands. Manually changing the configuration of each AP is not a viable solution in these cases. The Admin needs assistance. And the story sounds so great, "Implement our solution and it will fix itself when it breaks and protect itself when security policies are breached." Who wouldn't want that?

But the truth is a little more complicated. As we have seen from previous posts, sometimes the solution doesn't behave the way your business practices need. Similarly, sometimes there are security problems within the infrastructure itself. So what to do?

This will sound like an advertisement for the company I work for and I apologize ahead of time but there is a very good reason I continue to work there. Mainly, I believe in the message.

When the Duke network went down and the Assistant IT director looked at his WLAN infrastructure dashboard, what did he see? I have not spoken with him directly but my guess would be it said, "hey man, it ain't me. Everything looks good from my end" So what did he do? he pulled out a sniffer and got to work. With packet traces in hand and assistance from Cisco and Apple he solved the problem. Did the infrastructure fix itself? Did it correctly identify the problem and solution? No. A patch is now needed to keep this from happening again.

One should not blame the infrastructure for not getting this right at the outset nor should one blame Mr. Miller. He was correctly reading what the controllers were telling him. But it shows how important it is to have a separate, 3rd party solution also available to get down to the bits and bytes or even spectrum analysis (if the problem should be something other than 802.11 protocol madness.)

There are a few great WLAN security vendors out there and they make 3rd party, best of breed solutions for monitoring the security of your WLAN (one of which recently got snatched up pennies on the dollar and will probably be rolled into another integrated, self-healing, self-monitoring role; against my better judgment.) There are an even smaller number who both monitor your security and your connectivity and performance and give you great troubleshooting tools built-in (insert shameless plug here). These should be your trusted advisor's when things go wrong. I am in no way suggesting that they would have identified the problem and cause and given a solution at Duke either (although I think they at least would have shown alerts for denial of service and strange traffic behavior.) What I am suggesting is that with them in place you now have a set of tools to assist in solving the problem. Remote packet and/or spectrum analysis. Alarm thresholds that can be set by the admin and will continue surveillance. Reports. System-to-system notifications. Graphs of speed and traffic type. Lists of who is connected to what and how. All the things you would need to get to the bottom of any problem in that invisible Luminiferous Ether.

---

Meraki AirMagnet Stats

Some folks have requested more technical details on the Meraki nodes so I am uploading some AirMagnet Laptop Analyzer images for your perusal. Let me know what you think.

(Click an image to enlarge it)

Here, for example is the AirMagnet Start screen showing the 3 nodes I have up

And here we have the Infrastructure page showing how they are viewed.

But the details that most folks have been asking for is here on the Channel Page (notice the bytes and frames. Very good data speeds for the most part. Since the beacon interval is set to 500ms I have the channel scan time set to 750ms)...

...and here on the main portion of the Infrastructure page. I also had the Spectrum Analyzer integration enabled. For this image I selected the main "root" node to analyze.

I was in the papers a few times also...

So maybe I am in a, "toot your own horn" kinda mood. (I must not get enough love at work) but I thought I would catalog some props from my past.

I have been printed in a variety of articles in the mainstream press, most of which I am very proud of. For example, I just happened to be a a h@x0r convention in Washington DC called ShmooCon on behalf of AirMagnet when Simple Nomad relased a [kinda] zero day for wifi. I was just hangin out afterwards when a reporter from the Washington Post grabbed a seat at my lonely table to discuss it. Brian Krebs is very well respected and I was happy to talk to him so we chatted about how lame it was that Microsoft kept having stuff blow up on them and this one was such a silly thing. It really blew us away. He asked for a real interview to learn how this had been effecting some of my customers - which it had - and then we went a had a few beers.

You see I was getting calls from a bunch of my customers saying that they were seeing bizarre SSIDs showing up on the Dashboard of our IDS, AirMagnet Enterprise. And to top it all off, they were all in Ad-Hoc, or peer-to-peer mode. SSIDs with names like, linksys, tmobile, hpsetup and wayport-access. My customers were blaming our software, saying we were, "sending false positives". Well, it turns out it was Microsoft's fault the whole time. Go Figure.

Brian is a great guy so I will probably grab a few beers with him this year when I go out in March, but it just goes to show you that 80% of success is just showing up (thanks Woody Allen).

Here are some other links to press on me:

Here is the Washington Post piece. And almost exactly a year later, here is Microsoft's resolution to the vulnerability.

I conducted a walk around with the New York Times (didn't get a mention but my neighborhood did).

And here are a bunch from DefCon 13 where I found a bunch of radio interference:
Wireless Week
The IEEE
EE Times
Information Week


There is also some stuff from waaaaayyyyy back with computer world and a ton of ISSA, ISACA and other speaking engagements. too old to worry about.

Hey, I was on TV!

So a long time ago, I was asked by my V.P. of Marketing at the time, Rich Mironov (One of the best Marketing guys I know, BTW), to assist our PR firm with a show they were putting together. Tactical to Practical on the History Channel. It is a show where in the first half hour they show the military doing something really cool and then, for the second half hour they show you how you, The average American, can do something similar with stuff you can pick up from Frys.

It was a fun shoot. I brought along a friend of mine, Jon Erikson, who wrote a fabulous book called the Art of Exploitation. One of the most well received books on security exploits I know of. He and I were to conduct an actual hack over wireless at a hotspot in downtown San Jose for the cameras.

Jon had a few prepared 'splots he wanted to run. One was a MitM attack with stream injection. I would search for, oh, lets say, "shrimp" at Google and he would substitute, say, "giant" for "shrimp" so all the returns from Google were about really big things. Kinda funny but a hard concept to convey in 15 minutes to a TV audience.

The other idea was pretty simple (read:LAME), I would log into my mail account and he would snarf my password and go read my mail. It came off OK and they kept it as the final for the show. It was fun to do and we got a ton of inquiries. I actually get about 15 minutes of airtime. So there is my Andy Warhol quote for the day. Here is the link: Bruce_on_TV