Take away the Tools

Indi303 recently had a post on twitter

Dear pentester: Throw away metasploit.... are u still a hacker? If you make excuses about why u are,but need it.. you aren't

It seems like a lot of people did not understand what he was saying, which rather proves the point I think.  He is not saying that Pen Tester should not use Metasploit, or that tools are bad. What he is saying here is that knowing how to use tools does not make you a good pentester. It makes you a script kiddie. We have been interviewing candidates for two new PenTest positions at my work, and I can tell you I feel this keenly.

During our in-person panel interview we ask a long series of questions designed to gauge depth. We ask a number of basic questions. These first sets of questions we are jsut looking for typical responses. These questions can range from simple things like: "how does traceroute actually work" or "What is ring0" to more complex questions like "How do you exploit Blind SQL Injection on Oracle" or "Name two places besides the saved return pointer that you could overwrite to control program execution". The results we have seen on these questions alone are somewhat disappointing and very mixed.

Then we get to where the wheels always seem to come off. This is where we ask the candidates to actually demonstrate the things they have claimed knowledge of. We ask things like "Write out an HTTP GET request on the whiteboard". Some of you are probably saying to yourselves "That is simple". I would agree, and yet no candidate has done it correctly yet. We draw out a URL with GET parameters and ask them "Rewrite this request with a blind SQL injection attack".

The fact is that when asked to demonstrate these skills and knowledge disciplines outside the context of any sort of tool or crutch. One of my colleagues across the wall, in Incident Response land, has suggested that I am being too harsh. That people who can only use tools still have some value. He is right, as far as it goes. what happens though, when you have secured the environment past the point where you can just run metasploit modules and pop boxes. When you need to find design flaws, or 0days to exploit systems. A click-monkey is of no real value there, except maybe fetching coffee.


none of this means you should throw your tools away. Metasploit is a valuable tool and a framework for pentesting.  Those of you who know me, of course know, that when i find something Metasploit doesn't do that I want it to, i try and add it.  So while I can operate without Metasploit, and have to often, i try and continually reduce those occurrence by submitting enhancements to Metasploit. In this way I am also giving back to the community. Something i would encourage EVERY pentester to do: If you see something Metasploit should do, but doesn't, write it and submit it!

Or at least open up a feature request on their Redmine interface.

Infosec Institute Advanced Ethical Hacking

A while ago I made a post about Infosec Institute's 10 Day Penetration Testing Course . I had some not so great things to say about the first half of the course. I think, in retrospect, the first week would be good for someone just starting out in the field to get their feet wet. There are some things I definitely think I would change, to bring it more in line with that concept, but it's hard for me to judge since I was already outside of that target audience. I have finally had the time to delve into the second week of the training course. This portion of the course focuses on the real meat and potatoes of penetration testing and exploiting. There is still some tool-centric material at the beginning, but the course jumps pretty quickly into the good stuff. It starts covering program memory structure, and how buffer overflows really work. Pretty soon you find yourself writing basic shellcode, and doing memory analysis to perform true exploits.

There are ties back to tools, but mostly in how they can make your life easier. Everything this part f the course covers is done manually before they show you how to use a tool. In my opinion, this is exactly what they should be doing. I do not have an assembly background so some of this is valuable information I have been missing so far. From buffer overflows it moves on to format strings and heap overflows. There are sections on on fuzzing, fault injection and more that I have not gotten to yet. I hope to be finishing up the course in the next few days.

There are some benefits to the online version of this course, such as being able to set your own pace. That being said, I think this particular course would be worth paying the extra money for the classroom experience. These are much more complicated topics than the first week, and if you don't already have experience in assembly and memory structure you may find yourself wanting to ask questions that you will have to answer all on your own. There is nothing wrong with this, of course, but I personally prefer active discussion to simply reading things online.

All in all, my impression of the second half of this training is very different from the first. Anyone who has experience with penetration testing, but wants to delve into the real heart of the subject should take a course like this.

Training courses - Nerd steroids

A few years ago when I was trying to break free of the more mundane trappings of IT, I decided to take some certifications. I began with compTIA and took my Network+ and Security+ exams. Imagine my surprise when these certification exams took me no more than 15 minutes apiece to ACE. They were so easy it became embarrassing to tell people that i had bothered to take them. I have considered many times going for my CCNA and CCSP but never gotten around to it. I am now in the process of taking a 10day course from infosecinstitute. This course is actually comprised of two courses jammed together into a single bootcamp. I am doing the online version of the course, unable to get my company to buy in for the additional costs of actually attending a physical class. these courses are centered around the CEH, CPT, and CEPT certifications. I am not very far into the first week of material and I am starting to get that sinking feeling again.

I don't want to bad mouth infosecinstitute and it's training...at least not yet. However, the entire first day was essential an introduction into using vmware and linux. They do this because they want to be able to cater to people who might not have experience in those areas. My question is, what are such people doing taking courses on pentesting? If you don't know how to set up a VM, or how to kill a process in linux, you've got a long way to before you can be a pentester, and it is going to take a lot longer than two weeks. This is where the steroid analogy comes in. People seem to approach these classes as a quick fix, rather like steroids. "If I take this class, i will learn to be a 1337 h4x0r".

DarkNet has a post about training courses right now too. In it he talks about how the CEH is pathetic(I am inclined to agree so far) and then talks about a few other courses/certs. Frankly speaking, these look much the same as every other one I've looked at. They seem tantalizing at first, then you realize it's the same recap bullshit and you learn nothing new.

 Let's give up on steroids guys, and start thinking about some workout regimens. I want to see training courses out there that say outright "If you don't know what the different kinds of vulnerabilities are, or if you don't know how to find SQL injection, xss etc...don't take this class" Let's have some classes that start with "So you know how to find some vulnerabilities, let's talk about advanced techniques, and things you never thought to try before". Let's talk about how you maximize your extraction from a SQL injection, or what things work in Oracle or in MSSQL, or U2, or Sybase etc. Let's talk about some advanced encoding tricks, and how to pack javascript to get around filters. Let's talk about writing shellcode to try and exploit in a buffer overflow.

I am tired of having to rehash the same crap over and over again. Then I read what things RSnake or someone else is up to. I stop and think "hrm, what are they doing differently than me. What do they do better than me. Why?" I want to see training courses that answer those questions. I want something that says "okay, you're a pentester. now let me show you how the big boys do it"

Anyways, that is my rant for the day. Stay tuned as I am going to be working on putting together a bit of a SQL Injection cheat sheet in the coming weeks. I hope to have something comparable to RSnake's XSS cheat sheet and a lot better than the other ones I've seen.

Stored Procedures do not necessarily prevent SQL Injection

It seems that a lot of people think that just because an application uses stored procedures, it's queries must be safe. This absolutely false. Stored Procedures do not inherently add security, as they can be put together as poorly as any dynamically built query. I saw a perfect example of this the other day. An application took inputs, passed them to a stored procedures which then built a sql query by concatenating the inputs with predefined query strings. It then called sp_executesql to execute the dynamic query. The developer obviously had heard that stored procedures were safer than dynamic queries, so they went and made an SP, but they had their SP build a dynamic query. So all they succeeded in doing was pushing the problem back into the database layer instead of the app itself.

So testers and developers, please do not assume that an sp means safe. you still have to properly parameterize your queries and validate input and output. Security and shortcuts do not go together. If you think you may have vulnerable SPs like this, try running a query such as SELECT object_Name(id) FROM syscomments WHERE UPPER(text) LIKE  '%SP_EXECUTESQL%' OR UPPER(text) LIKE  '%EXECUTE%' OR UPPER(text) LIKE  '%EXEC%'

 to try and see where these venerabilities are.

NetSparker Community Edition Review

For those of you who do not follow DarkNET , it is a well run blog where they add their perspective on security news and events. They also post a never ending stream of new tools and updates. They area great resource for keeping up to date on the latests toys and tools. They have come through for me once again by introducing me to  Netsparker Community Edition. The last fire and forget web scanner I was enticed to check out in this manner was a horrible flop. It was called Acunetix, perhaps you've heard of it? If you haven't don't bother, it's rubbish.

So as you can imagine I was not expecting great things from Netsparker. However, as I was downloading it I noticed that RSnake had also posted about it. Like many people in my field, I tend to have an ego, but when RSnake speaks, I listen. So I installed the community edition and gave it some quick run through. As expected, many of the best features are turned off in the freebie version, but that's okay. They left enough good stuff in there to whet my appetite(good job marketing guys). So here are the things I noticed right off the bat:

1. The User Interface is very simple and straight forward. This is usually my first indication of a problem. In my experience, good products in this space tend to have absolutely wretched interfaces. they are tormented things that will try to bend your mind to it's will and subjugate you completely. The interface here is so simple most anyone could walk through setting up a scan. 
2. The User Interface makes sense. Acunetix is a perfect example of the simplistic but terrible User Interface. It is very simple, but anything but straightforward. Trying to understand how to make it do some of the things you'd like it to do is not an easy task. Netsparker does not suffer these issues. It presents you with almost everything you could possibly need and even more importantly, nothing you don't.
3. The sucker is FAST. I typically use IBM's Rational Appscan product. While AppScan is a good product, fast is never an adjective I would use to describe it. Netsparker is fast. Now part of why it is so fast is because the test profile is so limited in the community edition. So let's just look at the crawler. A 964 url page took appscan just over an hour to crawl. NetSparker did it in 15 minutes. It then ran all of it's tests in another 20-30 minutes. It may be that we will see these speeds drop dramatically with the full version, due to the expanded test profile.
4. SQLi right away. One of the apps I tested it on had SQL Injection right on the login page. AppScan had failed to detect it, but manual testing revealed it inside 10 minutes. Netsparker caught it immediately. While this is far from a comprehensive look at it's detection rates, I say bravo to netsparker.
5. Thoroughness. This is hard to gauge because it is the limited version. It FEELS like it is not very thorough. Part of this is psychological, because it runs so fast. Part of it is because it doesn't find some things because it is the 'community edition'. I can't shake the feeling that it is not being thorough, but I would really have to test the full version to make any honest assessment of this. 
6. No False positives, sorta. I performed several test scenarios, and it did not really generate false positives. The ambiguous language here is due to what I think is a very neat feature. On one of the test sites I saw a distinction in the results between 'we know there is cross-site scripting' and 'we think there might be'. I appreciate that it is extremely difficult to eliminate false positives, and I think this approach is great.
7. Testing framework. I have talked about this before, and I will talk about it again. We need to see testing harnesses, not just pas scanners. Once you are done with the scan, in Netsparker, it has tools you can use within the app to attempt to exploit the vulnerabilities. If you find a possible SQLi there is an actual injection tool built into the scanner to allow you to try and exploit it. It has similar tools for LFI and Command Injection. This, to my mind, represents the absolute right direction for these types of products to be heading in.
8. Pricetag. The community edition is free but limited. They then have two unlocked versions. The standard and enterprise edition. the key difference being the number of sites licensed for. I'm not sure if this means you predefine what sites you are licensed for or what. However, the unlimited Enterprise Edition comes with a pricetag of only $3000, which is extremely reasonable in my opinion. It also makes the product worthwhile even as a second scanner. I am considering recommending we purchase an Enterprise license so that we can have two scanners to see if we catch anything with one that we don't with the other. 

So let me summarize briefly. The Community Edition of Netsparker shows some very significant promise. It would seem to indicate a well thought out and well developed product. However, for professional assessments I would definitely recommend you not try to use the Community Edition.  Without having tested the Enterprise Edition, I won't recommend it out of hand, but at a pricetag of only $3000, it seems like a good idea.

Netsparker Community edition is created by Mavituna Security, and can be downloaded here.