Month: April 2017

The OWASP Top 10 — Response to the controversy from Jeff Williams

https://www.owasp.org/

The official response

Following my previous post about the OWASP Top 10 as well as the reaction from many others, Steve Ragan at CSO Online reached out to Contrast Security for their comments on the inclusion of “Insufficient Attack Protection” as the new A7.

Jeff Williams who is one of the OWASP Top 10 Co-Authors as well as being the CTO for Contrast Security provided the response. My thoughts as follows:

Contributions to the Project

The project is open for anyone to participate in. Unfortunately, like most OWASP projects, it is a huge amount of work and very very few contribute.

I think this a fair and important point. As I previously highlighted, there was a minimal response to the original call for data with only 11 companies responding with relatively large datasets and 13 additional companies with smaller datasets. As I said in my previous post and in my conclusion again here, OWASP needs more feedback and more contributors.

The proactive addition of items

The project uses the open data call data to select and prioritize issues, but has also always looked to experts for ideas on what we could include that would drive the appsec community to get in front of problems instead of being reactive. In 2007 it was CSRF, which is still a top ten item supported by tons of data. In 2013 it was use of libraries with known vulnerabilities, again an obvious yet serious and underappreciated problem, and the T10 helped to refocus the industry on it.

Again, I think this is a fair point. Being forward-looking and pro-active is important in such a fast-moving industry.

Certainly in hindsight, “CSRF” and “Libraries with Known Vulnerabilities” were worthy additions to previous releases of the Top 10 but note that they are both “Risks”, i.e. an issue/problem in the application. This is in keeping with the official title of the OWASP Top 10 which is “the OWASP Top 10 Web Application Security Risks”.

In this case, “Insufficient Attack Protection” is not a “Risk”, it is the lack of a “Control”. Note that OWASP already has a less famous but also very valuable Top 10 Proactive Controls list which already has as its item #8 “Implement Logging and Intrusion Detection”.

Moreover, despite the assertion above that the project has “looked to experts for ideas”, there is still no evidence of any discussion or consultation about the inclusion of A7 as I discussed in my previous post nor is any further information on this provided in this response.

Lack of a Control ≠ a Risk

Depending on where you observe the problem from, isn’t the lack of a defense a security vulnerability? It just depends on what we expect from our code, our vantage point on security.

Lack of defence is certainly an issue but in order to decide which controls should be put in place to defend an application, we first have to decide on the risks/vulnerabilities that are most concerning and prioritise accordingly.

The OWASP Top 10 was supposed to highlight the biggest risks to consider and including a control as part of this list confuses this assessment and takes up a space which could be taken by an actual application security risk.

Much of the appsec industry is focused on creating clean code, rather than protecting against attacks. But clearly we need both, as all the focus on hygiene hasn’t worked.

Agreed and again, I imagine this is why “Implement Logging and Intrusion Detection” is on the Top 10 Proactive Controls list.

In conclusion

Disappointingly, the response does not substantively address what I think is one of the key concerns with the latest release which is the lack of an appearance of independence. The response does not provide any further information to fill in the gap between the raw data and the final list nor demonstrate which other experts may have been consulted outside of the project team.

I think the response’s final sentence clearly demonstrates what the next steps should be:

I hope everyone interested in helping with the OWASP T10 will participate in the process, and discuss the pros and cons of this latest release candidate.

I set out my opinions for the future of the Top 10 risks project in my previous post but it is clear that there will still be a 2017 release.

The official instructions on the OWASP Top 10 site state:

Constructive comments on this OWASP Top 10–2017 Release Candidate should be forwarded via email to OWASP-TopTen@lists.owasp.org. Private comments may be sent to dave.wichers@owasp.org. Anonymous comments are welcome. All non-private comments will be catalogued and published at the same time as the final public release. Comments recommending changes to the items listed in the Top 10 should include a complete suggested list of 10 items, along with a rationale for any changes. All comments should indicate the specific relevant page and section.

I would therefore urge anyone in the application security industry to provide public comments by June 30, 2017 as has been requested by the project team. If enough constructive comments are submitted in the requested format, we will be in a good position at the final release of the list to assess to what extent the project team has taken the industry’s feedback into consideration.

Behind the The OWASP Top 10 2017 RC1

The power of OWASP

OWASP (The Open Web Application Security Project) was started in 2001 and describes itself as a:

“…worldwide not-for-profit charitable organization focused on improving the security of software”

The project has been enormously successful in reputation terms and is now considered the primary source of knowledge and truth when it comes to Web Application security.

In my job as an IT Security Consultant, I see many examples of companies relying on OWASP and the OWASP Top 10 Web Application Security Risks (hereafter “the OWASP Top 10”) and even considering the OWASP Top 10 as a de facto standard.

I have seen the following real life examples of this (without discussing whether each company is making correct use of the terminology):

  • Companies engaging with a software supplier will require them to have a secure development life-cycle which complies with OWASP guidelines.
  • Companies want us to provide (potentially based on own their client requirements) secure development training which covers the OWASP Top 10.
  • Companies expect that when we provide them with Application Security testing services, we follow a recognised methodology such as that set out by OWASP
  • Companies require us to provide an Application Security testing report which maps our findings to the OWASP Top 10. (We don’t like doing this as clearly the OWASP Top 10 cannot cover all types of findings)
  • Companies want us to provide just a “quick test” which “just covers the OWASP Top 10”. (We don’t do this!)
  • Companies want us to provide them with a “certification” that their application “complies” with the OWASP Top 10. (Good lord, no!)

The OWASP reality

The fact is that there are a large number of very high quality products and resources from OWASP (my personal favourites being OWASP ZAP, the OWASP Testing Guide and OWASP Juice Shop.

However, the quality of OWASP resources will generally be based on how much effort its unpaid volunteers are able or willing to put in and how much assistance they receive. For example, the OWASP ESAPI (Enterprise Security API) project was at one time a flagship project but was “demoted”, an action which its Java project owner agreed with. Its wiki page now recommends considering other alternatives before considering ESAPI.

To paraphrase the blog post above, not enough people were willing/able to spend time developing/maintaining it.

I don’t think that the ESAPI example should be considered to detract from the overall benefits which OWASP brings to the community but I think that people do not necessarily understand the relatively narrow base on which OWASP rests and the reliance on certain key people.

The OWASP Top 10 itself is considered a Flagship project and justifiably so given its success over the last 15 years. However…

Appearance of independence

Early in my professional life, I worked at a Big 4 accountancy firm where the idea of “independence” was drummed into me. In the Big 4 context, this is relevant for “Auditor Independence” where a Financial Auditor firm and its staff must demonstrate that they are able to perform a completely unbiased review of a company’s financial reporting without being exposed to external pressures which prevent it from being impartial such as a financial interest or inducement.

We were told that a Financial Auditor must both “be independent” and “be seen to be independent”. This second definition effectively means that even something that just looks like it would threaten independence, even if it does not, should be considered as a risk and avoided as carefully as an actual independence risk.

http://kfknowledgebank.kaplan.co.uk/KFKB/Wiki%20Pages/Audit%20and%20compliance.aspx

The OWASP Top 10 RC1 — Appearing independent

A Release Candidate of the OWASP Top 10 2017 was released a few weeks ago. Many people with more experience than I have debated both the technical merits of the latest release candidate and also examined the underlying data on which it was based.

However, I want to highlight a key point. The new items in the list are A7 — Insufficient Attack Protection and A10 — Underprotected APIs. Specifically about A7, the 2017 introduction says:

https://github.com/OWASP/Top10/raw/master/2017/OWASP%20Top%2010%20-%202017%20RC1-English.pdf (Page 5)

I don’t want to get into the merits of this new item but the analysis which I noted above highlights that there were three companies who suggested an idea similar to the new A7 risk, “Network Test Labs Inc.”, “Shape Security” and “Contrast Security”.

From: https://github.com/OWASP/Top10/blob/master/2017/datacall/OWASP%20Top%2010%20-%202017%20Data%20Call-Public%20Release.xlsx?raw=true

Shape Security are a vendor who make anti-automation software.

The vulnerability data which they have provided for the OWASP Top 10 relates to anti-automation and nothing else. They have recommended one additional item for the OWASP Top 10 and that is the problem which they can solve (h/t to Andrew Kalat at the Defensive Security Podcast).

Similarly, Network Test Labs also only provided vulnerability data in the anti-automation category and no other. They have performed some very limited research to support this:

We chose 3 US companies that had many users (as evidenced by their Alexa ratings) and were sizeable ($1B+ revenue) along with 2 other large US companies. We used a simple Selenium test to login to a website with 5 sets of credentials, 4 fake and 1 real. On 3 of the websites we tested, all 5 login attempts were possible, including the real set of credentials.

Finally, one of Contrast Security’s key products is a RASP (Runtime Application Self Protection) solution:

The new OWASP Top 10 2017 RC draft specifically name-drops “RASP” as a possible way of addressing the new A7 Insufficient Attack Protection risk.

https://github.com/OWASP/Top10/raw/master/2017/OWASP%20Top%2010%20-%202017%20RC1-English.pdf (Page 14)

Additionally, Contrast Security’s CTO and co-founder is Jeff Williams who is also the OWASP Top 10 Project Creator and co-author. It is important to note however that Jeff does have an impressive history in OWASP And AppSec in general.

Contrast Security was also the only contributor to suggest the other new risk which was added “A10 — Underprotected APIs”.

Having the​ only two new risks coming from one company with such a close tie to the OWASP Top 10 does not have the appearance of independence. Whilst, there is no attempt to disclose or highlight this connection in the OWASP Top 10 material, the company itself is already using the new Top 10 (which is technically still only a release candidate) in its marketing.

https://www.contrastsecurity.com/security-influencers/owasp-top-10-for-2017

Final Thoughts

The OWASP Top 10 project clearly provides its raw data sources but as the nVisium blog referenced above notes, the process between the raw data and the final Top 10 is not clear.

Additionally the Top 10 document states:

https://github.com/OWASP/Top10/raw/master/2017/OWASP%20Top%2010%20-%202017%20RC1-English.pdf (Page 3)

In my opinion, the process by which the new OWASP Top 10 release candidate has been produced does not have the appearance of independence and it is not currently clear whether it can demonstrate actual independence due to the missing link between the data and the end result.

On the other hand, as I noted above OWASP is entirely dependent on volunteers who are prepared to put time and effort into the its projects and therefore it can only work with what it has.

I think the response to this has to be three-fold.

  1. In the short term, I think the OWASP Top 10 project has to more clearly articulate its limitations. I would like to think that if the issues I have set out above were communicated correctly to companies and policy writers, they would understand the limitations and we would see less use of the OWASP Top 10 as a de facto standard. Companies should be using the more comprehensive Testing Guide and the ASVS (Application Security Verification Standard) as a starting point, potentially cherry picking the areas which will be most relevant to them.
  2. Perhaps the OWASP Top 10 Web Application Security Risks needs to be a data/risk driven view of the key issues which are being seen in the wild with more frequent updates but less focus on preparing a detailed and complex document. The focus should be on an ordered list of specific issues rather than trying to compress lots of issues into a top 10 list. The OWASP Top 10 Proactive Security controls which is a really useful and practical document for developers should be based on the this list of top issues (but not one-to-one) and provide actual hands-on ways to address security the most common security issues from the original list.
  3. Finally, the industry needs to be more involved in contributing to efforts like these. Only 11 companies contributed the vast majority of the data for the OWASP Top 10. I will certainly be encouraging my employer to start collecting the data required to submit and I think it is important that others do as well.

OWASP and its volunteers have worked hard to build this brand and reputation and it is our responsibility to help maintain and develop this.

Update: In a subsequent post, James Kettle pointed out that a similar issue involving Contrast Security has occurred in the past.