Tag: Information Security

AppSecEU 2018 – UNOFFICIAL  Frequently asked questions

AppSecEU 2018 – UNOFFICIAL  Frequently asked questions

I know lots of people still have questions about OWASP and the AppSecEU 2018 debacle. Other than being a member, I have no formal standing in OWASP, locally or globally so nothing below represents anything official but I thought I would prepare some answers based purely on publicly available information.

What happened after the initial backlash?

The surprise announcement was followed by an angry rebuttal and a lot of outcry but after a few days things went quiet. Really quiet. The OWASP board email list has historically been relatively busy with consistent traffic. In the past 10 years, the latest traffic has restarted on that list after the holiday period is January 4th and only once has there not been a board meeting by January 14th. In 2018 there was complete board silence until January 18th when a number of OWASP leaders started querying what was going on. A formal, follow-up statement about the decision only came on January 23rd. It appears that there were some discussions being held behind the scenes culminating in a recorded conference call with OWASP board representatives and the UK and Israel OWASP leadership on January 22nd.

Why did AppSecEU get moved to the UK?

The follow-up statements seem to indicate that the root cause of the move was that recent operational challenges at the OWASP foundation, due at least in part to understaffing, meant that the foundation felt it was not in a position to provide the required support for the event. Especially given that it appears that AppSecEU 2017 and AppSecUSA 2017 did not provide the expected financial benefits.

The impression is that an AppSecEU in the UK is a safe choice whilst the foundation tries to address its internal issues.

We would like to acknowledge the effort of the organizing team, while realizing the required level of support from the foundation was not achieved.

https://lists.owasp.org/pipermail/owasp-board/2018-January/018442.html

…our major fundraising activities, the AppSec-Eu in Belfast and the AppSec-US in Orlando ending up negative on the balance and making less money then expected.

https://lists.owasp.org/pipermail/owasp-board/2018-January/018445.html

What about the supposed lack of preparedness from the OWASP Israel committee?

On the initial board call in December, a big deal was made that despite the conference only (!) being six months away, various preparations had not been made including no signed contract with the venue.

In fact, on the call on January 22nd, the new Executive Director praised the third party why the Israeli organising committee had engaged to assist with the conference logistics and more importantly stated that the foundation would cover the costs of having to withdraw from the contract which had in fact been signed with the venue.

 

So what is next for OWASP and Israel?

On the call on January 22nd, the board expressed strong support for a global OWASP event to take place in 2019 once the foundation had had a year to address it’s operational challenges. This seems to be how others have interpreted that as well.

Given that going forward the Executive Director is keen to start planning OWASP global events up to a year in advance, it remains to be seen over the next few months whether these actions are translated into words.

Additionally, the Israeli chapter have now released their response to the final decision and they are understandably still unhappy about the outcome but also positive about the intentions of the new board to try and repair the relationship and champion an event in Israel for 2019.

Conclusions

I think it is clear to everyone that the initial communication around this decision was not good enough but it is particularly disappointing that the basis for this decision (e.g. the lack of a signed contract and the “support” of the Israeli chapter in the decision) was demonstrably incorrect and that the initial communication and board discussion made out that the root cause was a lack of preparedness and ability to deliver of the Israeli chapter.

It is encouraging that this has been walked back to a certain extent however it is clear that it will take more than that to address the hurt which is felt by the Israeli chapter leadership.

The support for the Israeli chapter over Twitter and the board discussion of a global event in Israel in 2019 is also encouraging and I hope that the OWASP board will proactively reach out to the Israeli chapter leadership to make sure that this comes to fruition.

In the meantime, the Israeli security community remains strong with the only Microsoft BlueHat event outside of Redmond happening here last month, the monthly DefCon9723 meetings, devseccon in May and of course CyberWeek including BSidesTLV coming up in June.

I hope that trust can be rebuilt between the local chapter and the foundation but it looks like it will be a tricky road.

Daily Pen Test reports — Pros and Cons

For some clients where we perform security testing, the client requests that we report on all findings on a daily basis.

Now, I am 100% behind reporting progress in terms of what has been tested (assuming there are multiple elements) or more importantly reporting problems in progressing as soon as possible. However, there are still some clients where they expect this plus findings to be reported.

I wanted to jot down some thoughts on some pros and cons to this approach.

Advantages

A1: Feeling of progress

The client feels like we are working, progressing and finding stuff. (Although status reporting without findings should also mostly accomplish this).

A2: Immediate feedback and fix

The client receives immediate feedback on findings and can start to look at how to fix them even before we finish testing.

They may even be able to fix the finding and allow us to retest before the end of testing. I am always a little wary of the client making changes to an application in the middle of testing but if they are going to fix something but break something else that is going to happen regardless of if it happens during the test or after the test.

A3: Enforces reporting as you go

There is a tendency for consultants to save all the reporting for the end of the project. Hopefully they took enough screenshots along the way but even still, suddenly you are at the end of the project and you have 20 findings to write up. Having a daily report ensures that findings are written up as they are found, whilst they are still fresh in mind.

Disadvantages

D1: Time consuming

Whilst we would have to write up all findings anyway, it is still more time consuming to have to prepare a report daily. The report has to go through a QA process every day instead of just once and if it is necessary to combine reports from multiple people, it can get even more complicated. Especially if we are using a complex reporting template.

D2: Difficult to update already reported findings

Sometimes we will find something and only afterwards find another angle or another element to the issue which means that the finding needs to be updated. This leads to more duplicated effort with the finding being reviewed multiple times and the client having to read and understand the finding multiple times.

D3: Less time to consider findings in detail

Sometimes it takes some time to consider the real impact of a finding. For example, what is the real risk from this finding, can it only be performed by an administrator? Will it only be relevant in certain circumstances? Having to rush the finding out in a daily report loses that thinking time and can lead to an inaccurate initial risk rating.

D4: Getting the report ready in time

Every day becomes a deadline day with a race to get the report ready in time. It can disrupt the testing rhythm and mean that consultants have to break from testing to prepare the daily report therefore losing focus and momentum.

D5: Expectation of linear progress

Testing doesn’t progress in a linear fashion. A consultant might spend a lot of time trying to progress on particular test on one day or on another day find a bunch of quick, lower risk findings. A daily report creates an expectation of news every day and a feeling that no news means a lack of progress.

D6: Increase likelihood of mistakes

With the increased pressure of daily output, the likelihood of mistakes is also increased as report preparers are under pressure to deliver the daily report by the deadline and reviewers are under pressure to quickly release the report to the client.

D7: It might not even get to the client!

If there are a few people in the review process, if just one of them is delayed in looking at the report and they have a query, the report may not make it to the client in time to be relevant before the next day’s report is released anyway!

D8: One size doesn’t fit all

Once you get into the habit of expecting daily reports or you create that expectation with the client, suddenly it is expected for any project regardless of whether it makes sense. This can mean that ongoing discussion with the client is discouraged because “we’re doing a daily report anyway” or alternatively a project which requires in depth thought and research is being constantly disturbed with unhelpful daily reports.

Conclusions

I agree that is is a bad idea to do a load of testing and then only weeks later the client finally sees some output. Especially where there are particularly serious findings that immediately expose the client to serious risk.

However, the need to provide a continual stream of updates leads to time inefficiency, lower quality findings and disturbs the progression of the test.

As such, whilst the reporting format should be discussed at the start of the project with the client, the aim should be to agree on the following points by communicating the reasons discussed in this post:

  1. If this is a large project where there are multiple parts which are being tested one after the other in a short time-frame then it is worth reporting on progress over these parts on a daily basis.
  2. Problems with testing should always be reported as soon as possible plus a daily status update on these issues to make sure these are not forgotten.
  3. Critical threats which immediately put the client at severe risk should always be reported as soon as possible.
  4. If the application is currently under development or there is specific pressure to deliver key findings as fast as possible, then high risk findings or medium risk findings can be delivered during the course of the test but should not be restricted to a strictly daily frequency.

Additionally:

  • If this is a short project (up to a week) without lots of different elements or if this a long project (several months) then daily status reporting is not appropriate.
  • Reporting of all findings on a strictly daily basis will never be appropriate.

I was recently involved in an application security testing project for a large client covering around 20 applications with multiple consultants working simultaneously in just three weeks of testing. By discussing with the client up front and agreeing on points 1, 2 and 3 above we kept the client fully in the loop whilst not burdening ourselves with reporting every tiny detail everyday.

I will probably update this post as I think of more advantages/disadvantages but feel free to send me feedback in the comments or via Twitter.