The Secret Sauce To Software Audits

Cyma: We at Cyma pride ourselves in creating a community for Solutions Architects and EAs alike. We do this by constructing content for you to enjoy, get informed with, and learn from. We use the community that we have created and give back to it in many ways. One way that we like to give back, is to allow guest bloggers to post on our website. With that said, we have given Neil Forster the opportunity to tell us his "secret sauce" in regards to Software Audits. If you would like to connect with Neil or learn more about him you can connect with him on LinkedIn. We are privileged to have Neil post on our behalf, and have no doubt that the value he brings will benefit you and Cyma. Thanks Neil for your contribution, enjoy.   


Neil's Contribution:

Software Audits

Having worked in a number of different companies and across both large and complex projects or existing systems I’ve regularly been involved in leading or instigating the need for software audits of a full environment or a major business solution.

This could be for a number of reasons, including:

  • Legacy systems with poor or out of date documentation (Architectural and operational)

  • Existing systems that seemed to be having performance issues that no one can address.  This could be performance in a number of areas, be it at the user interface, integration, latency, etc.

  • New or in progress projects that are well behind schedule and showing potential issues including CX or technical issues.

Firstly, to clarify, in my mind some of the key questions I ask around a software audit:

1. Audience:  Who is this for and why ?  Who is raising the issues or opportunities and what the the business implication of these.  Eg. for new projects, the delays could be leading to serious staff and potentially customer frustration.  If the solution is being further delayed, will the proposed business benefits be realised ?  Based on the size of the project or capital involved, this could mean the audit needs to be presented to executive teams or boards.

Alternatively, it could be to fully understand existing systems for a replacement of major overhaul.  Knowing this is critical as it determines the questions required and direction to take for the Audit.  

Additionally and in particular relating to project delivery issues with software (major upgrades or new applications) the focus needs to be clear.  Is this about understanding why the issues exist and where this problem comes from.  Or is this more about a focus on moving forward, getting an application, if that is possible, back on track.  Is it fit for what was proposed and what needs to be done going forward, with less emphasis on the history.

Project Scope

Project Scope

2. Scope:   What is the scope of the review.  A good audit should address more than just technology.  An audit should normally address the key areas of People (eg. design - including functional and non-functional areas, development and operations), Process and Technology.  This is key, as I’ve regularly seen technology only audits that result in a limited review and don’t actually get to the root cause of the problem.  If their is a need for an Audit, then these areas need to be address.  For example, a regular issue in existing systems is the lack of maintained architectural and operational documentation.  In many cases its what was initially delivered, with no updates or changes, in most instances, particularly in today’s environments with Agile delivery models and multi-tiered applications, this is very unlikely.   As a result issues exist as teams or new staff are operating with incorrect information.

Additionally within scope, is the depth of a review, particularly in a multi-tiered environment, (ie. with data layers, integration layers, etc).   This should be defined in discussion with the team completing the audit.

3. Who:  This should be thought about carefully.  Should it be an internal team or external. Depending on the issue, the audience receiving the audit and potential for actual or perceived bias this should be carefully considered.  Whilst internal teams may have the capability, they will also likely have pre-existing knowledge that can lead to assumptions.  These can lead to missing information and a poor or incorrect result from the audit.  Additionally, in particular for major problems or board level reports an independent report can carry more weight.  Even if existing teams are just as competent. I would also ensure that this Independence steers away form existing suppliers involved in any applications, either those who support it or have built it.  Again, the level of independence in these cases will help cover all bases without assumptions.

4. Timing:  It’s important to consider this, particularly based on the expectations of the audit and the audience.   Rushing an audit can lead to mistakes or recommendations that don’t have the time for real validation or analysis.   However, drawing out audits can exacerbate the frustration.   It is critical that this expectation is set up front and discussed clearly with the people leading the audit.   

With the above considerations all addressed, a couple of key areas need to be monitored to ensure the audit will be delivered as required.

1. Availability: This should apply to all areas, both staff, management and partners and even customers if appropriate.  The auditors will have a view, based on the above, of who they need to speak to or what systems/documentation they need to access.  

All of these groups should understand what is happening, why and the role the team leading the audit are playing.  In my mind it is critical that these area open and honest discussions.  Hence it is ideally not about blame or who is at fault, as these types of reviews lead to poor behaviours and incomplete information due to the approach.



2. Communication: The team leading the audit should keep in regular communication with you. Particularly about key issues identified and that the expected delivery time is on track. Sometimes, based on what is identified, it can be beneficial to extend the time, but this is never easy at the last minute and should be well managed.   

In all cases the audit report should initially be submitted ahead of the expected time for a first review.   If good communication has existed, there should be no surprises in this, but it allows for any questions to be raised, or ambiguous areas to be addressed, prior to submission to the final audience.

In summary, I have found a lot of value in these types of audits, across numerous areas from legacy systems to new applications in flight.  The process shouldn’t be onerous or long (though, note this point should be in context of the scope and size of the issue an application(s) be covered) and if good communication is in place, the results and recommendations will help a business make better decisions for the future.


If you have enjoyed our take on Software Audits and would like more information please fill out the form and we will get back to you. If you would like to be a contributor to our community fill the form out and we will get back to you to let you know how you can help the community.  

Name *