With the rise of Agile software development and DevOps, we realized that there was a fundamental disconnect between the actual work we do and auditors' measurement methods, making compliance more difficult. This prompted Sandeep Gangaram Panday, our Audit Manager, to work with the Dutch professional association for IT auditors, NOREA, to develop a new DevOps auditing framework.
A disconnect between practice and measurement methods
From the experience derived from our Agile teams the recent years we clearly noted that the old measurement methodology is not a good match for some of the new working methods. You can't test DevOps with an audit method based on the waterfall approach. For example, Agile favors control automation over manual handovers. As a result, there are no procedurally defined written test reports signed by managers or by Change Advisory Boards. The absence of these formal documents could lead to unjustified audit findings, even when there is nothing wrong with the work.
Sign off by the expert
In the Agile and DevOps approaches the auditor has continuous insight, thanks to the logbook of everything that's being coded and by whom. That is directly at the source, so it is more complete and more accurate than sign-offs. Our framework identifies the specific checks and control points in this logbook, such as peer review rules, build and automatic test results – including security testing – for each deployment into production. That shifts responsibility from the manager who signs off to an expert with a precise understanding of the consequences of his or her actions, which are all documented in the logbook.
In the past, Schuberg Philis was distinctive because we excelled in existing ways of working. Now we also make a difference by seeking and stimulating innovation for our customers. What makes us different is that we deliver security and compliance at the same time. In this respect we are ahead of other service providers, developers, and fintechs.
The rapid rise of automated ways of developing like DevOps means that in the future auditing could be continuous, automated, and outsourced. This provides great opportunities for e.g. AI, which could spot anomalies and flag them, effectively replacing the function of present day auditors. This affects the entire auditing industry: their new role will be to judge the validity and correctness of the process by which an outcome was produced, rather than the outcome itself. To be able to do so will require the technical expertise to understand and judge software and coding. Schuberg Philis is actively advising the University of Amsterdam and VU Amsterdam on how to adapt their auditing curriculums accordingly. We could be witnessing the final days of the small armies of suits setting up camp in large corporate offices on a yearly basis.