Uniting IT auditors and DevOps for better business (with love)

I understand; auditors like me are often seen as the spoilsports who interrupt the flow and the fun of DevOps . After all, when teams of engineers can work cozily together on their product, they master their sprints and get into the zen of automation. Why would they want to break that up to do paperwork and also automate security checks? Why would they want some external auditor to come waltzing in and asking for receipts, literally and figuratively? So yes, it’s been hard for the DevOps community to see an auditor as someone who can actually help the entire team fulfill its value-delivering goals.

Yet, mission-critical auditing needn’t be dichotomous to DevOps. A good IT auditor should in fact support, not slow down, engineering projects. Ultimately, the audit goal should be to help improve the quality of their processes and outcomes.

Life made better?

One of the main reasons auditing and DevOps have become so dichotomized can be traced to that classic three lines of defense setup. In this arrangement, there is a cut-and-dry separation of powers between departments that handle business; security and risk; and auditing . The whole system is not in the room , as Schuberg Philis always stresses is crucial to meet the multidisciplinary, multi-departmental demands of mission-critical IT. Instead, experts each work in their own silos and, as I’ve described before, none speaks each other’s language.

DevOps requires a different control approach than previously used. This has been the conclusion of multiple research papers and white papers , one notable example being the latest ISACA-published COBIT framework (short for Control Objectives for Information and Related Technologies). The DevOps community itself attempted to reinvent its ways in 2018, creating the DevOps Risk and Controls Matrix. Amusingly, the matrix is prefaced by a self-described “love letter to auditors from DevOps”, where they promise to make life better illustrated by a giant heart. “Dear Auditor” is written in white cursive, giving the impression of a Valentine’s Day card, which is cute but no doubt an indication of the need to improve feelings of affection between these two groups.

The matrix marks a starting point and gives some good clues. But unfortunately, four years later, we see no big improvements in our lives with DevOps engineers across the board. Both of us are still unfamiliar with engaging each other and gaining trust.

Bringing worlds together

But having audited software development projects for a decade now, I’ve observed ways we can depolarize these groups and bridge gaps in their relationship. I took a step in this direction with the report “DevOps and Agile in control ,” which is written in response to the lack of control frameworks specifically tailored toward DevOps . The report has been authored with NOREA, the IT auditors’ professional association in the Netherlands, where I’m chair of the DevOps and software development work group.

In the report, a control framework is presented to conceptualize the mitigation of the key IT risks associated with Agile and DevOps principles. The framework is based on my experiences at Schuberg Philis and interviews I conducted with engineers, IT managers, and project managers encountered through my work. And while the NOREA report serves as a kind of handbook, with concrete recommendations for auditors like me, I imagine equivalent guidelines would be helpful to the DevOps community too.

The control framework presented currently consists of 15 suggested controls. We advise teams to start anywhere in the framework according to the needs and risk areas. Wherever they start, the goal is to improve security and predictability by implementing the controls . Below we present a preview of the focus areas of the controls, along with common points for improvements I encountered.

1. To expedite the work of auditors and therefore accelerate the entire DevOps project, several controls are defined. They focus on improving the way of working in the DevOps teams. We still often observe teams changing their way of working multiple times per year without any documentation and having engineers still following old agreements or a combination of old and current agreements. This negatively impacts a team’s efficiency as well as quality.
2. In an effort to unite the worlds of auditors and DevOps, auditors should dare to leave behind a reality solely determined by policies, procedures, and documentation as evidence. A good way to merge worlds could be through accentuating the everything-as-code principle and a centralized version control system. The automated deployment pipeline also has a lot to offer because it is not just highly automated, but also sequential – and sequentiality really speaks to auditors.
3. Still, to bridge the two worlds effectively, teams do have to document their environment better. We seldom encounter a properly documented DevOps team, often leaving us with the same questions. What applications or software are in scope? Through which deployment pipeline are they managed? Which solutions, third-party or otherwise, are used in these pipelines? Are the pipelines managed through the version control systems as well? How have these pipelines been maintained and changed throughout the year? Are the product teams or separate platform teams responsible for the pipelines?
4. Once teams have defined and documented their way of working and deployment pipelines, we look to see if implementation or configuration has been done properly. We ask: Do all pipelines fulfill requirements set by the team ? How is access and authorization to these critical configuration settings managed? And crucially, do we feel a team is in control?
5. DevOps teams are renowned for high automation. Still, we need to know: has a team adapted a test strategy? Does it also include security tests? Are there differences between the different solutions or services they develop? Per type of test, have goals been defined and deemed relevant? For example, are test coverage goals measured? Without these answers, we would struggle to give a proper opinion on proper testing by the teams.
6. Peer review is a common and accepted practice in most DevOps teams. However, which checks are executed by the peer reviewer is often not visible to auditors. It would be awesome, therefore, if a reviewer could document which tests they encountered, what the achieved test coverage was, if it meets the quality and security requirements of the solution or service, and if no critical findings were open.
7. Another salient characteristic of DevOps teams is their independence. This is great, but proper involvement of the end user and an organization’s business is crucial not only for the requirements phase, but also for change acceptance or deployment. And so we want to know: has the team made agreements meriting these parties’ involvement for testing and deployment approval? Does this fit with the associated risk profile of the solution or service? Are the activities executed by these parties documented as well?

Business on the brain

At Schuberg Philis, we ensure that the three lines of defense are not treated foremost as distinct lines, but as a single circle of mutually reinforcing priorities. This implies that business, security, and auditing experts are all involved in a project from day one. Each contributes their own specialized skills while supporting and sparing with one other to collectively create the best fit-for-purpose solution. In this setup, all defenses work toward continually better understanding the problem and cooperate to provide the optimum defense. And we apply these standards consistently, whether the solution concerns an on-premises environment, sbp.cloud [we could hyperlink to our sbp.cloud page], or public cloud services provided by our partners AWS, Azure, or Google.

It’s therefore not only business stakeholders who are concerned with what value a solution will generate. It’s the entire team – auditors included – who have business value on the brain. In this context, software development becomes more efficient. Engineers can embrace working without being encumbered by manual processes, filing unnecessary documentation, and/or starting from scratch with each project or the onset of a new year. A well-audited, highly automated environment thus speeds up software development.

So, dear DevOps, we’d like to return your love. Know that our controls can expedite your DevOps and Agile ways while improving the security of the deployments. Let us in early on and we’ll ensure projects get off to a flying start and maintain their velocity. As auditors, we aren’t here to make your work fast and furious, but something more like fast and fastidious. Together we can then enjoy the ride in all its speed and success.