Working life
How does your role as a Mission-Critical Security Engineer fit within the organization?
Within the company, each customer has their own customer team and alongside those teams, we have Services. Services is a group of teams who provide IT services directly to Schuberg Philis and indirectly to our customers by providing semi-finished products that the customer teams deliver as a service. The Security team is a centralized team within Services, consisting of our CISO and several security engineers, like me. Our team is responsible for creating and executing a security vision and strategy; tracking vulnerabilities; creating and following up on security advisories; sharing security info, knowledge, and best practices; taking the lead on cross-team security incidents; implementing security tooling; and leading the security community within Schuberg Philis. That said, we have a strong security community within the company as a whole. Everyone in a customer team is in fact responsible for security, though each team has their own security lead. The security leads make sure the customer environment is secure both overall and specifically in line with what has been agreed on with the customer as well as Schuberg Philis's baseline security and best practices. In turn, everyone in the security team has multiple security leads they talk to on a regular basis, to check in. We ask: how's it going security-wise in your team? Do you need help from the Security team? Can we improve things centrally? There are always issues that target a specific customer, but then there are also issues that multiple teams are facing. It doesn't make sense if 20 people are trying to fix the same problem in isolation. That is where we can step in as well: if we see overlap between issues that multiple teams are facing, we collaborate on fixing them. We also have biweekly check-ins where all the security leads and the Security team meet.
What's a typical day like?
I'm in the office five days a week because I prefer to be onsite to speak to people in person. One of the things I worked on is tooling we use internally, which is also open sourced on our Github. Next to that I spend time automating our work using Python and CI/CD pipelines. Another big chunk of the work I do is on Splunk, the tool that we use for security monitoring. That is where most of our detection happens; we get logs in, we analyze the logs, and then we get alerts out. My focus is on analyzing these logs and creating detection rules based on these. The rules will in turn trigger alerts which need to be followed up. That follow-up is usually done by the mission critical security engineer responsible for responding to issues that week, who is announced in our dedicated Slack channel that can be followed by the entire company. If you have any security-related questions, then you can reach out; colleagues do that sometimes for simple stuff, such as a password reset for our password management tool, but sometimes also for a security incident that has come to light through preventative monitoring of our customers' environments or from the news, which we're always following.
What happens when there's a suspected security incident?
Incident management can indeed be triggered by a colleague who reaches out to us, but also through vulnerability notifications from the NCSC (Nationaal Cyber Security Centrum) or the news. Let's say we get a new high vulnerability alert concerning a software component that is used a lot in the industry and by our customers. As soon as this type of vulnerability is announced, we triage. So we check: everybody in the world says this is a high vulnerability, but is it applicable to us and the solutions we provide? Is this also critical for our customers and therefore us? Then we deep-dive to understand what the actual issue is. We try to apply all the information that is out there to our situation as well as that of our customers. If we verify that any of our teams are affected, an incident management team is formed to solve it. Together with the team, we make sure that all customers are accordingly informed and then make sure that, together with the customer, we have a plan to solve the issue as soon as possible. We are usually quite fast at tracking these vulnerabilities and getting to work on them.
Company culture
Schuberg Philis considers security business-critical. How does this translate into everyday operations?
Presently, there is an already high level of security that is always included in every customer contract negotiation. What customers and customer teams get in return is the Security team's support if they have any security questions or security incidents. Next to that, we have the aforementioned vulnerability management system. Then customers can stack on additional security services. Schuberg Philis, as a whole, is focused on prevention and defending. The fact that we so far have not had a lot of big escalations going out of control, also has to do with the fact that we try to build security as soon into any solution-making process as possible. That makes us quite different from other companies that have a security operations center (SOC), where you get a ton of alerts every day about a bunch of minor security incidents. Then you get flooded, which leads to alert fatigue, and you stop paying attention. That is what we don't want. Recently we’ve build a new managed detection and response (MDR) service. We started building this for internal use and we now also want to offer it to customers. We think we can make a difference and do MDR better. With this new service, we would target an entire organization, not just a specific application. We know that we can automate most of the tasks that a security analyst in a traditional SOC would do using a SOAR – security orchestration, automation, and response – tool. In short, we want to automate as much as possible.
What about the company culture enables such efficiency around security?
It is a couple of factors. At Schuberg Philis, security is not just a policy or a set of guidelines; it's so deeply ingrained in our culture that we say it's part of our DNA. All of us are security nerds or geeks, meaning we have a suspicious mindset that constantly asks: What could go wrong? Or if someone were to misuse or abuse this vulnerability, how would they do that? This approach enables us to demonstrate how real security goes beyond mere compliance; we actually maintain minimal security policies to encourage continuous thinking and understanding of security implications. Our culture of perpetual learning is thoroughly infused in our approach to security: it is not seen as a departmental role, but a collective responsibility by every member of every team. To respond quickly to security incidents, we must have that setting of open discussion and shared learning. Whenever needed, we sit together to find people with the most knowledge on the topic at hand and understand what's going on with the help of the customer teams. So as a Security team, we do not know every environment or all the ins and outs. However, the security leads is end responsible for the environment and are better aware of the details of the environment. When you combine the full security focus of the Security team with the full knowledge of the environment of the security leads, we can act quite fast. The security leads understand what we tell them about the security issue, and they have really short lines with our customers and should also be aware of how certain parts of the infrastructure are really important for the customers. Our customers like that we are highly security-conscious. Whenever we think that there is a security issue, customers tend to agree with us and we can move forward with patching or any other necessary steps.
In your 3.5 years at Schuberg Philis, what's an especially memorable security incident?
One of the things that was quite impactful was the Log4J vulnerability that even non-security people were aware of because it was big news. Suddenly everybody was in incident mode, and we needed to figure out where we and our customers may have been affected. Because this component was a library, it could be used anywhere. It was an interesting challenge to find out where it was used, but also how we could detect if it was misused or not. In this type of incident mode, usually I'm the one who works in Splunk to see what logs we have and how we can find from these logs if this vulnerability was successfully abused. For Log4J, we created some detection logic on-the-fly to be alerted during the crucial early days if anything were to happen. This brings up something that as a team we are wary of. In incident management, everybody can burn out really fast if you work too long too many days in a row. With these type of incidents, you know it's not something that will be fixed in two days, but more likely two weeks. It's a marathon instead of a sprint. You need to make sure that people do not wear themselves out.
Passion project
What are your hobbies?
I like home automation. Everything in my house is automated to the point that we do not use switches on the wall anymore. The light goes on when it needs to be on and off when it needs to be off. It took a couple of years to perfect, but I'm quite happy with the current status. Then again, since there are sensors everywhere, at some point I needed to think about how when I was in bed and turned over, I did not want the light to go on due to the motion sensor near the bed. So now there's a sensor underneath the mattress that detects if I'm in bed or not or if my girlfriend is in bed or not. And if either of us is in bed, then the light in the hallway won't go on. So there's a lot of tweaks done throughout to make sure that it totally fits how we live and use the house.
Do virtual home assistants on the market meet your security standards?
No, because most require internet access. Apart from the privacy concerns, if there's suddenly no internet, then everything in my home should still work. None of the devices that I use require or have internet access. Also: no microphones, no cameras. Everything is local, which is actually a hard requirement for me. Turns out, it is quite hard to find a smart thermostat that does not require internet access. But now I’ve found a way around that.
Curious to know more about how more colleagues spend their days? See the whole series here.