CTO Foundry: Learning Corner #2

Lessons from Didi Dotan:

Our second CTO Foundry Learning Corner was led by Didi Dotan, former Director of Engineering at Cisco. During his time at Cisco Didi created and built the Cisco Defense Orchestrator, a product with over 1000 customers, as well as SecureX Sign on, a gateway to Cisco security products. Today, Didi is CTO at Oort, a detection and response identity security platform, one of our portfolio companies.

Given Didi’s experience building mission-critical products in the security space, we spent a lot of the time talking about how to transition a non-security-minded team to a security-minded team. How should your team be trained? What are ways that security can be an enabler instead of a bottleneck? Another question we spent time on is how to transition from a Director role at a large company to a CTO one at a smaller one.

How to balance engineering agility and rigidity when it comes to processes:

Didi has a unique dynamic where he has brought a group of Israeli engineers to every company he has been at. This team has followed Didi from RSA to Cisco to Oort. This has made the transitions between companies easier, but with each company, there is a new set of processes he and his team have to adapt to.

Didi explained he built this loyal following by becoming what his team members refer to him as, “a human shield.” That is, he works to keep the manager-level pressure at his level (i.e. pressure on deadlines, product direction discussions, etc), and block any trickle down to his individual team members. What that has looked like for him at each company is different, but his entire goal is to maintain its own structure and delivery process without letting outside forces disrupt it. “Teams need to be able to build software in a way that works for them,” Didi explained.  It is up to management to synchronize roadmaps and create timelines.

Processes that Didi found missing when he transitioned to a startup, were less around engineering agility and more around program rigidity. While his team's engineering agility did not change from company to company, startups inherently have a lack program rigidity. This will take some time to establish and may take away from early agility. For example, when he joined Oort it was the simple things such as a Terms of Use page on the website, setting up a privacy policy, putting the right structures in place for selling B2B or B2C.

When it comes to selling into large enterprises, which are governed by processes, buyers expect a polished terms of use contract. Establishing the correct protocols during B2B sales process is critical, such as knowing:

These questions are all part of the program rigidity that must be established at a startup. Didi compared establishing these processes to his time in the military, it’s like polishing your shoes or making your bed, you don’t want to do it but you have to. Creating program rigidity is the largest responsibility change. As an engineering leader, your job is to find the right balance at different stages of the company.

Additional Commentary:

Other VPs and Directors of Engineers in the group noted the importance of Didi’s comment, “Teams need to be able to build software in a way that works for them.” A few discussed that they have had to curtail processes a bit to fit them into specific reporting methods and metrics, but as long as they can report in a transparent manner, their team's processes are relatively left alone.

Creating a security minded environment:

The number one thing is education. If you start with education, it makes the remaining processes much easier. If the motivation for self-driven education isn’t there, producing an “Oh Sh***” moment is critical. Creating that moment of vulnerability awareness can shock people into beginning the process of education. For example, by bringing in a white hat to hack people’s machines you can create this moment in a safe environment.

Didi explained he has two favorite sayings when it comes to creating a security-minded environment: “sunlight is the best cleanser.” That is keeping everything transparent, allowing security problems to be flushed out more quickly by avoiding people covering them up.  Second, when it comes to taking security measures abide by the “Yoda Effect.” There is no offensive or defensive security code, there is only secure code.

Transitioning from a non-security minded environment to a security-minded environment:

While many in the group acknowledged their desire to have a security-minded environment they expressed frustration with actually getting everyone there. Many in the group are at mid-size SaaS companies that are not security-focused but understand the necessity.  They are struggling with where to start when you have half a million lines of code.

Drawing on his experience in the military, Didi shared that during training he and his group were brought to a field with thousands of rocks. They were told to move all those rocks into one pile. It seemed impossible but after day 3 the field was cleared of rocks. It was tedious to start but it was not impossible. Taking it back to the world of software, start by creating a list of your ten most egregious users. Then deal with the next ten. Create a process like a PSIRT- where one is product-focused and one is internally focused, then tackle it step by step.

Lastly, when it comes to systematizing new security processes, it ties into how often you ship code. If you are shipping code once every nine months, the potential for a significant breach escalates as code might be forgotten about or changes might not be pushed in time. Their current spring cycle is two weeks; if you ship with that cadence, small mistakes are made and easier to fix. Additionally, if you ship every two weeks compliance, customer demands, and security can be rolled up into that. He also recommends shipping on Wednesday, not a Monday or Friday. If you ship Friday, you ruin weekends by fixing bugs. If you ship Monday you also ruin weekends, with people trying to prep for shipping.

Getting Leadership onboard:

When it comes to creating a security-minded environment, Didi explained that this is where you need to talk up. Leadership needs to understand security is a growth engine, and it enables sales to happen more smoothly. Didi recommended ensuring that security ended up on the CEO’s list of priorities.

“If it is not in the CEOs OKRs then it doesn’t matter, it needs to be there”

For example, after the RSA breach of 2011 became public, Microsoft quickly realized that they cannot allow such a security breach to happen to them. They then hired top executives from security companies and built up their security capabilities. This all came directly from Satya Nadella, who at the time was overseeing Azure. When you listen to him speak now, he will discuss the importance of security as a cloud company.

One VP of Engineering explained if you have the resources, having a group tasked to focus specifically on security, that reports to the CEO are valuable. This team can communicate what needs to be done, set a roadmap, and prioritize what gets across the finish line.

If you’re interested in joining our events, please drop us a note at [email protected] and [email protected].