HPE blog: CISO faces breach on first day on the job

Avast CISO’s Jaya Baloo has many lessons learned from her years as a security manager, including how to place people above systems, create a solid infrastructure plan, and best ways to fight the bad guys.

Most IT managers are familiar with the notion of a zero-day exploit or finding a new piece of malware or threat. But what is worse is not knowing when your company has been hacked for several months. That was the situation facing Jaya Baloo when she left her job as the corporate information security officer (CISO) for the Dutch mobile operator KPN and moved to Prague-based Avast. She literally walked into her first day on the job having to deal with a breach that had been discovered months ago.

Baloo had several reasons why she first started talking about working for Avast, which makes a variety of anti-malware and VPN tools and has been in business for more than three decades. “When I interviewed with their senior management, I thought that we were very compatible, and I thought that I totally fit in with their culture.” She liked that Avast had a global customer reach and that she would be working for a security company.

But after she accepted her job offer, the IT staff found evidence in late September that their environment had been penetrated since May. The evidence pointed to a compromised credential for their internal VPN. Baloo’s first day at Avast was October 1, and in the first three weeks she had numerous fires to put out. She never thought making the move to Avast was going to be a challenge. “Before I got there, I thought the biggest downside was that it was going to get boring. I thought this job was going to be a piece of cake.”

Fat chance of that. During those first weeks she quickly realized that she had to solve several problems. First was to figure out what happened with the intrusion and what damage was done. As part of this investigation, she had to go back in time six months and examine every product update that was sent out to ensure that Avast’s customers weren’t infected. This also led to understand what parts of their software supply chain were compromised. These things weren’t easy and took time to track down. They were hampered by having logs that weren’t complete or misleading. Evidence also had been inadvertently deleted.

Second was to build up trust in her staff. During her interviews, Baloo was very hopeful. “I felt that I didn’t have to sell them on the need for security, since that was their focus and their main business. I thought that they would be a source of security excellence.” To her surprise, she found out that they were a typical software company, “with silos and tribes and different loyalties just like everyone else.” As she began working there, she also had to climb a big learning curve. “I didn’t know who to believe and who had the right information or who was just being a strong communicator,” she told me. The problem was not that Avast staffers were deliberately lying to her, but that it took time to get perspective on the breach details and to understand the ground truth of what happened during and after the breach. Some stories were harder to elicit because staffers weren’t used to her methods.

Finally, she had to develop a game plan to restore order and confidence, and to ensure that the breach was fully contained. She made several decisions to revoke and re-issue certificates, to send out new product updates and to begin the process to completely overhaul the company’s network and protective measures. Twenty days into the job, she posted a public update that described these steps.

In my conversations with Baloo, I realized that she had developed a series of tenets from her previous jobs as a security manager. I call them Jaya’s CISO Gems.

  1. You have to continuously doubt yourself. First and foremost, avoid complacency and be paranoid about your own capabilities. “You need to have a plan for widening your own field of view, security knowledge and perspective. You have to include more potential threats and need to challenge yourself daily. If you don’t, everything is going to look normal.” Baloo told me that many security staffers have a tendency to pay more attention to their systems, and if a system isn’t complaining or issuing alerts, then the staff thinks all is well. This complacency can be dangerous, because “you tend to hunt for things that you expect and that means you are only going to find things you are looking for.” Part of the issue is that you have to be on the lookout for the unexpected and push the envelope and have a plan for improving your own security knowledge and skills.
  2. Trust people before systems. “We have a lot of faith invested in our systems, but not necessarily in our people. That is the reverse of what it should be. We tend to focus in our comfort zone, and our zone is in tech and metrics.” But a CISO needs to listen to her team. “I like a team that can tell you when you are wrong, because that is how you learn and grow in the job and have a culture that you promote too. And above all to do it with a sense of humor.”
  3. Build a functional SOC, not just a stage set. “A SOC should support your people, not have ten thousand screens that are pretty to look at but that really say nothing. The utility of a SOC is to able to provide the subtle clues that something is wrong with your infrastructure. As an example, you may still have firewall rules that allow for malware to enter your network.” Whether you have your own SOC or outsource it, its capabilities should match what is going on across your network.
  4. Everything in your infrastructure is suspect. Trust nothing and scan everything. She suggests starting with monitoring your oldest gear first, which is what Avast did after they found the breach. “Stop making excuses for this older equipment and make sure you don’t take away the possibility that you need to fix something old. You can’t be afraid of scanning something because this aging system might go down. Do pen testing for real.” Part of a good monitoring program is to do it periodically by default, and make sure that all staff know what the IT department is monitoring. “The goal isn’t big brother style monitoring but to find oddball user behavior and to make it visible. With cybersecurity, prayer is not an option.”
  5. Do your own phishing awareness training and do it often. While there are any number of awareness vendors that can help set up a solid program, the best situation is to craft your own. “You know your own environment best and it isn’t hard to create believable emails that can be used as a learning moment with those users who end up clicking on the bait. Phishing awareness training is really a people problem and very hard to get significant improvement, because all it takes is one person to click on something malicious. We were always successful at getting people to click. For example, we sent out one email that said we were changing the corporate policy on free coffee and tea and had users enter their credentials for a survey.” Part of rolling your own awareness program is being up on the latest email authentication protocols such as DMARC, DKIM and SPF so you can have confidence in your controls.
  6. Make sure you set the appropriate level of security awareness for every specific job role. “You don’t want your entire company knowing everything about your complete security policy, just what is needed for them to do their jobs,” she said. “And we should tell them how to do their jobs properly and not focus on what they are doing wrong, too.” As an example, she cites that the customer care department should understand the best practices on how to handle customer data.
  7. CISOs should be as technical as possible. “I see a lot of CISOs that come from a higher-level risk management background and don’t take the time or have the skills to understand the details how their security technology works. You shouldn’t be afraid to dive deeper.” She also sees CISOs that come from a regulatory background. Some of the biggest attacks, such as Target, were compliant with regulations at the time. Compliance (such as with satisfying GDPR) has turned into a paper exercise rather than checking firewall rules or doing more technical checks. Instead, you get caught up in producing “compliance porn that gets sent to the board and then you get pwned. Stuff gets lost in translation to management, and you need this technical background.”
  8. Prioritize your risk intelligence. You have to know what to act on first, it is all about triage. “You fix someone with a heart attack before fixing a broken bone,” she says. This means matching risk with relevance, as I mention in my blog post for RSA here. Part of this is doing a level of sanity checking with other organizations to see what they have included in their risk profiles. Don’t do the easy stuff first just because it is easy.
  9. Don’t panic and destroy evidence. As Baloo found out during her response to their own attack, you need to understand that an infected PC can be useful in understanding your response. “Every member of the enterprise needs to be part of your response,” she says. Part of this is being trained in how to preserve evidence properly.
  10. Start with open source security tools first. “I am not a fan of building custom security software unless nothing like it exists on the market and it is absolutely necessary. And if you write your own tools, go the open source route and embrace it entirely: build it, make it available with peer review and let someone else kick it. I have seen too many custom systems that never get updated.”

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.