Security, But Fun - Running A CTF Event for Your Dev Team

Security is an important part of software development, and at PZ we’ve tried to adopt “shift left” and “zero trust” approaches to everything we do. Unfortunately, security can also be a bit “dry” – imposing further checks on development, preventing certain approaches from being used and requiring additional verification and validation. Running red-team capture-the-flag (CTF) events is one thing we have done a few times to try and make security more interesting, help foster teamwork and just make work a bit more fun.  

For serious cybersecurity professionals CTF events at conferences like DEF CON are among the most difficult and competitive. Although we pride ourselves on our professionalism, our developers are not dedicated cybersecurity experts. If DEF CON is the Olympics of CTFs, our events are more like a social game of insert-your-favourite-sport-here. 

CTF events are usually organised in one of two ways – attack/defense where teams compete to steal flags from other teams and simultaneously defend themselves against the attacks, or jeopardy-style challenges where teams try to steal flags set up by the competition organisers. We ran the latter kind of events. What are these “flags” you speak of? Text strings – maybe database passwords, API keys, some dummy user PII or other pieces of information that would ordinarily represent something of value that should be protected in an IT system. Think of jeopardy CTFs kind of like a digital escape-room or puzzle, where the objective is to break IN instead of get out. 

CTF events normally run over a long-ish timeframe – maybe the length of an entire conference, or a couple of days. Although you could run an event of this duration in your company, we’ve favoured shorter timelines of a few hours to keep people engaged. To compress the timeline and to help assist staff who don’t have an extensive background in cybersecurity we added the element of the disgruntled insider – a persona who gives clues to the team after a certain amount of time has elapsed, to point them in the right direction. And because we’re a software development company we couldn’t resist creating a platform to deliver the clues and award points to each team automatically. We originally tried human moderation of this process, but it was hard for a person to keep track of the time each team had been working on each challenge, and so to ensure no team got a clue later than they should have we automated the whole thing. 

We scored the challenges in two ways. For challenges where the team received clues, we gave the challenge an initial point-value that decreased based on how long it took the team to solve it. For challenges where the team didn’t receive any clues, the points did not decrease over time. 

We found teams of 2-4 people worked best. Although we expected people to opt for remote teams where each person was working on their own computer, in practice people seem to enjoy being physically co-located and brainstorming ideas with each other. We recommend if one team-member is remote then everyone else should probably interact that way too (even if they are physically co-located) to avoid that one remote person being isolated. 

For some of the CTFs there was a natural order that the challenges needed to be completed in, where it wasn’t possible to progress to the later challenges until the first ones had been completed. For others there was no natural order, and for the most part all the “flags” could be worked on in parallel. Watching the way teams self-organised, and the dynamics that evolved in this competitive environment gave interesting insights into the strengths and weaknesses of some team members that were applicable beyond just the CTF challenge. 



Setting up the challenges is the most difficult and time-consuming part of running the event. Usually, this is best done by 1 or 2 people, who act kind of like a dungeon master in a fantasy role-playing game – they put in a lot of time behind the scenes to ensure everyone else has fun. The challenges need to be real-world to help staff learn real security practices. The difficulty needs to be balanced, and the challenges need to be somewhat varied. 

Consulting the OWASP top 10 list, and recent news articles of cybersecurity breaches should provide you with lots of ideas of the kinds of vulnerabilities you can add to a system. The security community has created some deliberately vulnerable systems for security training (often dubbed “goat” systems...a search for technology name + “goat” might yield some examples). 

It is highly desirable that the vulnerable system should also be easily re-deployable, preferably in an automated way using terraform, docker-compose or similar. This adds to the effort in creating the challenges but taking the time to do this will pay dividends. 

It is possible that creative attackers will compromise the environment during the event to the point where it is no-longer functional, and/or “pull up the ladder” to prevent other teams from completing challenges once they have passed them. Being able to re-set the environment is very useful for this reason, and to allow you to re-run the same set of challenges with a different audience from a known clean slate. 

Unfortunately, the “fun” of the challenges only really works for the first run, so it’s not really possible for the dungeon master to get feedback from the participants on the difficulty of the challenges they’ve set. If your organisation is hyper-competitive it is probably best not to get feedback from anyone in the org at all. The dungeon master might also want to make sure they’re running the challenge on infrastructure that is separated from the organisation too, for security reasons and also to reduce the temptation from hyper-competitive individuals to use their azure or AWS admin privileges to find the answers in a very creative way. 



Clues

Clues are a powerful lever the organiser can use to adjust the difficulty of the challenge. We recommend preparing the clues in advance, reviewing them, and delivering them such that each team gets the same clues verbatim. They can start with a vague suggestion early on to nudge the team and become more direct as the challenge comes to an end. 



Tools

Cybersecurity professionals have a vast tool-box of utilities they can bring to bear on a problem. To help level the playing field during the CTF event you could either provide a suggested list of tools ahead of time, a docker image they can download and use, or make sure all the challenges can be completed with basic system tools. 



Who should participate?

During the events we’ve run we haven’t been prescriptive about who can participate. Although each team probably needs at least one technical person, less technically inclined people can also bring something to the team – whether it be “out of the box” thinking about things to try, recognising when the team is going in circles, or listening to make sure less vocal team-members get their ideas heard. 



Summary

• “Dungeon Master”- like organiser creates a system with known security vulnerabilities that expose “flags” which teams earn points for discovering. 
• Ideally the vulnerable target system can be automatically re-deployed at the organiser’s discretion. 
• Teams know in advance any tools / prep / background reading they should do to be competitive. 
• Teams can be either physically co-located or remote. 
• Teams compete for points finding the flags in the vulnerable system. 
• If they choose to, the organiser can give pre-prepared clues to the teams to prevent them becoming blocked. 
• The team with the highest points is the winner.