An Assessment of Remote Development Teams

Brendan Homann • Jul 01, 2020
Across the world, organisations are adapting to working remotely. For some organisations this is new, and for other organisations, it’s something they are well versed in. At Patient Zero our preference and our clients’ preference has typically been to work face to face in our clients’ offices. 

We’ve been carefully looking at the effectiveness of remote work for our software development teams since we moved to remote working in early March 2020 due to COVID19. In order to assess how our teams are going, we investigated a range of factors to try to give us an indication of how our teams’ performance may have changed during this period. We carefully watched velocity, burndowns and sprint stability. We actively sought out our teams’ and clients’ feedback in group settings and one on one. 

All indications so far show that PZ development teams are just as effective remotely, as they are face to face, in fact, they may work better remotely.

We wanted to understand why our teams were successful with remote working, while our customers reported challenges and communication break downs on other projects. From our research across 6 PZ Development teams operating in a wide variety of technologies and industries we have distilled the feedback down to these key factors:

1. Governance
At PZ our Way of Working sees clients actively involved in reviewing and prioritising the backlog. They are also involved in approving the start and accepting the result of each sprint. When we moved to remote working, these approval practices easily translated to teleconference. Our clients reported their teams struggled to confirm their backlogs, verify priorities and get approval from stakeholders when working remotely, PZ seemed immune due to our strong governance practices pre-existing the shift to remote work.

One of our clients commented "What has worked better than expected is working remotely, we didn’t notice any change in velocity. The way the team is working in the our environment, as hybrid team of our people and PZ people together. We’ve even used a number of PZ ways of working to get our internal teams working better remotely."

2. Sprint Milestones
PZ Development Teams as a Service operates sprint milestones. Our clients’ deadlines depend on successful sprints and we are invested in successfully completing the same sprints! When we shifted to working remotely, our teams still retained their focus on ensuring successful kickoff of approved sprints and successful delivery of quality sprint goals. We found that Showcases worked even better remotely and afforded a chance to record and share with those who couldn’t make it. The chat capability of video-based showcases maximised feedback, giving those shy people a chance to ask questions during the presentation.

3. Reduced Spin Up Time
We’ve had clients suggest their concerns about forming a team remotely to take on a new challenge, citing the concern of an increase to ramp up time as these teams must agree on processes, technologies, scope, priority along with forming mutual bonds. Our pre-configured cross-functional teams are built with significant care. We monitor 17 factors when assembling teams. Ensuring we have the right technical skills, cultural alignment with the client and the ability to work together. We add to this our fully formed PZ Way of Working which ensures team members know their roles and play to position well. As a result, we established multiple new teams to take on new projects entirely remotely. Our clients received working software, great documentation and full suites of automated tests with their delivery on time and on budget with the additional scope included! Projects which kicked off remotely ran smoothly, the governance practices aided this by enabling us to start producing valuable work remotely as quickly as face to face.

4. Transparency
PZ teams practice absolute transparency in their delivery model. This transparency is provided daily insight via telemetry and metrics to monitor and also at the end of sprint showcases. Our teams track a range of critical metrics.

Each sprint we continued to measure:
  • Velocity (sum of points completed) – this largely remained constant or trended upwards
  • Unit test Coverage – our teams naturally aim for 70% coverage
  • Automated UI Test Coverage – our teams must end each sprint with tests passing
  • Load/Performance Tests – our teams end each sprint verifying these results 
  • Changes to the sprint plan – our teams are trained to carefully handle changes to sprint plans
As an example, the following is the Velocity Measurements from an 11-week project started remotely with a team who had not worked well together before. Sprint 0 was 1 week to get tooled up, environments available, and familiar with the new domain. Each subsequent sprint was closed and accepted by the client with a consistent velocity:  

Each day during a sprint we:

  • Ran standups and escalated issues through our PMO
  • Shared our burndowns with PZ




This is a typical burndown chart shared within our company-wide slack feed. This type of burn down chart is affectionately known as a “ski ramp”, where your progress is so good, you decide to commit to more work during the sprint (usually because something got blocked). We found a combination of fewer distractions and reduced commute time increased focus time and resulted in outputs that exceeded our own estimates.


Sharing burn downs with the entire company wasn’t about upper management oversight, but about finding ways we could support each other to deliver. Our teams had the whole organisation cheering them on if they appeared off track. We had team members from other teams offering to help any time the burndowns got off track. These metrics were an ideal opening to a conversation about what the challenges were and how the entire organisation could help a team. There were plenty of examples of team members supporting each other.


We also wanted to assess ourselves against the core Agile principles we subscribe to:


* People and Interactions – our teams shared Slack channels with client stakeholders and we provided access to a video conference location for team members to jump on and collaborate as necessary. Ceremonies like standup, showcase and sprint planning moved online successfully. As a result, we saw increased collaboration digitally, replacing the interactions we normally had face to face. This usually started early from 6am and continued late into the evening as people worked around personal constraints of supporting family and friends, school and the like. Teams commenting to say good morning, ask each other how their weekends were, celebrate each others birthdays and even started sharing pictures of their pet cats, dogs, chickens and snakes!


* Working Software – we delivered working software every 2 weeks due to our sprint milestone approach. There was no change from business as usual, each sprint the teams shipped functional software. Demonstrated the features to users in a live demo with the software running in a deployed environment. The only way to get it deployed was to go through the CI/CD pipeline with automated UI tests, Unit Tests and Load Tests passing. 


* Customer Collaboration – we shifted collaboration with our customer and their stakeholders to digital with chat and video conference. We were able to seek feedback early and often, having customers attend showcases, ask questions, and present feedback improved the cycle time for this collaboration.


* Responding to Change – when challenges occurred teams escalated the issues and obtained support as needed. When new insights emerged, the backlogs were reprioritised and changed. We saw no observable change in our team’s ability to respond to changes due to the governance practices and collaborative tooling they had access to


Overall, we’ve seen little to no drawbacks to working remotely, and even in some cases an improvement in moving to remote work. We are strongly considering adopting this as a practice going forward and encouraging our teams and our customers to establish remote work as the new business as usual. 



Share This Post

Get In Touch

Recent Posts

Prompt Injection Capture-the-flag – Red Team x AI

By Joe Cooney • 02 Apr, 2024
Red-team challenges have been a fun activity for PZ team members in the past, so we recently conducted a small challenge at our fortnightly brown-bag session, focusing on the burgeoning topic of prompt injection. Injection vulnerabilities all follow the same basic pattern – un-trusted input is inadvertently treated as executable code, causing the security of the system to be compromised. SQL injection (SQLi) and cross-site scripting (XSS) are probably two of the best-known variants, but other technologies are also susceptible. Does anyone remember XPath injection? As generative models get incorporated into more products, user input can be used to subvert the model. This can lead to the model revealing its system prompt or other trade secrets, reveal information about the model itself which may be commercially valuable, subvert or waste computation resources, perform unintended actions if the model is hooked up to APIs, or cause reputational damage to the company if the model can be coerced into doing amusing or inappropriate things. As an example, entrepreneur and technologist Chris Bakke was recently able to trick a Chevy dealership’s ChatGPT-powered bot into agreeing to sell him a Chevy Tahoe for $1 . Although the U.S. supreme court has yet to rule on the legal validity of a “no takesies backsies” contract (as an employee of X Chris is probably legally obligated to drive a Tesla anyway) it is not hard to imagine a future scenario with steeper financial consequences.

Five Key Areas for AI Disruption in Insurance

27 Feb, 2024
With the advent of ChatGPT, Bard/Gemini and Co-pilot, Generative AI, and Large Language Models (LLMs) have been thrust into the spotlight. AI is set to disrupt all industries, especially those that are predominately based on administrative support, legal, business, and financial operations, much like insurance and financial organisations.

Running AI locally when GPT-4 goes crazy

By Joe Cooney • 22 Feb, 2024
One of the features of life working at PZ is our brown bag lunch and learn sessions; presentations by staff on topics of interest – sometimes, but not always technical, and hopefully amusing-as-hell. Yesterday we took a break from discussing the book Accelerate and the DORA metrics to take a whirlwind tour of the current state of play running “open source” generative AI models locally. Although this talk had been ‘in the works’ for a while, one challenge was that it needed to constantly be revised as the state of AI and LLMs changed. For example, the Stable Video Diffusion examples looked kind of lame in comparison to OpenAI’s Sora videos (released less than a week ago) and Groq’s amazing 500 token-per-second hardware demo on Monday/Tuesday , and the massive context size available now in the Gemini 1.5 models (released a few hours before OpenAI announced Sora...coincidence? An effort by OpenAI to steal back the limelight! Surely NOT!). And now a day later, with the paint still drying on a highly amusing slide-deck for the talk, Google releases their “open-source" Gemma models! The day itself presented an excellent example of why having more control of your models might be a good thing. ChatGPT 4 users began reporting “crazy” and highly amusing responses to fairly normal questions . We became alerted to this when one of our own staff reported on our internal Slack about a crazy response she received to a question about the pros and cons of some API design choices. The response she got back started normally enough, but then began to seem to channel Shakespeare’s Macbeth and some other olde English phrases and finished thusly. "Choose the right charm from the box* dense or astray, it’ll call for the norm. Your batch is yours to halter or belt. When in fetch, marry the clue to the pintle, and for the after, the wood-wand’s twist'll warn it. A past to wend and a feathered rite to tend. May the gulch be bygones and the wrath eased. So set your content to the cast, with the seal, a string or trove, well-deep. A good script to set a good cast. Good health and steady wind!" The sample JSON payload was also in keeping with the rest of the answer. { "htmlContent": "

Your HTML here

", "metadata": { "modifiedBy": "witch-of-the-wood", "safety": "sanitized", "mood": "lunar" } } Hubble, bubble, toil and trouble. Although there were no reports of the GPT4 API being affected by this (only ChatGPT) it might have given people developing automated stock trading bots using GPT4 a reason to pause and contemplate what might have been if their stock portfolio now consisted of a massive long position on Griselda’s Cauldron Supplies. As ChatGPT would say, Good health and steady wind.
Bay McGovern Patient Zero

Meet Bay McGovern

By Demelza Green • 11 Feb, 2024
Bay didn’t start her career out in software development. At school, Bay excelled at maths and physics, but adored writing, English and drama; lost in a world of Romeo and Juliet and epic fantasy.
More Posts
Share by: