Want training, consulting or inspirational talks on this subject?
In 2019, many of the network devices in our infrastructure were end-of-life or getting very near an age where retirement was the only option. In other words, our network infrastructure was a clear case of technical debt. In Scrum@Scale we identified the issue as an impediment to the Network Team’s “Team process”, as the team was challenged on increasing their performance and operating in a way that was sustainable and enriching for the team.
Below you will get the short story of the symptoms of the technical debt, the impact on our business and our organization, the experiment we made in an effort to remove the symptoms – and the learnings we gained from it.
The legacy network devices and management tools were depending on hands-on resources for operations and maintenance, and the Network Team became reactive and firefighting. Most of their time and energy was spent on running the existing environment and very few resources could be used for preparing the network for the future.
The technology in the field of network was quickly moving towards infrastructure-as-code and software-based solutions – an entirely new paradigm that the engineers would need to spend time on embracing, but they weren’t able to find that time to develop their technical skills and imagine what the future could look like.
If network infrastructure is not your specialty, you might question how much requirements for connectivity could really change over 10 years? Does the Network Team really need to develop a completely new solution and live the DevOps dream? The answer to that is a sounding yes!
Today’s (not to mention tomorrow’s) requirements for security features and performance are significantly different from 10 years ago: the network infrastructure is key in the cyber security area of protecting vital business processes and applications by controlling data traffic, and the network must support the vastly increasing amount of data traffic that is the result of new streaming and IoT services, for instance.
The Network Team was not able to deliver to these expectations with the legacy technology that we were fighting to operate and maintain, and thus, the business was impacted.
Internally, the Network Team themselves were also impacted. They felt the heat from several CXOs who were frustrated that they couldn’t satisfactorily support top priorities such as the cyber security agenda. Ultimately, we were seeing a loss of pride and motivation in the team as the engineers wanted to deliver what was expected but could not see how they could set themselves up to succeed.
The Network Team needed to focus on developing a modern solution but they were swamped with operating the old network platform – and it wasn’t an option to stop doing operations of the legacy platform as much of the business still depended on it.
The issue was escalated to the Executive Action Team who introduced an experiment. We wanted to enable the engineers to focus, and to facilitate this we split the responsibility for the network between two teams. One team would manage the existing network and hosting infrastructure, another team (heavily supported by externals with knowledge about the new technology) would develop a new design and roll it out.
This setup would be temporary and the ambition is to merge the responsibility for the network sometime in the future.
It worked. And it worked just in time to ensure that when the world was closed down by Covid-19 in Q1 2020, we were able to establish a new VPN setup in a matter of days and ensure that the entire 14,000 white collar workers could work from home without issues when they were sent home.
A year later, the old platform is and has been running, keeping the business going, while a new network design has been developed and is in the process of being rolled out globally. This roll out, it turns out, comes with new challenges that we are planning experiments for, but that is another story.
We are trying out versions of this experiment in other teams who are heavily impacted by technical debt while under pressure for developing new solutions to our business.
Curious to learn more?
Want training, inspirational talks of consulting on the topic?