Even before moving into a DevOps role I always found the topic of DevOps an interesting one and spent time researching, reading, and investigating DevOps approaches and practices and still always take the opportunity to learn more when I can and see what other organizations are doing.

I was recently on the panel at a technical event and a discussion took place around the publicized success stories from other organisations and how these can be frustrating to organisations just starting out on their DevOps implementation when things aren’t going to plan. I found this interesting as I think it’s easy to forget that although some organisations have had DevOps in place for years with a very mature operation, others are just starting out and the discussions that followed raised a great point; organisations may publish a success story but how many times did they fail before they eventually succeeded?

For me, this is really key and something I have related to many times when looking back at where PA Media Group was to where we are now. I always hold a meeting with my team at the end of each year as we start to wind down for our Christmas change freeze and we use it as an opportunity to look back over the previous 12 months, looking over our achievements and failures and understanding what we can learn from them and what direction the team should take going forward.

I don’t think you should ever be afraid to fail, as usually if you don’t fail you aren’t trying new challenging concepts, but you should use it as an opportunity to learn from it, understand what went wrong, know when to accept that an idea or plan has failed and enjoyed the successes when they do happen.

These factors led me to think ‘is there a right or wrong way to implement DevOps?’

When thinking of DevOps the first thing that always springs to my mind is ‘culture’ and at PA Media Group we have tried to use DevOps as an enabler to assist in creating a culture of collaboration and cross-functional working. PA Media Group use Development squads made up of multi-skilled engineers which function well for delivering new products or feature enhancements, and although we have a dedicated DevOps team, we try to use DevOps practices, principles, its approach, and tooling across other areas of the department and a lot of our focus over the past few years has been around applying a DevOps culture within IT Operations.

We have worked to try to build a culture that is moving towards an automation-driven NoOps environment. One example of this would be building up the knowledge and skills of the organisations’ Desktop support team and moving the build and management of the desktop estate to Chef, seeing it more as infrastructure as code.

This doesn’t mean everything we have tried has been a success. We have had many bumps in the road and had to make big decisions, one of which we made a few years ago when we decided to remove the traditional style system administration role and instead created a larger DevOps team whose remit, alongside working within development squads and supporting the business, includes removing manual repeatable tasks and investigating ways to automate system and service recovery.

Fortunately, this change proved to be a good decision and we have seen great benefits from it. But as with all changes, there will always be challenges, and this was a big change. It required the traditional system administration engineers to shift their mentality and change their culture, and we lost both Sysadmin and DevOps engineers along the way. Luckily we had enough engineers who could see the benefits it would bring, but many times along this change we almost saw a failure. Had we not had enough engineers with this mentality present within the teams, then this could quite easily have been marked down as a failure for us.

Another aspect that I feel helped this change succeed was when we realised the need to try and standardise and simplify our tooling and technology choices. We moved to SaaS solutions where possible to remove the need for engineers to have a vast array of technical knowledge, to remove the need for them to carry out manual maintenance, and instead, putting the ownership and management of these technologies on the people who have the best knowledge of the product.

Our journey has since furthered and we have brought our SysOps Support engineers (the current entry point into IT Operations) closer to DevOps, giving them more freedom to learn and be involved with newer technologies such as Terraform. We have started exploring how Agile principles can be implemented within an ITIL-focused environment to complement existing processes and have moved to a Swarm support model.

This has started to increase the SysOps support engineers’ technical knowledge and we are hoping it will bring greater engagement between engineers of all levels/skillsets, increase service knowledge across all the teams, and will move us closer towards working as a single team.

We are constantly reviewing this process to understand how it’s working. Some may look at this change and see it as a bad choice, a failure waiting to happen. Why would you want multiple engineers getting held up on a Swarm call when a ticket could just be passed to one high-level engineer and be resolved at the same time?

But could keeping this mindset of just passing things over also be classed as a failure? A failure to share knowledge and upskill engineers of various levels? What we hope to see, is that we hit a peak then a downward slope in the amount of Swarm calls required, or a reduction in the length of time Swarm calls run for, but only time will tell if this is successful or is marked down as a failure.

Looking back at one of the original questions ‘is there a right or wrong way to implement DevOps?’, then no I don’t feel there is. The right way is one that works for your organisation and I don’t believe there is a ‘one size fits all’ approach. Just because Swarm has worked within other organisations, it doesn’t mean it’s guaranteed to work for us. The same as just because removing our System administration team worked for us, doesn’t mean it will everywhere else. Businesses vary in so many ways and every IT Department has different technologies, levels of engineering skillsets, and budgets.

So every approach should be different, this isn’t to say that what one organisation does won’t work within your organisation. To me, DevOps should be what you want it to be and you should take the approach of making sure it is right for your business model. You shouldn’t get down heartened if you try something that worked somewhere else and it fails within your organisation!

Article written by Matthew Levitt, Head of DevOps at PA Media