Engaging Software Engineers for FinOps Success

FinOps, also known as Cloud Optimization or Cloud Financial Management, is a growing discipline. FinOps practitioners help organizations proactively optimize cloud costs while preserving the flexibility and velocity of the cloud. This collaborative approach relies on engineering accountability for individual and team cloud spend.

FinOps practitioners can learn from the experts on the AWS Twitch show, The Keys to AWS Optimization, which features stories, concepts, and solutions to help companies optimize their spend at AWS.

Jon Thompson, CTO of cloud consultancy Virtasant, participated in an episode with host Stephanie Gooch, Commercial Architect at AWS, and Alex Head, Head of Optimization for AWS Strategic Industries. This roundup of the episode’s content will be valuable to internal FinOps teams, practitioners, and consultants interested in learning how to inspire engineers to be more active in FinOps.

FinOps and the Challenge of Optimizing Cloud Costs

Running infrastructure in the cloud spurs innovation and speed-to-market, but at what cost? Cloud operational expenses can be highly variable. Without higher-level oversight and coordination, costs can quickly exceed budget. When individuals and groups throughout the organization purchase cloud services or instances independently from each other, some sets of workloads will likely be under-provisioned while others will be over-provisioned. FinOps provides a way to optimize usage, resource types, rates, and costs without sacrificing the flexibility and speed that clouds provide.

According to the FinOps Foundation, “FinOps is an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology, and business teams to collaborate on data-driven spending decisions.”

FinOps is a way to bring teams together to get maximum value from the cloud across the organization.

‍

Engineering and the FinOps Culture

One rule of any successful enterprise is that business values should drive how we make decisions. However, it’s not that simple when working in the context of the cloud developer’s world, where primary drivers and job fulfillment come from achieving technical excellence and speed of innovation, not from watching costs.

Engineers already face the pressures of optimizing performance, reliability, and development speed. Because of this, they can view optimizing cloud spend as a distraction from their product focus, making it a secondary priority at best, and irrelevant at worst. This perspective can be reinforced when they have limited interaction with functions that are focused on cost, such as finance and procurement.  

Ideally, FinOps is a transformative process that brings cloud developers into a collaborative team in which IT, business executives, product owners, finance/accounting, and procurement work together on decisions about cloud usage. Getting engineering buy-in is paramount, as they hold the key to taking projects from concept to completion. However, it’s not uncommon for FinOps practitioners to face resistance from the engineers that are critical to FinOps success.

The State of FinOps 2022 reports that empowering engineering teams to take action on cloud cost optimization is the most common FinOps challenge for the second year running, with 40 percent of the survey respondents rating this their top FinOps-related challenge.

‍

Getting Buy-In from Engineers

Part of the FinOps practitioner’s job is to connect the dots between cloud costs and the engineer’s focus on choices about infrastructure, development, and design. For cloud optimization consultants and internal FinOps practitioners alike, the way they approach engineers can make all the difference. One thread that is woven through the Twitch show conversation is the importance of having empathy for engineers and taking the time to understand what drives them and the challenges they face.

Whether an organization is in the “Crawl, Walk, or Run” stage of FinOps maturation, the advice that follows can help organizations strengthen their FinOps practices.

‍

Start Off on the Right Foot

Every cloud optimization engagement starts with building trust with the customer’s management and engineering teams. Going in aggressively and telling customers what they need to do is an approach that can lead nowhere.

Here are some lessons the show participants have learned about the initial period of engagement with a customer.

• Listen and Learn: Do your best to have a client service mentality. It’s not about telling a team what their game plan will be and how you think they should hit the ground running. You have to want to learn from them. Let them know that you’re not there to bully them and make some low-grade savings. You can always learn from your clients, so ask them what practices have worked for them.
• Have a Positive Mindset: You may arrive to help a team and find that a year ago they invested in a tool to optimize cloud costs. However, the bill is still spinning out of control and nothing else has been done to fix it. Often, such unresolved situations can stem from an acrimonious, pushy, “you must do this” kind of approach directed at the engineering team. At that point, it can be an uphill struggle to get people back on board, become excited, and understand the idea that there are good things you can do to save money that aren’t going to cause you problems. Approaching the situation with a positive mindset can get projects back on track or prevent things from falling apart in the first place.

• Promote Partnership: Approach the initial engagement as a partnership, showing that you’re here to help them achieve the savings, rather than “I’m in control and you’ve got to do this.” It’s important to learn about their world to form a connection with the development team. Putting the practitioner’s and the developer’s brains together in the same room can help you learn new ways of doing things. Companies with existing FinOps teams and optimization experts may become defensive, saying they already have good optimization practices and are doing a good job — and they may be right. In that case, reiterate that you’re not there to replace them, but rather to offer them availability, act as a resource, and provide a broader perspective based on your work with other clients. Everyone has pain points. You can say, “Yes, you are doing well, and we can help make you even more successful.” After learning about their challenges, you can suggest or ask how you can be helpful, such as improving how they use a new tool, setting them up with a dashboard, or writing automation scripts.

• Celebrate Successes: Usually, discussions about cloud cost-savings will include engineering, finance,  and company leadership. A good rule is to first allow the technical side to show their leadership where they have succeeded in saving money. Celebrate that win before telling them the changes they have to make. This helps you earn their trust and put the relationship on a positive footing.

‍

Have Empathy and Understand Context

Engineers have a set of drivers and challenges related to their function, work processes, and the expectations placed on them. These are unique to each organization, and it’s important to recognize, acknowledge, and be responsive to the world they live in day to day.

You’ve got to understand each team’s people, systems, and processes, which can be quite different from team to team. Focus on understanding their nuances so you can help them with what they need, such as having the right conversations, putting together some informational slides, writing automation scripts, or whatever you can do to make things happen. A team may not want you digging into the technical fixes, but what they do want is direction and impact to help them show the wins.

The budget is another area where empathy is important. When a budget is released, FinOps may email reams of financial reports to engineers, causing them to feel spammed and overloaded. This may upset them to the point of ignoring the data. You’ll get more positive engagement by sorting through the financial data and providing only the information that is relevant in the context of their application. It also helps when someone from finance has a “techie” background and understands the developer’s perspective. That said, you won’t always have someone from FinOps with the exact background they’re looking for, but as long as you ask the right questions and genuinely want to learn about their goals, it can help bridge that gap.

‍

Separate the Signal from the Noise

Developers face many pressures and may become resentful when people make requests that they feel are meaningless or will create extra work that leads nowhere. We see instances where engineering teams say something like “The FinOps team will show off an opportunity they’ve found and send an email with a complete list of recommendations that the developers could be or should be fixing that will save millions a year.” Management might even be already planning next year’s budget as if all these savings have already happened.

Sending a seemingly random to-do list without pre-vetting it for executability will cause developers to feel that you don’t understand what’s happening and that you’re just distracting them from their work. You don’t want to be known as a person that doesn’t “get it” and will just waste their time.

Here are some ways to avoid such a scenario:

• Do Your Homework: Before talking to engineers, it’s important to review trends, sift through the data, and then think of concrete solutions that might save money. Then you can have a conversation along the lines of, “If it’s possible to do X, we could save Y amount per year. What do you think?” Maybe offer a few alternative cost-saving ideas and ask what might make that possible. In other words, be more specific and don’t make the engineer wade through an overwhelming amount of information. Then you can have a meaningful discussion where the engineer understands the opportunity and can provide input into whether it’s realistic and how to make it work.  
• Make Specific Suggestions: There’s a lot of groundwork you can do before talking to the team. Discuss the opportunity with people who have a more technical background than you, search the internet to avoid redundancies, and think about clusters of information. Overall, try to pare down information so that you can make sensible recommendations.

Jon Thompson provided this example:

A time-wasting optimization request will get a negative response, such as, “So, you’ve sent me a list of a thousand EC2 instances which you reckon I can right-size. That’s great, however I don’t really have time for that. I’m working on the next release, and I can already see that half of these optimizations are nonsense.” Before you’ve even started you’ve already lost them.

Instead of giving the engineer junk input, do your technical homework and analysis upfront. This will allow you to make specific suggestions that make sense in the context of their application—saving them time and helping them better understand the opportunity. You can make them more receptive by asking the right questions and showing that you want to learn.  

Going back to the example above, you might say, “We found this large prod Auto Scaling Group (ASG). All the instances are r5.24xlarge, but CPU and memory haven’t gone above the level that a 4xlarge could handle on any of them. Resizing to that would save $40K per instance and on average there are 100 instances—that’s $4M. I also see a similar ASG in staging, so maybe we could try the resizing there first?” Your job is to make it as easy as possible for the engineering team to take action.

‍

Take Action

Even if the engineer thinks a suggestion is sound, there’s a large gap between accepting a suggestion, putting a plan into action, and seeing it through to completion. You have to understand the context and identify everything that is needed to carry out the project.

You need to work with teams to find answers to questions such as who the decision maker is, who’s going to approve the buy, who’s going to work on it, what automation will be needed when it will be scheduled, what the change windows are, and so on. You have to identify what issues need to be resolved to make the project work.

‍

Provide Feedback Loops

You’ve done all your homework and put everything in context, but you’ll still get some things wrong. When you talk to engineers, they’ll give you information and feedback on their understanding. They might also tell you the challenges they’re facing, such as systems limitations that cause underutilization. They may say they can improve optimization a bit, but not as much as you’d think. It’s up to you to listen to them and understand the context.

Using the example of a recruiting team interacting with an engineering team, what doesn’t work is giving an engineer a stack of mismatched, underqualified candidates.. If the engineer then provides feedback about what they need, but the next batch of CVs is 80 percent the same as before, they may think, “Did they not listen? I’m not even going to bother this time. They’re just wasting my time.” At that point, you have to have a conversation with the engineer to make sure you understand their feedback, show them what they say matters, and start fixing the problem. It’s good to just ask, “How’s it going?” from time to time. “Do you have any feedback for me?” You can also ask what kind of feedback is helpful to them, and let them know what feedback is helpful to you.

‍

Reinforce Success

Gathering feedback makes projects and teams more efficient. Highlighting wins also reinforces success. As practitioners, we’re building relationships with our clients. We’re not done after we tell them how to fix a problem or when they have achieved the savings they wanted. The relationship doesn’t stop there because there’s more to learn. We get better by asking clients how they have implemented optimization ideas, what worked, and what didn’t. These conversations make us better, and they make our clients better.

When a team or a group inside a company starts making these changes with positive results, suggest that they create a video highlighting what they did, how much they saved, and why it worked. Number one, it’s a great opportunity for them and their career if they get to highlight a win and show the work that they’ve done. Number two, people are more likely to do something or make a change in their work habits if they see that a peer has done so and was really successful.

‍

FinOps is a Continuous Learning Process

The ultimate goal of the cloud optimization or FinOps practitioner is to make it as easy as possible for their clients to optimize their cloud resource usage and improve cloud efficiency. While many FinOps practitioners consider the biggest hurdle to be getting engineers on board, that is just one part of the larger challenge of having the entire organization adopt FinOps.

Jon, Stephanie, and Alex agree that engaging engineers to help create a FinOps culture requires building trust, understanding context, having empathy, and acting on feedback. This is supported by a desire to learn, understand, and build relationships that enable the practitioner and client to transform the culture together.

Jon closes the discussion by saying, “It’s not that I sit in a room, fire out a bunch of reports, and demand that you do stuff. It’s very much a people thing. Get out there, talk to people, learn, understand, iterate.

‍

See the complete episode on YouTube.

‍

‍

Jon Thompson is the CTO of Virtasant, a cloud consultant working with some of the biggest AWS customers. A developer by trade, Jon has 15 years of experience helping companies optimize cloud usage, what we now know as FinOps.

Stephanie Gooch is the host of The Keys to AWS Optimization on Twitch. She is a Commercial Architect in the AWS Optics Team, which optimizes customer deployments, identifies cost savings, and educates stakeholders on cloud best practices and how to create cost and usage transparency.  

Alex Head is Head of Optimization for AWS Strategic Industries. Her team serves as subject matter experts guiding customers through ways to optimize their current and future AWS spend.

‍