Hey! I am incredibly excited to finally release our industry report on how engineering teams are using AI. We have been running a big newsletter survey for the past two months, plus we held 1:1s and sat with several engineering teams to figure out what they are doing. The result is a ton of data points, both quantitative and qualitative, which should help us navigate what is it that teams are getting right, wrong, the challenges, and the opportunities of using AI in the software development lifecycle.
We have been doing this together with the team at Augment Code, which provided plenty of support and funded part of the work, making it possible. This is the largest scale research we have ever run at Refactoring, and it was possible thanks to them! So here is the agenda for today:
Let’s dive in!
🌐 DemographicsWe collected 435 respondents to the survey. As we did with other surveys in the past, we intentionally went for quality over quantity, which meant the survey was substantial, with plenty of free-form questions that took a while to answer. We took this route because AI is an extremely nuanced topic, and in many cases we didn’t want to pidgeon-hole answers by making people choose from predefined lists for things. A lot of free-form answers also mean a lot of manual work to clean up, categorize, and collect insights from such data. We reviewed every single answer, attached tags to it, and drew correlations. Here is more about the people who joined this: 1) GeographiesRespondents come from all over the world, with the following breakdown:
2) RolesAbout 52% of the respondents are pure ICs, while 34% are managers. There is also a 14% of tech leads that falls pretty much in the middle.
All in all this fits the distribution we have seen in previous surveys, with ~60% ICs and ~40% managers. 🙋♀️ Personal adoptionAbout personal usage of AI, here are our key insights: 1) Personal usage is strong77% of respondents use AI daily. 54% estimate saving 5+ hours / week with it, while 27.6% save more than 10 hours / week. Breaking down by role, Director+ roles are saving the most time, followed by pure ICs. Engineering managers are somewhat of the worst cohort. By looking at individual answers, the feeling is that a lot of classic line management has not been meaningfully transformed by AI (yet).
Speaking of use cases 👇 2) Coding is #1We asked people to input on the best ways AI is helping them with work. Coding is #1 across all roles, with automation of repetitive tasks being a close second.
But things are a bit more nuanced than this — when you look into the coding use cases, most of them are, in fact, about automating repetitive tasks: testing, boilerplating, recurring updates, and so on. So, AI is helping a lot with simple things, and, for many people, these simple things are a lot. Other simple quality of life improvements outside of coding include:
3) Managers are back to codingAnother meaningful pattern is engineering managers who are back to coding because of AI, which shows both in overall numbers (coding is the #1 AI use case for managers too) and in individual stories. AI is helping managers do more coding in two ways:
4) Documentation is a secret weaponWe also compared the preferred AI use cases of all respondents to those only from people who are very satisfied by their use of AI, to understand if there is any secret these folks have figured out, and found interesting numbers. People who are the happiest about AI use it less for high-value tasks like problem solving and research, and more for more mundane tasks — especially documentation.
Using AI for documentation creates a great feedback loop, because docs not only benefit humans — they benefit AI as well, which then is able to 1) understand code better, and 2) provide better answers about the codebase, which is another widespread use case. 5) ConcernsCode quality is by far the #1 concern in personal AI adoption, for all roles involved, closely followed by de-skilling:
🎽 Team adoptionIf we expand the scope to the team, we see somewhat a different picture. By now, 77% of teams formally recommend using AI tools, but the adoption process is chaotic. Management is largely happy to provide access to the tools that engineers ask for, but things usually stop there. The vast majority of respondents have no shared guidance around workflows and practices that go beyond personal usage.
The lack of top down—or just shared—direction not only hampers the amount of benefits the team as a whole can get from AI; it also fails to engage the minority of skeptics that exist in almost every team:
The main reason for this is that, simply put, we are still at a stage in which no one knows what they are doing. When asked for the biggest challenges in team AI adoption, the #1 factor mentioned by engineers is the lack of best practices and volatility of tools:
Most teams realize the quality of outcomes vastly changes based on context design and workflows, but very few are actually investing in learning these skills through dedicated resource allocation. Most engineers are just expected to learn on the job.
There is also less incentive to invest in shared practices when there is the perception that these may become obsolete in just a few months (or weeks).
Technical limitations are also a big factor. Testing AI performance and double-checking output is perceived as hard, and sometimes not worth the effort.
💼 Skills & JobsWe used the last section of the survey to understand how AI is changing hiring, and what engineers think about the future. 1) Headcount is not changing a lotAbout 29% of respondents report changes to their hiring due to AI:
In most cases, it’s a hiring slowdown, with some instances of headcount reduction, and extremely few about hiring increases. However, in our opinion these numbers are inconclusive about the impact of AI on org sizes and compositions. A lot of the slowdown stories seem to be defensive against the uncertainty around what types of roles, skills and seniority levels are now needed, rather than the result of considerations about throughput. About headcount reduction instances, it’s also hard to sort those that are truly about AI from those that come from the tougher market conditions, and would have happened anyway. 2) Execs believe they will need more engineersEngineers are largely skeptical that AI will permanently lead to fewer engineers:
Even more significantly, Directors, VPs and CTOs are the most skeptical about it. Only 11% of them believe they’ll need fewer engineers because of AI in the future, while 26% believe they will need more 👇
3) Skills are changingFor 73.2% of respondents, AI has changed the skills they look for, with the #1 effect being a stronger focus on high-level engineering chops (e.g. system design) vs expertise on specific languages and frameworks.
4) Interviews are changingA common challenge for hiring managers is filtering against over-reliance on AI and downright AI cheating. Several respondents mentioned changes in their interview process, including:
5) Engineers are optimisticOverall, people from all roles are optimistic that AI will make their jobs better and more enjoyable. You wouldn’t probably guess it by surfing online, but it seems there is a silent (vast) majority of engineers who truly enjoy using AI and look forward to using it more.
Here are some significant quotes 👇
6) Future predictionsFinally, we asked respondents to bet on future scenarios. By combining the most popular ones, the future that people in tech expect the most is one where:
As mentioned before, there is some concern among ICs about the need for fewer engineers, but this drops drastically when we ask the same to Directors, VPs, and CTOs. Do they know better? Or is it the classic disconnect between upper management and the trenches where real work happens? It’s hard to say — director+ folks seem to be at once more optimistic about the impact of AI, and also more conservative when it comes to how it will change workflows and orgs. They are bullish about the quality of software actually improving because of AI, are not convinced AI will write most of it, and are also less concerned with the fate of senior engineering talent. In other words, they believe software will get better, engineers are here to stay, and we will need more of them.
🪴 Adoption pathBased on everything we have seen, both numbers and individual stories, successes and failures, let’s try to write down some recommendations. If we look at the AI adoption journey, I believe we can identify three main steps: Explore, Embrace, and Empower.
Let’s look at each of them: 1) 🌱 ExploreThe first adoption step for almost all teams looks like personal exploration. Engineers get themselves more familiar with AI tools, figure out what they are good at, and learn the ergonomics of embedding AI into daily work. This exploration can be facilitated by management but it largely remains a bottom-up initiative that is up to individual engineers’ agency and proactivity. So what can managers do to encourage AI usage? Here is what works best:
The goal of this first step is to create awareness and momentum through some initial wins. Here's what these look like for most teams:
2) 🪴 EmbraceAfter the initial exploration has created a basic level of proficiency across the team, it’s time to embrace adoption and graduate it into team practices that go beyond individual usage. For most teams that got here, these look suspiciously similar to just good engineering practices — following the mantra that what’s good for humans is good for AI. So, good candidates are:
Teams who are the happiest about AI adoption consistently use it to 1) improve quality and 2) reduce cognitive load for developers, by taking on grunt work. It’s important at this stage that things happen at team level and turn from nice-to-haves into actual standards. For example, if we all agree AI makes writing tests easier, walk the talk by enforcing the presence of tests in PRs. If some docs now live inside the repo, for both human and AI convenience, reject PRs that do not also update those docs. You might feel uneasy about these, but hey, if it doesn’t feel somewhat risky, is it real change? To get real gains over the long run that go beyond simple code completions, we need to put something at stake. This is also the right time to create proper feedback loops. You won’t get everything right from the start, so it’s important to keep the pulse of how these practices are doing, what people think about them, and tweak them continuously. There are many ways of measuring gains from AI, but the best way to begin with is just through good conversations. How does this feel? How effective are AI code reviews? Are tests helping? Hold periodic retros, prod people during 1:1s, and keep the feedback going. 3) 🌳 EmpowerThe final stage is what we call the empowerment stage. If AI takes on some of the busy work and reduces developers’ cognitive load, what do we do with this residual capacity? The standard, incremental answer is just more of the same. More frontend coding if you are a frontend engineer, more design if you are a designer, and so on. Which is perfectly fine, mind you. But the best teams are using this capacity to empower people to go beyond their normal boundaries — to expand their scope, instead of simply being more productive at the same stuff. “[Engineers] shift focus on more business scoped things” — CTO The way you can do this depends on your role and your team: it could be engineers becoming full-stack, PMs creating prototypes instead of simple specs, designers having a shot at frontend, or else. The upsides of expanding people’s scope are so many:
This is also where top down vision is required the most. While you may rely on individual agency to plug AI tools into people’s existing work, it takes leadership to change the scope of such work, and how people think about it. Finally, these layers are not sequential: each of these is happening in some capacity at any given time:
AI will keep improving and software engineering will keep evolving with it, so we need to keep doing all of this all the time. Is it tiring? Yes, sometimes. But is it rewarding? You bet it. As engineers, we thrive on understanding things and turning chaos into rules and systems. That’s exactly what we need to do to get the most out of AI. Let’s get to work! 📌 Bottom lineAnd that’s it for today! Here are the main takeaways from the report:
I wish you a great week! Sincerely 👋
|
The State of AI Adoption in Engineering Teams 📊
Wednesday, 8 October 2025
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment