At Google, the fraction of code created with AI assistance via code completion, defined as the number of accepted characters from AI-based suggestions divided by the sum of manually typed characters and accepted characters from AI-based suggestions, now exceeds 50%.
"We achieved the highest impact with UX that naturally blends into users' workflows."
"We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value."
"Quick iterations with online A/B experiments are key, as offline metrics are often only rough proxies of user value. By surfacing our AI-based features on internal tooling, we benefit greatly from being able to easily launch and iterate, measure usage data, and ask users directly about their experience through UX research."
"High quality data from activities of Google engineers across software tools, including interactions with our features, is essential for our model quality."
"Human-computer interaction has moved towards natural language as a common modality, and we are seeing a shift towards using language as the interface to software engineering tasks as well as the gateway to informational needs for software developers, all integrated in IDEs."
"ML-based automation of larger-scale tasks -- from diagnosis of an issue to landing a fix -- has begun to show initial evidence of feasibility. These possibilities are driven by innovations in agents and tool use, which permit the building of systems that use one or more LLMs as a component to accomplish a larger task."
50% still seems like a lot. I wonder how much of that 50% has "code churn" -- has to be corrected again, even after being checked in? Maybe a lot of that 50% is actually correction code on previous LLM-generated code, lol.
Also, you would think if Google engineers are now writing code 2x as fast, we ought to be seeing rapid innovation in Google products. I'm not holding my breath. To be fair, Google is trying to innovate with Gemini, "AI summaries", and various other AI products. But, Google Search seems like it's been getting slowly worse for a long time (although I still use it and it's ok for most searches), and Google has a history of canceling a lot of products. I feed oddly doubtful this 2x productivity boost will make any visible difference to us users.
AI in software engineering at Google: Progress and the path ahead
#solidstatelife #ai #genai #llms #codingai
