CodeAid is a coding LLM designed to be learner-centric rather than productivity-centric.
This reminds me of how people sometimes ask ChatGPT to play "tutor" instead of just giving direct answers to things.
"Instead of generating code, CodeAid generated an interactive pseudo-code. The pseudo-code allowed students to hover over each line to see a detailed explanation about that line."
"Not everything needs to be AI-generated. CodeAid uses Retrieval Augmented Generation (RAG) to display official and instructor-verified documentations of functions relevant to students' queries."
"CodeAid also generates several suggested follow-up questions for students to ask after each response."
"When using the Help Fix Code, CodeAid does not display the fixed code. Instead, it highlights incorrect parts of the students' code with suggested fixes."
"Instead of just displaying a high-level explanation of the entire code in a paragraph, CodeAid renders an interactive component in which students can hover over each line to understand the purpose and implementation of each line of the provided code."
Thematic analysis of surveys and interviews revealed four types of queries from CodeAid:
"Asking Programming Questions" (36%): "Code and conceptual clarification queries" about the programming language, "function-specific queries" about specific functions, and "code execution probe queries."
"Debugging Code" (32%): "Buggy code resolution queries," "problem source identification queries," and "error message interpretation queries."
Writing Code (24%): "High-level coding guidance queries" ("how to" questions), and "direct code solution queries" (students copy the task description from their assignment), and
"Explaining Code" (6%): "like explaining the starter code provided in their assignments."
"Students appreciated CodeAid's 24/7 availability and being 'a private space to ask questions without being judged'."
"Students also liked CodeAid's contextual assistance which provided a faster way to access relevant knowledge, allowed students to phrase questions however they wanted, and produced responses that were relevant to their class."
"In terms of the directness of responses: some students indicated that they wanted CodeAid to produce less direct responses, like hints."
"In terms of trust some students trusted CodeAid while others found that 'it can lie to you and still sound confident.'"
"When asked students about reasons for not using CodeAid, they mentioned a lack of need, preference to use existing tools, wanting to solve problems by themselves, or a lack of trust over AI."
And if you're wondering how CodeAid compares with "ChatGPT-as-tutor":
"Comparing CodeAid with ChatGPT: even though using ChatGPT was prohibited, students reported using it slightly more than CodeAid. They preferred its easier interface, and larger context window to ask about longer code snippets. However, some students did not like ChatGPT since it did a lot of the work for them."
CodeAid: A classroom deployment of an LLM-based coding assistant
