#openai

waynerad@diasp.org

OpenAI announces GPT-4o. The "o" is for "omni". The model "can reason across audio, vision, and text in real time."

There's a series of videos showing conversation by voice, recognizing "bunny ears", two GPT-4os interacting and singing, real-time translation, lullabies and whispers, sarcasm, math problems, learning Spanish, rock paper scissors, interview prep, "Be My Eyes" accessibility, and coding assistant and desktop app.

Hello GPT-4o

#solidstatelife #ai #openai #genai #llms #gpt #multimodal

waynerad@diasp.org

Why this developer is no longer using Copilot. He feels his programming skills atrophy. He writes code by pausing to wait for Copilot to write code, and doesn't enjoy programming that way. The AI-generated code is often wrong or out-of-date and has to be fixed. Using copilot is a privacy issue because your code is shared with Copilot.

I thought this was quite interesting. I tried Copilot in VSCode and I figured I wasn't using it much because I'm a vim user. So I tracked down the Neovim plug-in & got it working in vim, but still found I don't use it. Now I've come to feel it's great for certain use cases and bad for others. Where it's great is writing "boilerplate" code for using a public API. You just write a comment describing what you want to do and the beginning of the function, and Copilot spits out practically all the rest of the code for you function -- no tedious hours studying the documentation from the API provider.

But that's not the use case I actually engage in in real life. Most of what I do is either making a new UI, or porting code from PHP to Go. For the new UI, AI has been helpful -- I can take a screenshot, input it to ChatGPT, and ask it how to improve the AI. (I'm going to be trying this with Google's Gemini soon but I haven't tried it yet.) When it makes suggestions, I can ask it what HTML+CSS is needed to implement those suggestions. I've found it gets better and better for about 6 iterations. But you notice, Copilot isn't part of the loop. I'm jumping into dozens of files and making small changes, and that's a use case where Copilot just isn't helpful.

For porting code from PHP to Go, I modified a full-fledged PHP parser to transpile code to Go, and this has been critical because it's important that certain things, especially strings, get ported over exactly -- no room for errors. So this system parses PHP strings using PHP's parsing rules, and outputs Go strings using Go's parsing rules, and is always 100% right. Copilot isn't part of the loop and doesn't help.

Another place I've found AI incredibly useful is debugging problems where I have no clue what the problem might be. This goes back to using other people's large systems such as the public APIs mentioned earlier. Every now and then you get cryptic error messages or some other bizarre malfunction, and endless Google searching doesn't help. I can go to ChatGPT, Gemini, Claude, Perplexity, DeepSeek (and others, but those are the main ones I've been using) and say hey, I'm getting this cryptic error message or this weird behavior, and it can give you a nice list of things you might try. That can get you unstuck when you'd otherwise be very stuck.

It's kinda funny because, obviously I'm an avid follower of what's going on in AI, and happy to try AI tools, and I constantly run across other developers who say "Copilot has made me twice as productive!" or "Copilot has made me five times as productive!" or somesuch. I've wondered if there's something wrong with me because I haven't experienced those results at all. But AI has been helpful in other areas nobody ever seems to talk about.

Why I'm no longer using Copilot - Dreams of Code

#solidstatelife #ai #genai #llms #codingllms #openai #copilot

waynerad@diasp.org

OpenAI is apparently partnering with a robotics company, Figure 01. The robot seems to do a remarkable job of integrating language with vision and body movements. They say it works using neural networks entirely. No teleoperation or hand-coded code. Nothing is scripted ahead of time.

"Can I have something to eat?"

"Sure thing."

"Great. Can you explain why you did what you just did while you pick up this trash?"

"On it! So I gave you the apple because it's the only edible item I could provide you with from the table."

"Great. So, based on the scene right now, where do you think the dishes in front of you go next?"

"The dishes on the table, like that plate and cup, are likely to go into the drying rack next."

"Great. Can you put them there?"

"Of course."

OpenAI's 'AGI Robot' Develops SHOCKING NEW ABILITIES | Sam Altman Gives Figure 01 Get a Brain - Wes Roth

#solidstatelife #ai #openai #figure01 #genai #llms #robotics

waynerad@diasp.org

"Introducing Rubra: Develop your AI assistants locally."

"Rubra is a full-stack OpenAI implementation, designed for local development. Built for developers, Rubra is an open-source project that delivers the same simplicity and intelligence as working with ChatGPT, but with a unique focus on building AI assistants powered by a locally running open-source LLM."

"Rubra serves as a drop-in replacement for OpenAI that can be deployed locally, for private, hassle-free AI development and testing."

"Rubra includes a highly tuned local model based on Mistral, that is ideal for local development. You can also add API keys for Open AI and Anthropic to compare how your agents perform. As models improve, we'll continue to upgrade the local model to provide the best experience."

If my attempt to build a PC with a powerful GPU capable of running these models hadn't failed, I'd be trying this. If you have a chance to try it, let us know how it goes.

Introducing Rubra: Develop your AI assistants locally

#solidstatelife #ai #genai #llms #openai #opensourcemodels