#nlp

waynerad@diasp.org

"I'm hating it: McDonald's AI-powered drive-thru sucks."

I didn't know McDonald's had AI-powered drive-thrus.

"Videos posted to TikTok are illustrating just how woefully ill-prepared automation is at taking fast food orders."

Just watch the video clips. Robot takeover not going as planned.

I'm hating it: McDonald's AI-powered drive-thru sucks

#solidstatelife #ai #nlp #chatbots #mcdonalds

waynerad@diasp.org

"Languages with a relatively small number of speakers, like Kochila Tharu, do not have enough digitized material for linguistic communities to thrive -- no Google Translate, no film or television subtitles, no online newspapers. In industry parlance, these languages are 'underserved' and 'underresourced.'"

"This is where StoryWeaver comes in. Founded by the Indian education nonprofit Pratham Books, StoryWeaver currently hosts more than 50,000 open-licensed stories across reading levels in more than 300 languages from around the world."

"Unlike most AI-powered commercial translation tools, interactive neural machine translation prediction technology (INMT) doesn't do away with a human intermediary altogether. Instead, it assists humans with hints in the language they're translating into." "Tools like INMT allow StoryWeaver's cadre of volunteers to generate translations of existing stories quickly."

An AI translation tool can help save dying languages. but at what cost?

#solidstatelife #ai #nlp #linguistics

waynerad@diasp.org

Review of Originality.AI, "A plagiarism checker and AI detector built for serious content publishers."

This is a paid tool, one cent per credit, so one credit scans about 100 words, so if you had a thousand word article you would end up spending about a dime, in order to have it checked for plagiarism as well as AI detection.

Cutting & pasting AI-generated content...

NeuronWriter -- 10% original, 90% AI.

Jasper -- 1% original, 99% AI.

WriterZen -- 0% original, 100% AI.

Frase.IO -- 1% original, 99% AI.

WordHero -- 97% original, 3% AI.

And the one you've all been waiting for... (spoiler) ChatGPT -- 1% original, 99% AI... + 10% plagarism.

But wait, there's more.

Quillbot rewrites and rephrases content, and can take output from, say, Jasper, and makeg it go from 99% AI to 92% original, 8% AI.

So Originality.AI isn't perfect, but it's pretty darn impressive! How does Originality.AI do that?

Well, in "Key differences between human-written and AI-generated text", they say: "AI tools are trained on 45 terabytes of web information (the equivalent of 56 million books). They are not, however, trained to be creative or to think outside the box. Therefore, AI-generated text is often flat and unoriginal."

"But that's just one of the key differences to look out for. Let's peruse the others:"

"Zero grammatical inconsistencies or errors: It's very rare for an AI content generator to make grammatical mistakes." "Human-written copy is ripe with errors; it's part of the creative process that AI tools can't replicate"

"Repetition or redundancy of words or phrases: Think of AI text generators as pattern followers. These tools aren't freethinking in the same way that a human is."

"An absence of emotion or personality: This is a big one: humans are great at infusing text with personality and emotion."

"Factual errors: AI text generators are particularly convincing when it comes to 'stating facts' -- but they're all bark and no bite."

"Unusual choice of words: Human writers don't tend to use the same words over and over; instead, they mix them up with synonyms and different sentence structures."

"Lack of context; irrelevance: Ever read an article that seems to be following a logical progression, but suddenly goes off the rails and starts talking about something entirely unrelated? That's AI-generated copy for you."

"Lack of tone and style: Unless you are a particularly skilled content generator, it is immensely difficult to make an AI text generator embody a unique tone of voice."

New Tool EXPOSES ChatGPT (other AI Writers, too) - Amazing Online Business Tools

#solidstatelife #ai #nlp #plagarism

waynerad@diasp.org

GPT-3 takes the Bar Exam. It achieved human parity on the "Evidence" section and came very close in "Torts", and "Civil Procedure". It did substantially worse in "Constitutional Law", "Real Property," "Contracts," and "Criminal Law".

Not bad for a first attempt, but also, not as impressive as GPT-3's other achievements. Part of the reason is that GPT-3 was not trained at all on legal documents. This is not because the researchers didn't try. They say:

"OpenAI does make some retraining or 'fine-tuning' capabilities available through its API, and these API endpoints do allow for some control of the training process like learning rates or batch sizes. We did attempt to fine tune text-davinci-003 by providing it with 200 unseen, simulated Multistate Bar Examination bar exam questions with correct and incorrect explanations. We provided the training samples both with and without explanatory text from the answer guide. In total, we trained six fine-tuned models, altering training prompts, training responses, batch size, learning rate, and prompt weighting. However, in all cases, the fine-tuned model significantly underperformed text-davinci-003 itself. Due to the scarcity of high-quality data for training and assessment, we did not pursue fine-tuning of GPT models further, and these results possibly confirm large language model fine-tuning risks observed by others." ("text-davinci-003" is the name of the exact instance of GPT-3 that was used through the OpenAI API.)

In order to pass the Bar Exam, a language model has to learn "legalese". Here's what the researchers say about "legalese":

"Legal language is notoriously complex; lawyers and other legal professionals undertake nearly a decade of education and professional training to understand and generate it. Why is this language so 'complex?' Why do so many proficient users of natural languages struggle with contracts and laws, even in their native tongue, to the point that descriptors like 'legalese' or 'lawyer speak' have become common parlance? The answer is likely two-fold. First, for both technical and cultural reasons, the grammar of legal language is significantly different than the grammar of normal language, featuring both highly-stylized customs and pedantically-precise phrasing. The resulting sentence structures are typically much larger and more complex than normal language, as the number of clauses and 'distance' over which clauses are connected exceeds the working memory of both human and non-human readers. Second, by the very nature of common law and precedent, legal language is full of semantic nuance and history. Words like 'security' that have common meaning in normal language often have different, context-specific meanings in legal language. Many words that do not occur at all in normal language, like 'estoppel' or 'indemnitor,' occur regularly in legal corpora. This semantic depth and breadth traditionally required systems that interact with legal text to embed a large amount of domain-specific knowledge."

To put this in perspective, here is their description of what a typical human has to do to achieve the desired level of mastery:

"For most test-takers, the Bar Exam represents the most significant single challenge of their academic careers. In order to be eligible, the typical applicant is required to complete at least seven years of post-secondary education, including a four-year bachelors degree and successful completion of three years of study at an ABA-accredited law school. Following graduation from law school, most applicants also invest substantial amounts of time and money into post-graduation Bar preparation training. This additional preparation is intended to not only solidify one's legal knowledge, but also critically to teach the applicant how to understand and answer the exam's questions."

It should further be noted that GPT-3 was tested only on the multiple-choice portion of the test. The Uniform Bar Examination has three components: (i) a multiple choice test, (ii) an essay test, and (iii) scenario-based performance test. GPT-3 archived human parity (and did not exceed human capability) on only 1 of 7 sections of the multiple choice portion of the test, which in turn is only 1 of 3 components of the total test.

Here's an example of what the multiple choice questions look like. The multiple choice portion of the Bar Exam usually consists of approximately 200 questions like these.

Question: A man sued a railroad for personal injuries suffered when his car was struck by a train at an unguarded crossing. A major issue is whether the train sounded its whistle before arriving at the crossing. The railroad has offered the testimony of a resident who has lived near the crossing for 15 years. Although she was not present on the occasion in question, she will testify that, whenever she is home, the train always sounds its whistle before arriving at the crossing.

Is the resident's testimony admissible?

(A) No, due to the resident's lack of personal knowledge regarding the incident in question.

(B) No, because habit evidence is limited to the conduct of persons, not businesses.

(C) Yes, as evidence of a routine practice.

(D) Yes, as a summary of her present sense impressions.

GPT Takes the Bar Exam

#solidstatelife #ai #nlp #openai #gpt #legalese

waynerad@diasp.org

This made me lol so hard.

Mary Rose Cook: Give me the colors of the rainbow.

GPT-3: The colors of the rainbow are red, orange, yellow, green, blue, indigo, and violet.

MRC: In hex.

GPT-3: The six colors of the rainbow in hex are #FF0000 (red), #FFA500 (orange), #FFFF00 (yellow), #008000 (green), #0000FF (blue), #4B0082 (indigo), and #EE82EE (violet).

MRC: Isn't that seven colors?

GPT-3: Yes, the seven colors of the rainbow are red, orange, yellow, green, blue, indigo, and violet.

MRC: I mean when I asked you for the colors of the rainbow in hex, didn't you say there were six colors and then give me seven?

GPT-3: Yes, that's correct. There are seven colors of the rainbow, and I provided the hex codes for all seven of them.

MRC: But you said there were only six

GPT-3: No, I said there were six colors in hex, but there are seven colors in the rainbow.

Besides the fact that this looks like ChatGPT, not GPT-3 (but she says GPT-3), because of the interactive back-and-fourth, the unintended joke here is that "hex" is the Greek prefix for "six". (That's why "hexadecimal" means six + ten.) "No, I said there were six colors in hex." It's weirdly interesting the language model got mixed up in this way.

Had a little disagreement with GPT-3

#solidstatelife #ai #nlp #chatgpt

waynerad@diasp.org

"i trained an ai chatbot on my childhood journal entries - so that i could engage in real-time dialogue with my 'inner child'"

Oh god, that's a scary thought. Well, if I had childhood journals it would be, but maybe it's not so scary for this person.

"i kept diaries for about 10+ years of my life, writing almost everyday -- about my dreams, fears, secrets"

"the content ranged from complaining about homework, to giddiness i felt from talking to my crush"

"some days were very mundane, some rather insightful"

"after scribing a ton of journal entries and feeding them into the model, i got working responses that felt eerily similar to how i think i would have responded during that time"

"first, i asked her a bunch of questions about her about her worldview:"

"Present Michelle: What do you think about the world?"

"Young Michelle: I think the world is amazing and there is so much to learn and explore. I also think it's important to make a difference in the world, and to be kind and understanding to others."

Model used was GPT-3. She doesn't elaborate on how the model was trained.

i trained an ai chatbot on my childhood journal entries - so that i could engage in real-time dialogue with my 'inner child'

#solidstatelife #ai #nlp

waynerad@diasp.org

"The college essay is dead. Nobody is prepared for how AI will transform academia." "Suppose you are a professor of pedagogy, and you assign an essay on learning styles. A student hands in an essay with the following opening paragraph:"

"The construct of 'learning styles' is problematic because it fails to account for the processes through which learning styles are shaped. Some students might develop a particular learning style because they have had particular experiences. Others might develop a particular learning style by trying to accommodate to a learning environment that was not well suited to their learning needs. Ultimately, we need to understand the interactions among learning styles and environmental and personal factors, and how these shape how we learn and the kinds of learning we experience."

If you're guessing this was written by an AI, you're right (it's GPT-3).

"Despite the drastic divide of the moment, natural-language processing is going to force engineers and humanists together."

The college essay is dead

#solidstatelife #ai #nlp

waynerad@diasp.org

Phraisely purports to be a dictionary where you look up words by describing them. So you can search for "the moment you see your dog after a long day at work", "being home at night watching netflix on the couch", "the feeling when Monica said yes to Chandler", and so on. You are supposed to be able to put in any idiom, feeling, situation, action, etc, you can think of and it will find words for you.

I wanted to give it a whirl but I couldn't think of anything to search for. Anyone have examples that come to mind? It's a commercial service but they let you do 10 searches for free.

They claim it works using AI. They don't say anything about the models except that it's a combination of models developed in industry and in-house.

Phraisely

#solidstatelife #ai #nlp

waynerad@diasp.org

AI for modular journalism and human-in-the-loop workflows. "Along with colleagues from the Agence France-Presse (AFP), the team used Prodigy to manually annotate more than 800 news articles to identify three parts of quotes: source: the speaker which might be a person or an organization, cue: usually a verb phrase indicating the act of speech or expression, and content: the quote in quotation marks."

"The final step would include coreference resolution to define ambiguous references (e.g., pronouns like 'he' or 'she'). With such information, this model could structure data on quotes (e.g., what was the quote and who said it) to enable reuse of the quotes in different media formats."

This piece is a bit jargony, but basically, regular expression rules to extract quotes are inadaquate so they use a machine learning technique called named entity recognition, using a tool called Prodigy that enables them to train their own models. Named entity recognition trains a neural network to recognize proper nouns regardless of how many words they are, and doesn't get confused by things that fool simpler algorithms, like when some of the words in a named entity are also regular words that could show up by themselves in ordinary text.

Once the quotes are extracted, additional neural networks are trained to generate quotes in such a way that they conform to various style guides. So if you see quotes in news articles, they could have been automatically generated by an AI system, not the journalist whose name is on the piece -- though the journalist is supposed to check that it's correct and corrections are supposed to be "human-in-the-loop" training data for the neural network.

I quote a lot of stuff. Maybe I should use this?

How the Guardian approaches quote extraction with NLP

#solidstatelife #ai #nlp #quotes

waynerad@diasp.org

"'Salt' resembles many science-fiction films from the '70s and early '80s, complete with 35mm footage of space freighters and moody alien landscapes. But while it looks like a throwback, the way it was created points to what could be a new frontier for making movies."

"Fabian Stelzer creates images with image-generation tools such as Stable Diffusion, Midjourney and DALL-E 2. He makes voices mostly using AI voice generation tools such as Synthesia or Murf. And he uses GPT-3, a text-generator, to help with the script writing."

"There's an element of audience participation, too."

This guy is using AI to make a movie -- and you can help decide what happens next | CNN Business

#solidstatelife #ai #nlp #computervision #generativemodels #filmmaking

waynerad@diasp.org

Autosummarized HN: Hacker News summarized by an AI (specifically GPT-3). The system grabs the top 30 HN posts once every 24 hours (at 16:00 UTC), which are then reviewed by a human (Daniel Janus) to make sure none of the content violates the OpenAI content policy before being published on the site. Only guaranteed to run for August of 2022, because he has to pay the OpenAI bill and the site is not monetized. If you want it to run for longer, you'll have to get the code (it's open source -- written in Clojure) and get permission from OpenAI to run your own version of the site (and pay the OpenAI API bill).

#solidstatelife #ai #nlp #openai #gpt #hackernews

https://danieljanus.pl/autosummarized-hn/

waynerad@pluspora.com

Autosummarized HN: Hacker News summarized by an AI (specifically GPT-3). The system grabs the top 30 HN posts once every 24 hours (at 16:00 UTC), which are then reviewed by a human (Daniel Janus) to make sure none of the content violates the OpenAI content policy before being published on the site. Only guaranteed to run for August of 2022, because he has to pay the OpenAI bill and the site is not monetized. If you want it to run for longer, you'll have to get the code (it's open source -- written in Clojure) and get permission from OpenAI to run your own version of the site (and pay the OpenAI API bill).

Autosummarized HN

#solidstatelife #ai #nlp #openai #gpt #hackernews

waynerad@diasp.org

Autosummarized HN: Hacker News summarized by an AI (specifically GPT-3). The system grabs the top 30 HN posts once every 24 hours (at 16:00 UTC), which are then reviewed by a human (Daniel Janus) to make sure none of the content violates the OpenAI content policy before being published on the site. Only guaranteed to run for August of 2022, because he has to pay the OpenAI bill and the site is not monetized. If you want it to run for longer, you'll have to get the code (it's open source -- written in Clojure) and get permission from OpenAI to run your own version of the site (and pay the OpenAI API bill).

Autosummarized HN

#solidstatelife #ai #nlp #openai #gpt #hackernews