Good morning.
CFOs are slow to adopt generative AI, and the fact that a chatbot can hallucinate doesn’t help.
The large AI-powered generative language models (LLMs) that power chatbots are designed to understand and generate human-like text. However, because they mine billions of data points to predict the next word in a string of text, sometimes when they don’t know the correct answer to the prompt, they hallucinate or create an answer that may seem plausible but is factually incorrect or irrelevant to the context.
A Group of MIT Researchers Published New Paper What Argument Finding Among Chatbots Can Improve LLMs’ Logical Reasoning and Factual Accuracy. It’s like a bot debating club, except a bot can basically debate its own iterations.
“The discussion procedure allows the linguist to critique and reflect on his opinions and those of other agents, allowing him to refine his reasoning and responses,” Yilun Du, a researcher at MIT and one of the authors of the study, tells me. ‘article. Researchers have documented several instances of language models arguing over multiple rounds and proposed an improved joint response.
How does this work? “Discussions can take place in a single model (or bot),” says Du, a former OpenAI researcher. “A language model is repeated multiple times to create multiple bots. When asked a question, each bot generates a different response (the learned pattern behind a bot is the same for all bots). The bots can then chat with each other. »
However, the study also revealed that competing chatbots can compete with each other. “We also showed that you can have discussions between different models like [OpenAI’s] ChatGPT and [Google’s] cool to solve a task,” Du said. “But the majority of experiments use the same paradigm. »
Michael Schrag, a research fellow at MIT’s Sloan School Initiative on the Digital Economy, is not one of the paper’s authors, but says he thinks the research is well done. “This type of collective intelligence/voting approach is not uncommon,” says Scragg. “But as far as I know, this is the first publication I’ve seen as part of an LLM. »
Schrage explored generative and mastered AI systems with a focus on exploiting them as next-generation recommender systems. “I’ve used large language models before to generate business scenarios (some related to finance, some not) for clients and classrooms,” he says. “I found these scenarios constructive, provocative and plausible. But, again, these are LLM models, not big IT models. »
A foundational LLM needs to be honed and connected to software where counts and calculations are more likely to be accurate, as well as transparent, streamlined and explainable, he says. “However, I think any financial analyst, auditor, or accountant would be grossly irresponsible and unprofessional to rely on LLM-backed financial accounts at this time,” Schrag says.
He continues, “I strongly believe that – with thoughtful and deliberate safeguards and affirmations – FP&A people and other financial designers can achieve a lot of value very quickly by skillfully using an MBA. The MIT article shows just how important the “reliable computing” LLM space is.
Do you think the issues of hallucinations or misinformation are valid concerns for finance professionals? He said yes”. It’s very important to treat the answers of generative AI “not as a fundamental truth, but simply as a potential source of information”, he says. Du suggests using the answers as “ideas”, but then “validating them separately”. He adds: “I think my research is a step towards the accuracy of this source of information.
Let the discussion begin.
Sheryl Estrada
sheryl.estrada@fortune.com
Big deal
A new report from the Pew Research Center found that 58% of American adults surveyed had heard of ChatGPT. Of that percentage, 19% said they used it for entertainment, 14% used it to learn something new, and 12% currently work for pay and used ChatGPT for work. Adults under 30 who have heard of ChatGPT are more likely than those 65+ to use a chatbot for entertainment (31% vs. 4%). Pew also asked respondents about their experience with the chatbot. Fifteen percent say it was very helpful and 20% say it is very helpful. Meanwhile, 39% said it was somewhat helpful. The data is based on a survey conducted March 13-19, 2023 of more than 10,000 American adults.
Go further
“The Rise of AI: Is Your Business Ready for Generative AI?? It’s a new episode of The Wharton School training effect Podcast. Professor Rahul Kapoor explains why now is the time for business leaders to develop new frameworks to manage the changes ahead.
ranking
Julia Brown Donnelly His name was CFO Pinterest, Inc. (NYSE: PINS), as of June 20. Donnelly will reprise the role of Todd Morgenfeld. As previously announced, Morgenfeld will be leaving Pinterest to pursue new career opportunities on July 1. Donnelly joined Pinterest from Wayfair, where she most recently served as vice president and global head of finance and accounting. During her tenure of more than seven years, she held several positions of increasing responsibility in the finance function. She led a global team of 250 employees in all areas of accounting and finance, including strategic finance, investor relations, corporate development, FP&A, accounting, tax and financial operations. Prior to Wayfair, she was a private equity investor in technology and media companies at Thomas H. Lee Partners in Boston.
Yaffy (Roxy) Wayne Stepping down as Chief Financial Officer of Inviting (NYSE: NVTA), a medical genetics company, effective June 30. The company is looking for a new Chief Financial Officer. Wen will continue in his role until the end of the second quarter. Christine Georgink, longtime chair of the board’s audit committee, will assume the role of interim chief financial officer effective July 1. Wen’s resignation is not the result of a disagreement with the Company regarding any matter relating to operations, policies or procedures pursuant to Invitae.
he heard
“It’s not going to solve this problem if you’re sitting across the Pacific yelling at each other. So hopefully we will have a real commitment. »
— JPMorgan Chase CEO Jamie Dimon said at the JPMorgan Global China Summit in Shanghai on Wednesday, Reuters reported. Simon was responding to a question about diplomatic relations between China and the United States and stressed the need for “genuine engagement” to resolve security and trade issues.
