{"id":118769,"date":"2024-10-27T00:55:47","date_gmt":"2024-10-26T17:55:47","guid":{"rendered":"https:\/\/hotvideos24.online\/?p=118769"},"modified":"2024-10-27T00:55:47","modified_gmt":"2024-10-26T17:55:47","slug":"researchers-say-an-ai-powered-transcription-tool-used-in-hospitals-invents-things-no-one-ever-said","status":"publish","type":"post","link":"https:\/\/hotvideos24.online\/?p=118769","title":{"rendered":"Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said"},"content":{"rendered":"<p> <script async src=\"https:\/\/pagead2.googlesyndication.com\/pagead\/js\/adsbygoogle.js?client=ca-pub-3711241968723425\"\r\n     crossorigin=\"anonymous\"><\/script>\r\n<ins class=\"adsbygoogle\"\r\n     style=\"display:block\"\r\n     data-ad-format=\"fluid\"\r\n     data-ad-layout-key=\"-fb+5w+4e-db+86\"\r\n     data-ad-client=\"ca-pub-3711241968723425\"\r\n     data-ad-slot=\"7910942971\"><\/ins>\r\n<script>\r\n     (adsbygoogle = window.adsbygoogle || []).push({});\r\n<\/script><br \/>\n<\/p>\n<div>\n<p>SAN FRANCISCO (AP) \u2014 Tech behemoth OpenAI has touted its artificial intelligence-powered transcription tool Whisper as having near \u201chuman level robustness and accuracy.\u201d<\/p>\n<p>But Whisper has a major flaw: It is prone to making up chunks of text or even entire sentences, according to interviews with more than a dozen software engineers, developers and academic researchers. Those experts said some of the invented text \u2014 known in the industry as hallucinations \u2014 can include racial commentary, violent rhetoric and even imagined medical treatments.<\/p>\n<p>Experts said that such fabrications are problematic because Whisper is being used in a slew of industries worldwide to translate and transcribe interviews, generate text in popular consumer technologies and create subtitles for videos.<\/p>\n<p>More concerning, they said, is <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/apnews.com\/article\/chatgpt-ai-health-doctors-microsoft-f63d7fcc4b361cf8073406bf231e2b92\">a rush by medical centers<\/a><\/span> to utilize Whisper-based tools to transcribe patients\u2019 consultations with doctors, despite <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/apnews.com\/article\/openai-sam-altman-nonprofit-equity-investment-irs-1447dc7589ae312017f7141384b19535\">OpenAI\u2019<\/a><\/span> s warnings that the tool should not be used in \u201chigh-risk domains.\u201d<\/p>\n<p>The full extent of the problem is difficult to discern, but researchers and engineers said they frequently have come across Whisper\u2019s hallucinations in their work. A <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/umich.edu\/\" target=\"_blank\" rel=\"noopener\">University of Michigan<\/a><\/span> researcher conducting a study of public meetings, for example, said he found hallucinations in 8 out of every 10 audio transcriptions he inspected, before he started trying to improve the model.<\/p>\n<p>A machine learning engineer said he initially discovered hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed. A third developer said he found hallucinations in nearly every one of the 26,000 transcripts he created with Whisper.<\/p>\n<p>The problems persist even in well-recorded, short audio samples. A recent study by computer scientists uncovered 187 hallucinations in more than 13,000 clear audio snippets they examined.<\/p>\n<p>That trend would lead to tens of thousands of faulty transcriptions over millions of recordings, researchers said.<\/p>\n<p>Such mistakes could have \u201creally grave consequences,\u201d particularly in hospital settings, said <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/www.ias.edu\/sss\/faculty\/nelson\" target=\"_blank\" rel=\"noopener\">Alondra Nelson<\/a><\/span>, who led the White House Office of Science and Technology Policy for the Biden administration until last year.<\/p>\n<p>\u201cNobody wants a misdiagnosis,\u201d said Nelson, a professor at the Institute for Advanced Study in Princeton, New Jersey. \u201cThere should be a higher bar.\u201d<\/p>\n<p>Whisper also is used to create closed captioning for the Deaf and hard of hearing \u2014 a population at particular risk for faulty transcriptions. That\u2019s because the Deaf and hard of hearing have no way of identifying fabrications are \u201chidden amongst all this other text,\u201d said <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/www.deafhhtech.org\/rerc\/meet-the-team\/gallaudet-university\/\" target=\"_blank\" rel=\"noopener\">Christian Vogler<\/a><\/span>, who is deaf and directs Gallaudet University\u2019s Technology Access Program.<\/p>\n<h2>OpenAI urged to address problem<\/h2>\n<p>The prevalence of such hallucinations has led experts, advocates and former OpenAI employees to call for the federal government to consider AI regulations. At minimum, they said, OpenAI needs to address the flaw.<\/p>\n<p>\u201cThis seems solvable if the company is willing to prioritize it,\u201d said William Saunders, a San Francisco-based research engineer who quit OpenAI in February over concerns with the company\u2019s direction. \u201cIt\u2019s problematic if you put this out there and people are overconfident about what it can do and integrate it into all these other systems.\u201d<\/p>\n<p>An <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/apnews.com\/article\/openai-nonprofit-application-tax-exempt-status-ebf38549cdf0529fc5ec727e2815ed12\">OpenAI<\/a><\/span> spokesperson said the company continually studies how to reduce hallucinations and appreciated the researchers\u2019 findings, adding that OpenAI incorporates feedback in model updates.<\/p>\n<p>While most developers assume that transcription tools misspell words or make other errors, engineers and researchers said they had never seen another AI-powered transcription tool hallucinate as much as Whisper.<\/p>\n<h2>Whisper hallucinations<\/h2>\n<p>The tool is integrated into some versions of OpenAI\u2019s flagship chatbot ChatGPT, and is a built-in offering in Oracle and Microsoft\u2019s cloud computing platforms, which service thousands of companies worldwide. It is also used to transcribe and translate text into multiple languages.<\/p>\n<p>In the last month alone, one recent version of Whisper was downloaded over 4.2 million times from open-source AI platform HuggingFace. Sanchit Gandhi, a machine-learning engineer there, said Whisper is the most popular open-source speech recognition model and is built into everything from call centers to voice assistants. <\/p>\n<p>Professors <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/koenecke.infosci.cornell.edu\/\" target=\"_blank\" rel=\"noopener\">Allison Koenecke<\/a><\/span> of Cornell University and <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/datascience.virginia.edu\/people\/mona-sloane\" target=\"_blank\" rel=\"noopener\">Mona Sloane<\/a><\/span> of the University of Virginia examined thousands of short snippets they obtained from TalkBank, a research repository hosted at Carnegie Mellon University. They determined that nearly 40% of the hallucinations were harmful or concerning because the speaker could be misinterpreted or misrepresented.<\/p>\n<p>In an example they uncovered, a speaker said, \u201cHe, the boy, was going to, I\u2019m not sure exactly, take the umbrella.\u201d<\/p>\n<p>But the transcription software added: \u201cHe took a big piece of a cross, a teeny, small piece &#8230; I\u2019m sure he didn\u2019t have a terror knife so he killed a number of people.\u201d<\/p>\n<p>A speaker in another recording described \u201ctwo other girls and one lady.\u201d Whisper invented extra commentary on race, adding \u201ctwo other girls and one lady, um, which were Black.\u201d<\/p>\n<p>In a third transcription, Whisper invented a non-existent medication called \u201chyperactivated antibiotics.\u201d<\/p>\n<p>Researchers aren\u2019t certain why Whisper and similar tools hallucinate, but software developers said the fabrications tend to occur amid pauses, background sounds or music playing.<\/p>\n<p>OpenAI recommended in its online disclosures against using Whisper in \u201cdecision-making contexts, where flaws in accuracy can lead to pronounced flaws in outcomes.\u201d<\/p>\n<h2>Transcribing doctor appointments<\/h2>\n<p>That warning hasn\u2019t stopped hospitals or medical centers from using speech-to-text models, including Whisper, to transcribe what\u2019s said during doctor\u2019s visits to free up medical providers to spend less time on note-taking or report writing. <\/p>\n<p>Over 30,000 clinicians and 40 health systems, including the Mankato Clinic in Minnesota and Children\u2019s Hospital Los Angeles, have started using a Whisper-based tool built by <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/www.nabla.com\/\" target=\"_blank\" rel=\"noopener\">Nabla<\/a><\/span>, which has offices in France and the U.S.<\/p>\n<p>That tool was fine tuned on medical language to transcribe and summarize patients\u2019 interactions, said Nabla\u2019s chief technology officer Martin Raison.<\/p>\n<p>Company officials said they are aware that Whisper can hallucinate and are mitigating the problem.<\/p>\n<p>It\u2019s impossible to compare Nabla\u2019s AI-generated transcript to the original recording because Nabla\u2019s tool erases the original audio for \u201cdata safety reasons,\u201d Raison said.<\/p>\n<p>Nabla said the tool has been used to transcribe an estimated 7 million medical visits.<\/p>\n<p>Saunders, the former OpenAI engineer, said erasing the original audio could be worrisome if transcripts aren\u2019t double checked or clinicians can\u2019t access the recording to verify they are correct.<\/p>\n<p>\u201cYou can\u2019t catch errors if you take away the ground truth,\u201d he said.<\/p>\n<p>Nabla said that no model is perfect, and that theirs currently requires medical providers to quickly edit and approve transcribed notes, but that could change.<\/p>\n<h2>Privacy concerns<\/h2>\n<p>Because patient meetings with their doctors are confidential, it is hard to know how AI-generated transcripts are affecting them. <\/p>\n<p>A California state lawmaker, <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/a16.asmdc.org\/\" target=\"_blank\" rel=\"noopener\">Rebecca Bauer-Kahan<\/a><\/span>, said she took one of her children to the doctor earlier this year, and refused to sign a form the health network provided that sought her permission to share the consultation audio with vendors that included Microsoft Azure, the cloud computing system run by OpenAI\u2019s largest investor. Bauer-Kahan didn\u2019t want such intimate medical conversations being shared with tech companies, she said.<\/p>\n<p>\u201cThe release was very specific that for-profit companies would have the right to have this,\u201d said Bauer-Kahan, a Democrat who represents part of the San Francisco suburbs in the state Assembly. \u201cI was like \u2018absolutely not.\u2019\u201d<\/p>\n<p>John Muir Health spokesman Ben Drew said the health system complies with state and federal privacy laws.<\/p>\n<h2>___<\/h2>\n<p>Schellmann reported from New York.<\/p>\n<h2>___<\/h2>\n<p>This story was produced in partnership with the Pulitzer Center\u2019s AI Accountability Network, which also partially supported the academic Whisper study.<\/p>\n<h2>___<\/h2>\n<p>The Associated Press receives financial assistance from the Omidyar Network to support coverage of artificial intelligence and its impact on society. AP is solely responsible for all content. Find AP\u2019s <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/www.ap.org\/about\/standards-for-working-with-outside-groups\/\" target=\"_blank\" rel=\"noopener\">standards<\/a><\/span> for working with philanthropies, a list of supporters and funded coverage areas at <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/www.ap.org\/discover\/Supporting-AP\" target=\"_blank\" rel=\"noopener\">AP.org<\/a><\/span>.<\/p>\n<h2>___<\/h2>\n<p>The Associated Press and OpenAI have a <span class=\"LinkEnhancement\"><a class=\"Link AnClick-LinkEnhancement\" data-gtm-enhancement-style=\"LinkEnhancementA\" href=\"https:\/\/apnews.com\/article\/openai-chatgpt-associated-press-ap-f86f84c5bcc2f3b98074b38521f5f75a\">licensing and technology agreement<\/a><\/span> allowing OpenAI access to part of the AP\u2019s text archives.<\/p>\n<\/p><\/div>\n<p><script>\n  window.fbAsyncInit = function() {\n      FB.init({\n              appId : '870613919693099',\n          xfbml : true,\n          version : 'v2.9'\n      });\n  };\n  (function(d, s, id){\n     var js, fjs = d.getElementsByTagName(s)[0];\n     if (d.getElementById(id)) {return;}\n     js = d.createElement(s); js.id = id;\n     js.src = \"https:\/\/connect.facebook.net\/en_US\/sdk.js\";\n     fjs.parentNode.insertBefore(js, fjs);\n   }(document, 'script', 'facebook-jssdk'));\n<\/script><br \/>\n<br \/><script async src=\"https:\/\/pagead2.googlesyndication.com\/pagead\/js\/adsbygoogle.js?client=ca-pub-3711241968723425\"\r\n     crossorigin=\"anonymous\"><\/script>\r\n<ins class=\"adsbygoogle\"\r\n     style=\"display:block\"\r\n     data-ad-format=\"fluid\"\r\n     data-ad-layout-key=\"-fb+5w+4e-db+86\"\r\n     data-ad-client=\"ca-pub-3711241968723425\"\r\n     data-ad-slot=\"7910942971\"><\/ins>\r\n<script>\r\n     (adsbygoogle = window.adsbygoogle || []).push({});\r\n<\/script><br \/>\n<br \/><div data-type=\"_mgwidget\" data-widget-id=\"1660802\">\r\n<\/div>\r\n<script>(function(w,q){w[q]=w[q]||[];w[q].push([\"_mgc.load\"])})(window,\"_mgq\");\r\n<\/script>\r\n<br \/>\n<br \/><a href=\"https:\/\/apnews.com\/article\/ai-artificial-intelligence-health-business-90020cdf5fa16c79ca2e5b6c4c9bbb14\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>SAN FRANCISCO (AP) \u2014 Tech behemoth OpenAI has touted its artificial intelligence-powered transcription tool Whisper as having near \u201chuman level robustness and accuracy.\u201d But Whisper has a major flaw: It &hellip; <a href=\"https:\/\/hotvideos24.online\/?p=118769\" class=\"more-link\">Read More<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-118769","post","type-post","status-publish","format-standard","hentry","category-business","entry"],"_links":{"self":[{"href":"https:\/\/hotvideos24.online\/index.php?rest_route=\/wp\/v2\/posts\/118769","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hotvideos24.online\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hotvideos24.online\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hotvideos24.online\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hotvideos24.online\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=118769"}],"version-history":[{"count":0,"href":"https:\/\/hotvideos24.online\/index.php?rest_route=\/wp\/v2\/posts\/118769\/revisions"}],"wp:attachment":[{"href":"https:\/\/hotvideos24.online\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=118769"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hotvideos24.online\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=118769"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hotvideos24.online\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=118769"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}