The Robo-Journalists Are Here – How Are They Doing?

After years of being relegated to reporting sports recaps, corporate earnings, and data mining, AI is playing a bigger part in news generation. The first news articles to be produced entirely by generative AI have hit the proverbial newsstands — and the results are mixed.

The enthusiasm around tools like ChatGPT may have led editors to jump the gun, unleashing AI on articles before they are ready. In this article, we explore how publishers can harness the undoubted productivity of these new tools — and overcome their limitations.

The limitations of generative AI

Unsurprisingly, given the limitations of large language models (explored in our recent post), the first outings of the AI authors have run into difficulties. Last year, CNET quietly started publishing AI-generated articles without disclosing that machines authored the pieces. Web sleuths quickly figured it out and forced the publication to admit it relied on AI — and, despite the editor-in-chief claiming each article was reviewed, simple mistakes made it through. For instance, The Washington Post reports, “An automated article about compound interest, for example, incorrectly said a $10,000 deposit bearing 3 percent interest would earn $10,300 after the first year. Nope. Such a deposit would actually earn just $300.”

CNET wasn’t the only site owned by Red Ventures to find itself in hot water over AI. “Two more of its subsidiary sites, Bankrate and CreditCards.com, were also regularly publishing (erroneous) AI-written articles for a while,” according to Slate.

Despite these troubles, AI-generated news stories continue to appear — most notably NewsGPT, a news site consisting of entirely AI-authored pieces. Of course, NewsGPT has been met with criticism. Its creator claims that because the site doesn’t rely on reporters it lacks bias; a claim many find dubious.

As Maggie Harrison points out on Futurism, NewsGPT has not revealed what AI model it uses: “AI software doesn't just spring into existence. Models are conceptualized, built, and programmed by humans, and disclosing which humans are making the underlying tech seems like it should be pretty important to Levy's alleged mission.”

Moreover, nearly every generative AI engine has been known to not only make mistakes but to make things up. “They don't know what words mean, they just predict what might come next in a sentence, even making up phony sources and numbers to support BS claims,” Harrison added. This is especially problematic for a media outlet that aims to provide “fact-based” news.

AI endangers reader trust

There is already a trust problem with today's news media. The NewsGPT creators may think AI is the solution, but the ability to get the facts right is essential to establishing trust. Journalists have a responsibility to fact-check their work. From relying on multiple sources to verify facts to employing actual fact-checkers to essentially re interview sources, journalists take accuracy seriously. AI, on the other hand, is only as good as the data sets it can draw on. When that data set is the entire internet, it will inevitably get things wrong. And when an algorithm gets something wrong, it's unclear who is to be held accountable.

And then there is the plagiarism problem. As we recently explored, there are several legal challenges facing generative AI. At the heart of the matter are concerns about plagiarism. Generative AI requires massive data sets to train on, and draws on that data to generate answers to queries — or, as the case may be, write articles. Not only does this mean that AI inherently can’t break new news, but it can also lead to plagiarism of its sources.

Harnessing the power of generative AI for media

All concerns aside, generative AI can enhance the way journalists do their jobs and allow media outlets to add new, engaging features. For instance, The New York Times used Chat GPT to create a Valentine’s Day message generator. This kind of fun, low-stakes use is perfect for media outlets looking to engage readers without expending too many resources.

In a recent report, Reuters talked to several experts about the role generative AI will play in journalism, and the consensus is that its role will be limited. Over the past decade or so AI has fallen into one of three categories, according to Francesco Marconi, computational journalist and co-founder of the real-time information company AppliedXL. Those categories are automation, augmentation, and generation.

Newsrooms have been using — and will continue to use — AI to automate and augment processes. From combing through massive data sets to transcribing calls, AI has been a boon to many journalists. However, as Madhumita Murgia, AI editor at the Financial Times, told Reuters, when it comes to actual reporting, AI just doesn’t have what it takes: “Based on where it is today, it's not original. It's not breaking anything new. It’s based on existing information. And it doesn't have that analytic capability or the voice.”

AI in the newsroom - in practice

At a practical level the Insider news portal is taking a measured, pragmatic approach to AI. The global editor-in-chief Nicholas Carlson told Axios, “A tsunami is coming. We can either ride it or get wiped out by it." With this in mind, Insider is establishing a working group to find responsible ways to incorporate AI into its newsroom. In the meantime, “the rest of the newsroom will be encouraged to use AI to generate outlines for stories, fix typos, craft headlines optimized for search engines, and prep interview questions. They are discouraged from putting sensitive information, particularly sourcing details, into ChatGPT.”

Carlson emphasized that the human editor will always play a key role in the news creation process. "Even if Insider were to eventually get to a point where an entire article is written by AI, it will still be required to go through the same editing process that human-created work undergoes."

"Nothing we publish, will not be vetted and signed off on by a meticulous fact-checking human editor," he said.

AI integration in Eidosmedia workflows

Eidosmedia is actively exploring the integration of AI-powered tools and services into its authoring and publishing workflows.

Current integrations include automatic print pagination, instant podcast generation and NLP services for financial publishing.