Flag to indicate AI generated content

[ryan-lp]

With the increased availability of both commercial and open source AI models that are capable of generating text, images, audio and transcripts, I'd like to start a discussion (and later, a concrete proposal) on flags we can introduce at various points in the spec to clearly label content that has been generated via AI.

Two key examples are:

• Podcast enclosures that were generated by AI rather than a human
• Transcripts that were generated by AI rather than a human

Use cases: Listeners may simply want to be informed of what they're getting, and whether it is authentic or generated, or to make a judgement of the quality or reliability of the content. Software developers also will want to build tools that can filter based on this information. For example, the average listener might want a search engine that can filter out podcasts generated by AI, or a deaf listener may want to filter their search results to that they only get shown news sources that come with high quality transcripts edited by real humans rather than AI generated transcripts (which in many non-English languages have atrocious quality.)

Other examples of content that may be generated by AI include chapters, titles, description and images. A few of these might be considered inconsequential, but it could still be informative to know if they were generated by an AI.

We might also want to indicate which AI produced the content. For example, whether the transcript was generated by Otter.ai or Whisper, or some other tool, rather than embed "Transcribed by Otter.ai" into the transcript itself(*).

(*) Which, it has to be said, ruins the transcript. Transcription services are adding the authorship or the generation tool as a regular line in the transcript itself because we currently don't have any other place to put it or any standard way to tag it. Someone might be searching for marine biology podcasts covering otters and probably doesn't want all podcasts transcribed by otter.ai to show up as matches.

If you have any thoughts on how this could be done (or whether this should be done) for each different type of content, please share below.

[theDanielJLewis]

Here's one proposal for indicating it on transcripts: #458.

[ryan-lp]

AI-generated content labels in the wild:

TikTok adopts AI-Generated content labels
YouTube Official Blog: Our approach to responsible AI innovation
EU wants Google, Facebook to start labeling AI-generated content
Instagram is working on labels for AI-generated content
Meta Working on Labels for AI-Generated Content: Report
‘Created By AI’ Warning Labels Are Coming To Social Media

[theDanielJLewis]

Since what we call the tag doesn't matter, I suggest <podcast:thisTextIsProbablyCrap>true</podcast:thisTextIsProbablyCrap>

🤣

[samsethi]

I think flagging content as AI will be helpful. In TrueFans we have extended the tag to experiment with having the host tagged as AI. Regarding the transcript isn't all transcript AI? But what is the user case? Should app developers offer an AI filter that hides any AI host?

[ryan-lp]

Regarding the transcript isn't all transcript AI?

I've talked with SBS in Australia who is a TV/radio/podcast broadcaster. Naturally, in their position they professionally transcribe all of their television programmes as captions for the hard of hearing, and they have been doing this since before AI was a thing. They do have a small number of podcasts with human-transcriptions but they do not use the podcast:transcript tag to publish them, and this was the motivation for my discussion with them. I am also subscribed to some science-based podcasts that have human created transcripts annotated with lesson notes, again not using the podcast:transcript tag, but they want to sell the transcripts + notes to help support their podcast (obviously we can think of better alternatives). I have also encountered podcasts with human-transcribed short episodes that embed the transcript directly in the item description.

Note also that AI transcripts are also terribly unfit for purpose in most languages, and when they do exist, human transcription is the only useful option.

But what is the user case?

There are a couple of general use cases that come to mind:

• For languages where AI transcripts are so awful as to be not fit for their purpose, AI generated content labels could help people who rely on transcripts to filter out podcasts that are essentially useless and unlistenable to them.
• We all want AI transcripts to get better, particularly in languages where they are of far lower quality. Many AI transcription models that are used to transcribe podcasts are actually trained on existing podcast transcripts, and what we will see as the number of AI-generated transcripts continues to increase is that it will be harder for these AI models to actually get any better, because they always need more and more data, and that data is getting harder to find amidst AI generated content. One case is OpenAI's open source Whisper model. They recently released a new v3 snapshot of their large multilingual transcription model in which they massively expanded the training dataset size (something that's important in deep learning), but they expanded it by including a lot of AI generated data that they obtained from the output of the previous v2 snapshot. The result of that experiment was an increase in hallucinations. Basically, labelling our podcast transcripts will help to improve the quality of future podcast transcripts.

I also built a free podcast transcription platform that can produce both machine and human transcriptions (which are basically machine transcripts that are then edited by humans), and for the system as a whole, such a label would be useful. I have not released it yet because it hinges on some proposals which have not yet been accepted. (The reason is a bit more complicated than that. You might think I should release it anyway, but in order for it to be sustainable as a free service, a number of pieces need to fall into place, and I don't have the power (or the health) to move those pieces.)