Is AI Filtering Just a New Form of Censorship?

Despite challenges in the film industry, "Wicked" has found success at the box office. Although the story was spoiled years ago in a children's book, the identity of the great wizard of Oz was revealed to be less than impressive in the film.

Behind the grand machinery was not a great wizard but a mere charlatan. However, the citizens of Emerald City accepted the oppressive policies against talking animals simply because they were deemed to be the decisions of the "great wizard." One can only imagine how shocked Elphaba was when she first learned the truth.

As we look ahead to the new regulations set to take effect on July 1, one might wonder if we have been swept away by a tornado into the land of Munchkin. Amendments to Article 22-5, Section 2 of the Telecommunications Business Act and Article 30-6 of its enforcement decree mandate that platforms of a certain size must use artificial intelligence (AI) to determine whether images uploaded by users contain illegal filming before allowing them to be posted.

This obligation, which has applied to videos since 2021, has now expanded to include images, screenshots, and photographs. While the intention to protect victims is commendable, one must question what we would call this if it were done by a human instead of AI. The answer would be "prior censorship."

The Constitution of South Korea explicitly prohibits prior censorship. The Constitutional Court has consistently ruled that it is unconstitutional for the government to review and filter content before it is made public. If officials were to scrutinize online posts individually, it would be difficult to avoid constitutional challenges.

However, a few months ago, the Constitutional Court ruled that the new system is not prior censorship because it mechanically compares the "characteristics" of already registered illegal filming materials without human review. Yet, the criteria for AI's judgment are ultimately set by humans. Just as the voice of the great wizard of Oz came from an ordinary person behind the machinery, if AI conducts the comparison, how is it different from the citizens of Emerald City who blindly believed in the existence of a great wizard?

Is it acceptable to replace the "censorship" that a human cannot perform with "filtering" done by AI? The structure of automatically reviewing and blocking content before publication is not fundamentally different.

Moreover, should we entrust AI with the judgment of right and wrong in the first place? One could argue that comparing characteristics is akin to making moral judgments. However, that is precisely what "judgment" entails: determining whether a subject meets given criteria. The process of a human assessing morality is not fundamentally different from AI filtering based on the criteria for "bad filming materials."

Even if humans set the criteria, the final judgment on whether individual cases meet those criteria ultimately falls to AI, meaning AI has already taken on some responsibility for moral judgments. The temptation of convenience always flows in one direction. What starts as a small concession to AI today could become a much larger one tomorrow.

Even if we concede that it is acceptable to delegate such judgments to machines, we have long designed our systems to ensure that judgments come with accountability. When a human makes a decision, there is an opportunity to contest that decision and hold them accountable for mistakes.

But if AI makes the judgment, to whom can we appeal, and who can be held responsible? There have already been reports of false positives, such as a baseball game being classified as illegal filming and iconic game moments being deleted. Some have referred to this as a "black box" where no one knows what gets filtered and why. Who can be held accountable for a "mechanical judgment" that is wrong? We must consider what it means to entrust judgment to an entity that cannot be held accountable as we approach this new system.

Of course, AI is not a fraud like the wizard of Oz, and I am not suggesting a long-outdated Luddite movement for the 21st century. I do not aspire to a fugitive's life like Elphaba. I acknowledge that AI is a powerful tool for quickly sorting through vast amounts of data that humans find difficult to manage.

However, using AI as a tool is entirely different from positioning it as a judge, and the logic that it is acceptable for AI to perform a role that is illegal for humans (while trying to make it act like a human) seems fundamentally flawed. When humans engage in censorship, it is called that; when AI does it, it is termed filtering. Does changing the name alter the essence? And under this new nomenclature, can we truly rely on AI to judge right and wrong so easily?