Urbanomic Machine Decision is Not Final

In anticipation of the publication of Machine Decision is Not Final, we present some short extracts from some of the texts in the collection.

From Wang Xin, ‘Machine Envy’

Since 2018, the social media platform Weibo has seen a curious rise in bots, by which I don’t mean automated zombie accounts for hire or coordinated troll armies, but rather, humans simulating bots by creating accounts with ‘bot-sounding’ handles. […] The ‘Lu Xun bot’ was named after one of modern China’s most formidable and incandescent writers (1881–1936), whose trenchant critique of the ills of imperialism and Confucian conservatism continues to resound in national textbooks and public discourse alike. His cultural and political ideas have been widely recited and, more crucially, promoted by the authorities as representative of their central values. One may argue that his work constitutes an essential data-set on which generations of modern Chinese people have been trained, making Lu Xun’s words uniquely potent and thorny when deployed to critique the social ills of today. […] One radical implication of the ‘Lu Xun bot’ is the future potential of bringing cultural luminaries back to ‘life’ using an advanced AI system. Although legendary artists such as the late Hibari Misora have been revived in stunning performances enabled by VOCALOID:AI, the prospect of AI enabled cultural criticism still feels distant, with its necropolitics, its ethics, and its ontology remaining rather murky. On the other hand, ‘Lu Xun bot’ feigns a sense of machine-induced objectivity and randomness, hence escaping the liabilities and political consequences that would be activated by a concession of human agency. Self-deprecating yet defiant, this satirical gesture belies a sense of resignation that speaks volumes about the actual state of human agency, which seeks anonymity and shelter through nonhuman camouflage in a mass surveillance state. As Chinese netizens make creative censorship circumvention a national pastime—one may argue this is one of the cutting edges of cultural production—and the swift and constant erasure of these strategies, data points, and new semantics make scaled, deep learning unfeasible, let alone history writing of any practical continuity or validity.

Since 2018, Weibo has seen a curious rise in humans simulating bots by creating accounts with ‘bot-sounding’ handles

One wonders not only how this zeitgeist might be captured or modelled, but indeed how it might also be transformed, as people intuitively adapt to coded speech in Chinese cyberspace. What is profoundly odd and ironic here is a palpable sense of machine envy, where advanced technology doesn’t necessarily embody the all too familiar tropes of servitude or existential threat, but rather, presents a viable, aspirational model of how to be. Artificiality not only feels more desirable but also more tangible than the real.

From Shuang Frost, ‘Translating Chinese AI: From “Human-made Intelligence” to “Black Tech”’

The Chinese term for Artificial Intelligence, rengong zhineng (人工智能), was originally introduced to China via Japan in the mid-twentieth century as a localisation of the Japanese term jinkō chinō (人工知能). Whereas the first element of the English compound ‘Artificial Intelligence’ clearly implies artificiality and perhaps even artifice—that AI is something alien to humanity—the Chinese and Japanese phrases harbour no such association. Rengong zhineng and jinkō chinō translate most faithfully as ‘human-made intelligence’. The first half of each double compound is composed of two characters 人, literally person or human, and 工, work or labour. These characters define AI as a product of human creation and link it to an entirely different constellation of associations. We can sense the effect of this shift in orientation in popular discourse. There is, for example, a Chinese saying which states that many of the products currently being marketed as AI involve more rengong, manual-labour, than zhineng, intelligence (e.g. many ‘AI-enabled technologies’ currently rely on armies of human workers to manually code data and process information instead of machine-learning algorithms).¹

Whereas the first element of the English compound ‘Artificial Intelligence’ clearly implies artificiality and perhaps even artifice, the Chinese and Japanese phrases harbour no such association

In a recent episode of Qipashuo, a popular Chinese debate show, contestants were presented with the following prompt: ‘If there existed a “black technology” that would enable a person to trade a portion of their life for money, ought we collectively permit it?’² […] In English the term ‘black technology’ evokes images of digital markets for untraceable transactions, omnipresent control systems, and other dystopic technologies placed in the service of immoral ends. But in the Sinosphere (the linguistic space of Chinese-language discourse), ‘black technology’ is a concept with overwhelmingly positive connotations. First popularised around 2016, the year that AlphaGo defeated Go master Lee Sedol in the legendary human-versus-machine game, the term refers to those transformative technologies—AI foremost among them—that are believed to hold the potential to radically reshape society in collectively desired ways. […] In order to make sense of this evocative term and its counter-intuitive associations, it is helpful to explore how ‘black technology’ was introduced into and was re-invented within the Chinese cultural context.

The phrase first appeared in Japanese as burakku tekunoroji ブラックテクノロジ, a Romanji rendering of the English words ‘black’ and ‘technology’. This compound was coined by Shoji Gatoh, author of Full Metal Panic!, a serial light novel and anime series.³ In the fictional world of Full Metal Panic!, burakku tekunoroji refers to a class of technologies so advanced that they exceed present human comprehension. These include both futuristic forms of real-world technologies, such as artificial intelligence, as well as mythical ones, such as the ‘Fairy’s Eye’ and the ‘Electronic Concealment System’.⁴ Gatoh juxtaposes burakku tekunoroji with the term haiteku ハイテク, from the English ‘high-tech’, to distinguish it from ordinary technologies that exist at or near the edge of the current technological frontier. Haiteku or high-tech is presentist in its orientation. In both English and Japanese, the phrase is generally invoked either in reference to the most advanced forms of technology presently available or to those technologies which have existed for many decades, but are just now beginning to find widespread application (e.g. carbon nanotubes, which were discovered in 1952, but have only recently been used to produce carbon fibre objects for mass-consumption). Burakku tekunoroji, by contrast, shifts the frame of reference away from the present and into the future. It is used to convey the incomprehensible power of unknowable, almost mystical technologies as well as their unbounded potential for creation and destruction. The Chinese term heikeji (黑科技), a direct translation of burakku tekunoroji, retains from its Japanese progenitor connotations of ineffable power but these are stripped of the corresponding existential concerns. As the 2018 annual report of the Chinese National Linguistic Committee (an entire section of which was dedicated to explicating the meaning of ‘black technology’) explained, the hei (black) in heikeji (black technology) falls outside of any existing meaning of the term ‘black’ in modern Chinese, which ranges from ‘dark’ and ‘hidden’ to ‘illicit’ and ‘evil’. On the contrary, it signals an intensely positive outlook towards the future; heikeji refers to conceptual technologies with immensely transformative potential that are ‘difficult to realise at the present moment but could be realised in the future’.⁵

From Vincent Garton, ‘Automaticity and the Mystery of State’

Artificial intelligence is the origin and limit of the genealogy of the state. The concepts of machine and state in their familiar form emerged simultaneously in European thought at the turn of the medieval and modern eras, reflecting in each case the ascendant concept of a universe governed by the interaction of mechanical forces. They attained a seminal joint articulation in Thomas Hobbes—at once the definitive theorist of the modern state and the reformulator of reason as calculation. It was his Leviathan, in 1651, that brought to bear a novel materialist theory of the cosmos to provide the first account of the state as a truly independent entity, an artificial person with its own particular machinery and a single will distinct from that of any of its members and its sovereign. […] Artificial man and mortal god: in a rehearsal of future science fiction, Hobbes’s state by its very definition is also an AI that dominates its creators.

[In the West] the [concept of the] state begins as a form of simulated artificial intelligence, dependent on a theologising conception of sovereignty, which serves as the mystery that continually sustains this simulation. The contemporary prospect of AI as a truly automatic machine system capable of self-sufficient reason transforms the simulation into actuality, but in doing so it abolishes the possibility of human sovereignty that is necessary to the functioning of the modern state. This prospective despotism of the algorithm demands a fundamental reassessment of the conceptual inventory of political thought that moves beyond the inherited [Western] concept of the state and its particular political theology. One alternative, I suggest, may be found by undertaking a great historical leap—to the theoretical framework of the 3rd century BC Han Feizi, an enduring classic of Chinese political thought.

[T]he ancient tradition of political thought in China lacks a concept of the state. Even in modern Mandarin Chinese, the words that customarily translate the term and designate its contemporary realisation are an uncomfortable fit. Guojia (国家), the most typical translation, is a nineteenth century repurposing, and its present usage bears the intellectual characteristics of that age, collapsing state into ‘nation’ and ‘country’ […]. Aside from its radically divergent philosophical presuppositions, Chinese political thought has in any case until very recently lacked a parallel to the fundamental concept of artificial personality that lies genealogically at the heart of the concept of the state, which in the West allowed the construction of that entity as a simulated artificial intelligence.⁶

In interpreting the political theories collected within the Han Feizi⁷ this has a radical consequence: they cannot, in any relevant sense, be said to have been oriented around ‘the state’. […] It is true that, abstracted from its context, the theoretical system of the Han Feizi can appear quite similar to that of Hobbes. Both articulated a vision of politics that broke decisively with previous moral concepts, resting upon an apparently materialistic conception of the world and taking as their political reality a world of relentless ambitions that must be restrained by force. Yet for all that, their thought remains grounded in radically different cosmologies, embodied in divergent articulations of the structure of political power. The incorporation within the Han Feizi of two commentaries on the classic attributed to Laozi, the Daodejing, demonstrates its central place. Mystery, to be sure, remains the ultimate basis: the Way is ‘what is eternal’, which ‘has neither a changing location nor a definite principle, and is not inherent in an eternal place. … The sage observes its mysterious emptiness [玄虚] and makes use of its comprehensive course.’⁸ But this is not the involuted trace of a separate sovereign God, hidden behind the world-machine: the mystery is the cosmos itself. […] The ruler function is not the mysterious sovereign of the Western state, the wielder of ineliminable excess whose responsibility it is to simulate the state as artificial intelligence. Rather, its objective in the Han Feizi is to abolish itself wholly transparently through the operation of the law that it sets in motion, becoming identified with the mystery of the Way itself. […] [T]he objective of the Han Feizi was not simply to inflate the power of a particular individual: it was precisely for the political community to ‘spontaneously do what is needed’ through the far-reaching identification of power, instantiated in the legal operation of reward and punishment, with the cosmic flux of the Way.⁹ Automaticity, then, is in fact the ultimate goal, actualised through the continual application of the law as a definite program.

If Zhang Yan is right that the purpose of the law in the age of AI is the coercion of the virtual world, conceived as AI in itself, the law may become a system for the generation of worlds

Zhang Yan’s recent consideration of the ‘concept of law in the age of artificial intelligence’ leads us further. Arguing on his own grounds that the advent of AI marks the end of the nation-state (民族国家) as the organising principle of politics, Zhang Yan opposes law and AI as principles of stability and experimentality, and notes that ‘the relationship between law and the state (国家) is bound to be revised’, since in the age of AI the law may be re-grounded in a power that escapes containment within the state.¹⁰ As the antithesis between the physical world and its ‘mirror’ world of virtuality is constituted ever more sharply through AI, law becomes a system of coercion of the virtual itself. The Han Feizi suggests that we may go beyond even this, identifying the law itself with AI. If Zhang Yan is right that the purpose of the law in the age of AI is the coercion of the virtual world, conceived as AI in itself, the law may become a system for the generation of worlds. There is, in this case, no center point of sovereignty—no mystery of state—merely a program of flux in which sovereignty itself is rendered irrelevant.

From Bogna Konior, ‘The Dark Forest Theory of Intelligence’

In Remembrance of the Earth’s Past, Liu Cixin’s first contact science fiction trilogy, extraterrestrials discover with surprise that for humans, ‘think’ and ‘say’ are not synonyms. In concealing information, humans have an unfair advantage because they can manipulate the expression of their thoughts: ‘it is precisely the expression of deformed thoughts that makes the exchange of information in human society…so much like a twisted maze’.¹¹ Human-level’ intelligence is then the ability to control the exchange of complex communication, especially by concealment. On the contrary, alien intelligence is described as radically explicit—the aliens communicate unreflexively and transparently, as if they were mere display technologies: ‘[they] do not have organs of communication, [their] brains display [their] thoughts to the outside world; thought and memories transparent like a book placed out in public, or a film projected in a plaza…totally exposed.’¹² Such exhibitionism of one’s reasoning processes is how the famous American thought experiments about AI have conceptualised computer intelligence: having it is showing it. From Alan Turing’s ‘imitation game’ to John Searle’s ‘Chinese Room,’ computer intelligence has been about demonstrating linguistic ‘ability’. Both Turing and Searle speculate that a computer might fluently converse with humans, but in neither of these thought experiments are computers presumed to lie. Computer communications are judged at face value as simply the best that a computer can do. Even though an intelligent computer should ‘be able to alter its own instructions’,¹³ it is not imagined as acting deceitfully or manipulatively. Throughout the history of AI thought experiments, it has been unusual to assume that an intelligent computer might appear unintelligent for its own purposes. Just like in the case of Liu’s aliens, computer intelligence is imagined as transparent—if it is there, it should communicate itself unreflexively, because a computer cannot decide to withhold its own abilities. Yet, Liu’s description of intelligence as the skillful use of communication as ‘trick, camouflage, deception’¹⁴ would also suggest that not all approaches to intelligence have to be exhibitionist. If intelligence is defined as deception, trickery, and camouflage, having it is hiding it […]. Could we then move from asking ‘how well could an intelligent computer communicate with humans’ to ‘why would an intelligent computer communicate its intelligence at all’?

If intelligence is defined as deception, trickery, and camouflage, having it is hiding it

From Anna Greenspan, ‘Mou Zongsan and AI Ethics’

Norbert Wiener, reflecting on the cultural consequences of the new science of communication and control in his last book God & Golem, Inc.: A Comment on Certain Points Where Cybernetics Impinges on Religion, warned that the literal-mindedness of cybernetic machines was analogous to the demonic dangers of magic. ‘The reprobation attaching in former ages to the sin of sorcery,’ he wrote, ‘attaches now in many minds to the speculations of cybernetics.’¹⁵ Wiener illustrates this by referring to Goethe’s poem ‘The Sorcerer’s Apprentice’ in which an inexperienced wizard conjures up a magical broom that can assist him with his chores. The young sorcerer activates his enchanted item, confident that he can relax, as he now has a tool that will obey all his commands. Yet, he is driven into a frenzied panic trying to stop it when the automated cleaner, for whom a room is never clean enough, starts drowning the house in streams of water. Wiener ponders worryingly the ominous idea that magic ‘grants you what you ask for, not what you should have asked for or what you intend.’¹⁶ He turns to the classic horror fable The Monkey’s Paw, in which a poor family is granted three wishes. They first wish for money, which comes in the form of insurance for their son who has tragically died. They next wish for their son to return, but when he comes home, it is only to haunt them as a ghost. Terrified, they use their final wish to banish their lost son’s phantom.¹⁷ Cybernetic machines, he warns, operate in a similar fashion: ‘Set a playing machine to play for victory, you will get victory [without] the slightest attention to any consideration except victory according to the rules.’¹⁸

What happens to questions of AI ethics if, following the lead of philosophers such as Yuk Hui, we shift perspective and employ an altogether different cosmological approach?

As the title of his book suggests, these morality tales that trouble Wiener, intimately entangled with questions of discipline and restraint, are embedded in a very specific philosophical and conceptual domain—that of Abrahamic religion. In the Abrahamic religions, from the moment Adam and Eve eat the apple, the problem of knowledge and disobedience are intertwined. For Weiner, the resulting dilemma of how to balance intelligence and control is a feature, not a bug of monotheistic tradition. But what happens to questions of AI ethics if, following the lead of philosophers such as Yuk Hui, we shift perspective and employ an altogether different cosmological approach?¹⁹

During my fieldwork in China, I have often heard industry experts invoke the saying ‘rengong zhineng doushi rengong, meiyou shenme zhineng’ (人工智能都是人工，没有什么智能), which translates as ‘Artificial Intelligence [Rengong zhineng] is all human labour [rengong] and no intelligence [zhineng]’. Such playful language is deeply revealing. It shows how the concept of rengong zhineng in Chinese is intrinsically centered on the human. […].
Qipashuo is a TV program produced by Ma Dong, founder and CEO of Mewe. seven seasons spanning 2014–2021 have been aired. The show has become a cultural phenomenon in China: it produced 1,179 trending phrases that are rated ‘top searches’ on the Chinese internet. And its viewership attained No. 1 in language-related shows and top 3 in general entertainment shows. Data from ‘Data Will Tell You: The Cultivation of Qipashuo in Seven Years’ [数据告诉你,《奇葩说》的七年之’养’], QQ, 9 March 2021.
フルメタル・パニック! [Full Metal Panic!], written by Shoji Gatoh and illustrated by Shiki Douji, published between 1998 and 2011.
In the series, haiteku were created by a sub-race of humans, known as the ‘Whispered’, who were created as a result of scientific experimentation and whose existence is unknown to the general population. Some Haiteku later entered the world and became commonplace without civilians knowing their origins as a ‘black technology’.
National Language Working Committee [国家语言文字工作委员], ‘Report on Chinese Linguistic Life, 2018 [中国语言生活状况报告2018].
Fang Liufang (方流芳), ‘“Faren” jinru dangdai Zhongguo falü, yiyi hezai?’ ‘法人’进入当代中国法律，意义何在?, Zhongguo falü pinglun 中国法律评论 6 (December 2019), 154–62.
I prescind here from examining the composition of the Han Feizi in more detail. For a general account of the problem, Bertil Lundahl, Han Fei Zi: The Man and the Work (Stockholm: Institute of Oriental Languages, 1992). As Peter Moody notes, different parts of the Han Feizi make contradictory political assertions: Peter R. Moody, ‘Han Fei in His Context: Legalism on the Eve of the Qin Conquest’, Journal of Chinese Philosophy 38:1 (March 2011), 14. Nevertheless, they proceed from a more or less coherent set of theoretical assumptions.
Sarah A. Queen, ‘Han Feizi and the Old Master: A Comparative Analysis and Translation of Han Feizi Chapter 20, ‘Jie Lao’, and Chapter 21, ‘Yu Lao’, in Dao Companion to the Philosophy of Han Fei, ed. Paul R. Goldin (Dordrecht: Springer, 2013), 241; Han Feizi 韩非子, ed. Gao Huaping (高华平), Wang Qizhou (王齐洲) and Zhang Sanxi (张三夕) (Beijing: Zhonghua shuju, 2010), 34, 36.
Han Feizi, ed. Gao et al., 59.
Zhang Yan (张龑), ‘Rengong zhineng shidai de fa gainian’ 人工智能时代的法概念, Aisixiang 爱思想, May 26, 2019.
Liu Cixin, The Dark Forest, tr. Joel Martinsen (New York: Tor, 2015) epub.
Ibid.
Turing here stresses acting intelligently rather than having intelligence. The Essential Alan Turing, ed. B J Copeland (Oxford: Oxford University Press), 375.
Liu, The Dark Forest.
N. Wiener, God & Golem, Inc.: A Comment on Certain Points Where Cybernetics Impinges on Religion (Cambridge, MA: MIT Press, 1964), 60.
Wiener, God & Golem, 71.
In his essay ‘Y2K Positive’, Mark Fisher describes this literalness with reference to the millennial computer glitch Y2K. M. Fisher, ‘Y2K Positive’, Mute (January 2004).
Wiener, God & Golem, 71
See Yuk Hui, ‘On the Limit of Artificial Intelligence’, Philosophy Today 65 (2):339–357 and The Question Concerning Technology in China (Falmouth: Urbanomic, 2016). Also see Bing Song, ‘Introduction: How Chinese Philosophers Think About Artificial Intelligence?’ in Intelligence and Wisdom (Singapore: Springer, 2021), 1–14.

UFD0057

Document

China and AI: Human Bots, Black Tech, the Dark Forest, and the State

Bogna Konior

Anna Greenspan

Vincent Garton

Shuang Frost

Wang Xin

RELATED

publications

events

Publications

Machine Decision is Not Final

Benjamin Bratton (ed)

Bogna Konior (ed)

Anna Greenspan (ed)

Events

Machine Decision is Not Final for SVBKVLT x Umanesimo Artificiale