Humans As Software Extensions
In 2008, the science-fiction movie Sleep Dealer by Alex Rivera envisioned a future which couldn’t be more timely: the border between Mexico und the US has been closed. Therefore, immigrant workers in the US have been replaced by robots. However, these robots are remotely controlled by people in Mexico who have their bodies plugged directly into the network.
Two years later, in 2010, Crowdflower CEO Lukas Biewald speaks of a similar situation:
Before the Internet, it would be really difficult to find someone, sit them down for ten minutes and get them to work for you, and then fire them after those ten minutes. But with technology, you can actually find them, pay them the tiny amount of money, and then get rid of them when you don’t need them anymore.
Biewald’s remarks, however, were not science-fiction. Instead, at that time they described a reality for some that has since become a reality for many: a contemporary condition and configuration that I would call humans as software extensions. In this configuration, people are extending computational systems by offering their bodies, their senses, and their cognition; and specifically, bodies and minds that can be easily plugged in, rewired, and discarded. These are bodies and minds that are algorithmically managed and under the permanent pressure of constant availability, efficiency and perpetual self-optimization.
Being Universally Addressable and Programmable
As such, humans as software extensions are both the foundation and the result of something that we could call a computational megastructure. Benjamin Bratton offers a helpful model of such a megastructure which he calls The Stack. This model describes a computational totality of planetary-scale that is comprised of six interdependent layers: earth, cloud, city, address, interface, and user; from rare-earth minerals to fingers touching screens to bots living somewhere inside this global stack.
What makes Bratton’s model interesting is his proposition to consider all machines, software, etc. that surround us, and that we inhabit, not as separate but as forming a whole, even if that whole has come about accidentally. This totality both describes and generates a new geography, e.g. the Costa Rica-Nicaragua border dispute that was fueled by Google Maps in 2010, or potentially the scenario described in Sleep Dealer.
As Bratton characterizes users as expressions of platforms (the stack of layers), the Stack can also be considered a model that generates humans as software extension. A user is that which can initiate a so-called column inside the stack. Hence, a user can be a human, a bot, a car etc. For example, a message sent from one user to another travels from the sender at the top to the bottom of all layers, and then back up again to the receiver. Coming into existence as parts of columns that pass through the address and interface layers, all users inside the Stack, including humans as software extensions, are universally addressable, and programmable.
To be clear, from my point of view the model of planetary-scale computation as a totality is as much a reality – and not, at the same time, as Bratton points out – as it is also used as a gigantic fantasy and ideology of power, optimization, and efficiency.
Let me give you a few examples of what I mean by humans as software extensions, and what effects this way of managing digital labor has on us.
Managing My Extensions
In 2015 I published a piece in which I explored digital colonialism, and among other things Google’s and Facebook’s attempt to integrate into their services those 2/3 of the world’s population who are not online yet. Using drones and balloons circling above areas without internet connection, Google and Facebook want to “suck” that which is below up into the network.
The piece which is called How to Appear Offline Forever consist of a mix of found material – videos, images, and questions. There is also a layer of stories written and sometimes recorded by people from Silicon Valley, Sri Lanka, and Zambia which are all locations of importance in the piece’s story (Zambia is the country in which Facebook’s Internet.org premiered, and Sri Lanka was meant to be the first country to be served by Google Loon).
In order to get in contact with people in Sri Lanka, I ended up using the outsourcing platform Upwork which offers a highly efficient interface for hiring freelancers from all around the world. On their platform you can sort freelancers by price, skills, and ratings. Their user experience of hire & fire is well crafted, offering freelancers as software extensions that can be plugged in, and removed again easily. It doesn’t matter where, what or who they are – as long as they get it done.
Once the freelancers in Sri Lanka got to work, I noticed that I was not only able but also encouraged to spy on them: Upwork records key strokes and regularly takes screenshots while freelancers work, building a growing diary of their activity. I found myself in a situation in which I wasn’t only being surveilled by states or corporations. I was also doing the same myself, managing my extensions, spying on them in order to monitor their performance.
This is not them spying on us, us protecting ourselves against them through encryption. This is all of us fighting for our place in the network, trying to be valuable nodes.
Only 15 years ago, outsourcing via the Internet was a practice that could only be employed by big IT companies. Today, it is cheap and easy and can be done by anybody. A new service by Amazon called Amazon Key illustrates this rather new situation perfectly:
With Amazon Key, you can remotely grant access to your apartment, and using the Amazon Key camera, lock, and app, you can spy on the otherwise completely invisible workers from your smartphone. In Amazon’s promotional video, outsourcing doesn’t generate free time. Instead, it is born out of necessity. Implicitly, Amazon Key is also marketed as making possible the transformation from being managed to being able to also manage others – while not having to deal with them in person at all.
Now, everybody not only can but has to – and of course: wants to – employ humans as software extensions, at the same time having to remotely track and rate their performance.
The Total Freedom of Survival Creativity
Greatly simplifying a complicated situation and development, we might say that while factory workers were and still are extending machines with their bodies, freelancers could escape the factory. Yet, they ended up as “flexible” extensions to the modern media assembly line through platforms like Upwork.
Now, in a more recent development, the alleged independence and freedom of entrepreneurs (or more precisely: gigtrepreneurs) turned out to be a diktat under which people have to invent their own jobs. Trying to anticipate possible ways in which they can extend temporary global assembly lines (or, in the framework of the Stack: configurations of columns), gigtrepreneurs package and offer their creativity in the form of so called gigs. Such gigs are sold on platforms like Fiverr for a fixed price (initially, each gig on Fiverr was priced at $5 with the platform keeping 20%).
Some months ago, I found a way to directly access all videos that are uploaded to Fiverr in real time, including every single video that people working on the platform are producing for their clients. Through this crack in the platform’s surface, I’ve been looking at the leaked stream of videos for days and weeks, downloading over 100 gigabyte of video.
Let me share some observations. On Fiverr it is dog-eat-dog: be the best, the cheapest, the most creative, and the most efficient. At the same time, everybody is also fighting against the platform’s algorithms and clean interface that hide most gigs on page two, three or four, no matter how cheap, creative or efficient these gigs are.
As many gigs offer unrealistically short delivery times for creative work, it also becomes clear that the gigtrepreneurs themselves use bots, generators, and templates, simulating creative work, creating yet another layer of man-machine-complexity in which they are using automation in order to not be replaced by automation. Doing so, their biggest selling point seems to be their low cost coupled with a truly natural interface, a human being.
In contrast, there is another group of people consciously offering their bodies, often as carriers of messages. Here, the fantasy of the universal addressability, programmability and availability of all nodes manifests itself in the distant and often “exotic” body that acts as a screen. This goes hand in hand with gigs offering personalized porn and fetish videos.
Here we can summarize: being a human software extension on a hyper-competitive platform fosters and demands something that I would call survival creativity: doing whatever it takes to survive in a competitive environment.
Let me remind you that Fiverr might be an extreme example but it exemplifies a development that has become a reality for many already. Furthermore, don’t be mistaken, there is no we and them. We are all human software extensions.
The Future of Work is a Lot of Work
So far, I’ve drawn this bleak picture of a quasi-totality of work and exploitation. In the previous examples, platforms, software, and artificial intelligence acted as “scientific management”, as the taylorist boss algorithmically distributing and modulating human workers as software extensions. However, don’t automation and artificial intelligence supposedly bring about a future without work? According to a rather popular concept of a technophile post-work society, this could be achieved by not only automating the management of working bodies and minds, but also completely replacing all human nodes with artificial intelligence as well.
While I think this is an excellent foundation for discussing our society’s obsession with work, I would argue that this accelerationist concept is as appealing as it is flawed (unfortunately). Artificial intelligence is an appropriation and a possible extrapolation of existing knowledge and skills, and as such it might as well do (some of) our jobs. But it is first and foremost a scheme to fragment work into tasks that can be done anywhere 24/7, and to make this labor invisible.
Artificial intelligence’s recent awakening from hibernation cannot be explained by better algorithms alone. Next to big data, the new possibility of outsourcing clickwork via the internet on a massive scale for little or no money has probably been the most important factor in making possible this recent development. Here, machine vision is a good example that allows us to understand artificial intelligence as a global infrastructure that has a massive but mostly invisible workforce at its core.
Like many other machine vision datasets (e.g. ImageNet), Microsoft’s COCO image recognition dataset is based on photos culled from Flickr. The dataset’s true value derives from the 600.000+ segmentations manually created by Mechanical Turk crowd workers, based on those photos from Flickr. With Segmentation.Network I created a piece in which all segmentations are played back in a random order, making visible some of the manual labor that goes into building artificial intelligence. Furthermore, it is also a piece about absence and irrelevance. Looking at the outlines, it becomes obvious that in each photo only a specific set of objects has been annotated and segmented while everything else has been deemed irrelevant. Therefore, each dataset can also be defined by its incompleteness.
Against this background, we have to realize that we can automate as much as we want, we will still have to train and maintain machines and software constantly. Thus, artificial intelligence creates yet another layer of badly or unpaid care and maintenance work which is often invisible on purpose.
(Of course, this needs to change, and feminist theory and practices have a lot say about this issue, as does for example maintenance art).
Seizing the Means of Magic
Here, a case in point and, maybe, a possible solution: At the end of 2011, while still being students sharing a studio, Silvio Lorusso and I started to take a screenshot of every single captcha that we had to solve while navigating the web. Over the years, proving that we are human time and time again, we captured hundreds of captchas, chronicling years of micro-labor as well as a history of captchas: from a technique to merely prevent spam to a method for deciphering house numbers and transcribing books to a means of teaching image recognition to AI software.
While we were still just collecting, in 2015, Gabriela Rojas-Lozano filed a class action lawsuit against Google. She claimed that Google “operates a highly profitable transcription business built upon free labor, which it deceptively and unfairly obtains from unwitting website users”. Unfortunately, her claims were rejected on the grounds that “Plaintiff has not alleged any facts that plausibly suggest the few seconds it takes to type a second word is something for which a reasonable consumer would expect to receive compensation.”
However, her attempt to sue Google was still a success because it lead to the proof that Google has perfected a magical process in which work is transformed into literally nothing. Welcome Post-Work Society, after all!
How does this magical process work? It is rather simple: You take a job – let’s say transcribing books or the New York Times archive – and you fragment this job, and you fragment it more and more until, suddenly, the job is magically done without anyone ever having worked on it! Because, if nobody has to get paid then nobody had to work either, right? Hence, the judge’s statement is proof that this magic actually works. It gets better: Google still ends up being paid even though there is no job anymore.
What I want to suggest now is to seize the means of magic. Why not fragment those platforms that algorithmically manage us to such a degree that they simply do not exist anymore? Magically, their job will still be done, and in the end we get the money (I guess we could call a less magical version of this platform cooperativism).
By the way, Silvio and I have since published our complete collection of captured captchas as a series of five handmade leporello books, and the books span a total length of 90 meters.
Covert Interventions
If you do not believe in magic then I have two more suggestions about what we as software extensions can do.
In 2016, Donald Trump’s team hired a Singaporean teenager through Fiverr (the platform I talked about earlier) to convert a PowerPoint into a Prezi, basically outsourcing the “Make America Great Again” campaign. In my recent speculative video piece I Will say Whatever You Want In Front Of A Pizza, the protagonist who is not only working as a pizza delivery bot but also as a cloud-worker on Amazon’s Mechanical Turk platform gets to know this Singaporean teenager.
And, at least in my speculative video, she’s got an idea. The idea is this: when we are extending software with our bodies and minds, we are also extending our reach into the software. And reaching into the software, being part of the software, we can start to manipulate these systems that govern us, and with which we have to govern others. Once we are plugged in, we can manipulate data, and we can create new and weird and slow and inefficient software from within. It can be fun, like leaving easter eggs for others to find, realizing, yes, there are actual people inside these systems.
An Aesthetic of Detachment
Finally, I will end by making a fool of myself, talking about Mark Zuckerberg. Being a software extension can also offer a new aesthetic, and a new way of being, and I think this video which could be called the father of all stupid demos illustrates this in a rather interesting way:
Here, for whatever reason, Zuckerberg demonstrates Facebook’s VR by visiting Puerto Rico in the wake of hurricane Maria. Contrary to what he had intended, Zuckerberg as a crudely abstracted version of himself, turned into a software extension, detaches and dissociates himself from the reality that he attempts to visit and connect to. This is what I like, and I think you will agree with me: software is not perfect, it’s full of bugs, it often behaves in unexpected and weird and glitchy ways, doing stupid things, often over and over again in an infinite loop.
Therefore, embracing the weird and abstract aesthetic of being a software extension could actually allow us to detach ourselves from circumstances under which we are required to be our best working selves all the time. Pretending to be a bot could be a mask behind which we can hide.
Sebastian Schmieg, 2018
This text is a slightly edited version of the talk I gave at #34c3 in Leipzig on December 28, 2017