What is AI? Everything you need to know about Artificial Intelligence


An executive guide to artificial intelligence, from machine learning and general AI to neural networks.

What is artificial intelligence (AI)?

It depends who you ask.

AI might be a hot topic but you’ll still need to justify those projects.

Back in the 1950s, the fathers of the field Minsky and McCarthy, described artificial intelligence as any task performed by a program or a machine that, if a human carried out the same activity, we would say the human had to apply intelligence to accomplish the task.

That obviously is a fairly broad definition, which is why you will sometimes see arguments over whether something is truly AI or not.

AI systems will typically demonstrate at least some of the following behaviors associated with human intelligence: planning, learning, reasoning, problem solving, knowledge representation, perception, motion, and manipulation and, to a lesser extent, social intelligence and creativity.

What are the uses for AI?

AI is ubiquitous today, used to recommend what you should buy next online, to understand what you say to virtual assistants such as Amazon’s Alexa and Apple’s Siri, to recognise who and what is in a photo, to spot spam, or detect credit card fraud.

What are the different types of AI?

At a very high level artificial intelligence can be split into two broad types: narrow AI and general AI.

Narrow AI is what we see all around us in computers today: intelligent systems that have been taught or learned how to carry out specific tasks without being explicitly programmed how to do so.

This type of machine intelligence is evident in the speech and language recognition of the Siri virtual assistant on the Apple iPhone, in the vision-recognition systems on self-driving cars, in the recommendation engines that suggest products you might like based on what you bought in the past. Unlike humans, these systems can only learn or be taught how to do specific tasks, which is why they are called narrow AI.

What can narrow AI do?

There are a vast number of emerging applications for narrow AI: interpreting video feeds from drones carrying out visual inspections of infrastructure such as oil pipelines, organizing personal and business calendars, responding to simple customer-service queries, co-ordinating with other intelligent systems to carry out tasks like booking a hotel at a suitable time and location, helping radiologists to spot potential tumors in X-rays, flagging inappropriate content online, detecting wear and tear in elevators from data gathered by IoT devices, the list goes on and on.

What can general AI do?

Artificial general intelligence is very different, and is the type of adaptable intellect found in humans, a flexible form of intelligence capable of learning how to carry out vastly different tasks, anything from haircutting to building spreadsheets, or to reason about a wide variety of topics based on its accumulated experience. This is the sort of AI more commonly seen in movies, the likes of HAL in 2001 or Skynet in The Terminator, but which doesn’t exist today and AI experts are fiercely divided over how soon it will become a reality.

A survey conducted among four groups of experts in 2012/13 by AI researchers Vincent C Müller and philosopher Nick Bostrom reported a 50 percent chance that Artificial General Intelligence (AGI) would be developed between 2040 and 2050, rising to 90 percent by 2075. The group went even further, predicting that so-called ‘ superintelligence‘ — which Bostrom defines as “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest” — was expected some 30 years after the achievement of AGI.

That said, some AI experts believe such projections are wildly optimistic given our limited understanding of the human brain, and believe that AGI is still centuries away.

What is machine learning?

There is a broad body of research in AI, much of which feeds into and complements each other.

Currently enjoying something of a resurgence, machine learning is where a computer system is fed large amounts of data, which it then uses to learn how to carry out a specific task, such as understanding speech or captioning a photograph.

What are neural networks?

Key to the process of machine learning are neural networks. These are brain-inspired networks of interconnected layers of algorithms, called neurons, that feed data into each other, and which can be trained to carry out specific tasks by modifying the importance attributed to input data as it passes between the layers. During training of these neural networks, the weights attached to different inputs will continue to be varied until the output from the neural network is very close to what is desired, at which point the network will have ‘learned’ how to carry out a particular task.

A subset of machine learning is deep learning, where neural networks are expanded into sprawling networks with a huge number of layers that are trained using massive amounts of data. It is these deep neural networks that have fueled the current leap forward in the ability of computers to carry out task like speech recognition and computer vision.

Download now: IT leader’s guide to deep learning

There are various types of neural networks, with different strengths and weaknesses. Recurrent neural networks are a type of neural net particularly well suited to language processing and speech recognition, while convolutional neural networks are more commonly used in image recognition. The design of neural networks is also evolving, with researchers recently refining a more effective form of deep neural network called long short-term memory or LSTM, allowing it to operate fast enough to be used in on-demand systems like Google Translate.

ai-ml-neural-network.jpg
The structure and training of deep neural networks.

Image: Nuance

Another area of AI research is evolutionary computation, which borrows from Darwin’s famous theory of natural selection, and sees genetic algorithms undergo random mutations and combinations between generations in an attempt to evolve the optimal solution to a given problem.

This approach has even been used to help design AI models, effectively using AI to help build AI. This use of evolutionary algorithms to optimize neural networks is called neuroevolution, and could have an important role to play in helping design efficient AI as the use of intelligent systems becomes more prevalent, particularly as demand for data scientists often outstrips supply. The technique was recently showcased by Uber AI Labs, which released papers on using genetic algorithms to train deep neural networks for reinforcement learning problems.

Finally there are expert systems, where computers are programmed with rules that allow them to take a series of decisions based on a large number of inputs, allowing that machine to mimic the behavior of a human expert in a specific domain. An example of these knowledge-based systems might be, for example, an autopilot system flying a plane.

What is fueling the resurgence in AI?

The biggest breakthroughs for AI research in recent years have been in the field of machine learning, in particular within the field of deep learning.

This has been driven in part by the easy availability of data, but even more so by an explosion in parallel computing power in recent years, during which time the use of GPU clusters to train machine-learning systems has become more prevalent.

Not only do these clusters offer vastly more powerful systems for training machine-learning models, but they are now widely available as cloud services over the internet. Over time the major tech firms, the likes of Google and Microsoft, have moved to using specialized chips tailored to both running, and more recently training, machine-learning models.

An example of one of these custom chips is Google’s Tensor Processing Unit (TPU), the latest version of which accelerates the rate at which useful machine-learning models built using Google’s TensorFlow software library can infer information from data, as well as the rate at which they can be trained.

These chips are not just used to train up models for DeepMind and Google Brain, but also the models that underpin Google Translate and the image recognition in Google Photo, as well as services that allow the public to build machine learning models using Google’s TensorFlow Research Cloud. The second generation of these chips was unveiled at Google’s I/O conference in May last year, with an array of these new TPUs able to train a Google machine-learning model used for translation in half the time it would take an array of the top-end graphics processing units (GPUs).

What are the elements of machine learning?

As mentioned, machine learning is a subset of AI and is generally split into two main categories: supervised and unsupervised learning.

Supervised learning

A common technique for teaching AI systems is by training them using a very large number of labeled examples. These machine-learning systems are fed huge amounts of data, which has been annotated to highlight the features of interest. These might be photos labeled to indicate whether they contain a dog or written sentences that have footnotes to indicate whether the word ‘bass’ relates to music or a fish. Once trained, the system can then apply these labels can to new data, for example to a dog in a photo that’s just been uploaded.

This process of teaching a machine by example is called supervised learning and the role of labeling these examples is commonly carried out by online workers, employed through platforms like Amazon Mechanical Turk.

Training these systems typically requires vast amounts of data, with some systems needing to scour millions of examples to learn how to carry out a task effectively — although this is increasingly possible in an age of big data and widespread data mining. Training datasets are huge and growing in size — Google’s Open Images Dataset has about nine million images, while its labeled video repository YouTube-8M links to seven million labeled videos. ImageNet, one of the early databases of this kind, has more than 14 million categorized images. Compiled over two years, it was put together by nearly 50,000 people — most of whom were recruited through Amazon Mechanical Turk — who checked, sorted, and labeled almost one billion candidate pictures.

In the long run, having access to huge labelled datasets may also prove less important than access to large amounts of compute power.

In recent years, Generative Adversarial Networks ( GANs) have shown how machine-learning systems that are fed a small amount of labelled data can then generate huge amounts of fresh data to teach themselves.

This approach could lead to the rise of semi-supervised learning, where systems can learn how to carry out tasks using a far smaller amount of labelled data than is necessary for training systems using supervised learning today.

Unsupervised learning

In contrast, unsupervised learning uses a different approach, where algorithms try to identify patterns in data, looking for similarities that can be used to categorise that data.

An example might be clustering together fruits that weigh a similar amount or cars with a similar engine size.

The algorithm isn’t setup in advance to pick out specific types of data, it simply looks for data that can be grouped by its similarities, for example Google News grouping together stories on similar topics each day.

Reinforcement learning

A crude analogy for reinforcement learning is rewarding a pet with a treat when it performs a trick.

In reinforcement learning, the system attempts to maximize a reward based on its input data, basically going through a process of trial and error until it arrives at the best possible outcome.

An example of reinforcement learning is Google DeepMind’s Deep Q-network, which has been used to best human performance in a variety of classic video games. The system is fed pixels from each game and determines various information, such as the distance between objects on screen.

By also looking at the score achieved in each game the system builds a model of which action will maximize the score in different circumstances, for instance, in the case of the video game Breakout, where the paddle should be moved to in order to intercept the ball.

Many AI-related technologies are approaching, or have already reached, the ‘peak of inflated expectations’ in Gartner’s Hype Cycle, with the backlash-driven ‘trough of disillusionment’ lying in wait.

Image: Gartner / Annotations: ZDNet

Which are the leading firms in AI?

Google’s DeepMind and the NHS: A glimpse of what AI means for the future of healthcare

The Google subsidiary has struck a series of deals with organisations in the UK health service — so what’s really happening?

With AI playing an increasingly major role in modern software and services, each of the major tech firms is battling to develop robust machine-learning technology for use in-house and to sell to the public via cloud services.

Each regularly makes headlines for breaking new ground in AI research, although it is probably Google with its DeepMind AI AlphaGo that has probably made the biggest impact on the public awareness of AI.

Which AI services are available?

All of the major cloud platforms — Amazon Web Services, Microsoft Azure and Google Cloud Platform — provide access to GPU arrays for training and running machine learning models, with Google also gearing up to let users use its Tensor Processing Units — custom chips whose design is optimized for training and running machine-learning models.

All of the necessary associated infrastructure and services are available from the big three, the cloud-based data stores, capable of holding the vast amount of data needed to train machine-learning models, services to transform data to prepare it for analysis, visualisation tools to display the results clearly, and software that simplifies the building of models.

These cloud platforms are even simplifying the creation of custom machine-learning models, with Google recently revealing a service that automates the creation of AI models, called Cloud AutoML. This drag-and-drop service builds custom image-recognition models and requires the user to have no machine-learning expertise.

Cloud-based, machine-learning services are constantly evolving, and at the start of 2018, Amazon revealed a host of new AWS offerings designed to streamline the process of training up machine-learning models.

For those firms that don’t want to build their own machine learning models but instead want to consume AI-powered, on-demand services — such as voice, vision, and language recognition — Microsoft Azure stands out for the breadth of services on offer, closely followed by Google Cloud Platform and then AWS. Meanwhile IBM, alongside its more general on-demand offerings, is also attempting to sell sector-specific AI services aimed at everything from healthcare to retail, grouping these offerings together under its IBM Watson umbrella — and recently investing $2bn in buying The Weather Channel to unlock a trove of data to augment its AI services.

Which of the major tech firms is winning the AI race?

Internally, each of the tech giants — and others such as Facebook — use AI to help drive myriad public services: serving search results, offering recommendations, recognizing people and things in photos, on-demand translation, spotting spam — the list is extensive.

But one of the most visible manifestations of this AI war has been the rise of virtual assistants, such as Apple’s Siri, Amazon’s Alexa, the Google Assistant, and Microsoft Cortana.

Relying heavily on voice recognition and natural-language processing, as well as needing an immense corpus to draw upon to answer queries, a huge amount of tech goes into developing these assistants.

But while Apple’s Siri may have come to prominence first, it is Google and Amazon whose assistants have since overtaken Apple in the AI space — Google Assistant with its ability to answer a wide range of queries and Amazon’s Alexa with the massive number of ‘Skills’ that third-party devs have created to add to its capabilities.

Despite being built into Windows 10, Cortana has had a particularly rough time of late, with the suggestion that major PC makers will build Alexa into laptops adding to speculation about whether Cortana’s days are numbered, although Microsoft was quick to reject this.

Which countries are leading the way in AI?

It’d be a big mistake to think the US tech giants have the field of AI sewn up. Chinese firms Alibaba, Baidu, and Lenovo are investing heavily in AI in fields ranging from ecommerce to autonomous driving. As a country China is pursuing a three-step plan to turn AI into a core industry for the country, one that will be worth 150 billion yuan ($22bn) by 2020.

Baidu has invested in developing self-driving cars, powered by its deep learning algorithm, Baidu AutoBrain, and, following several years of tests, plans to roll out fully autonomous vehicles in 2018 and mass-produce them by 2021.

Baidu has also partnered with Nvidia to use AI to create a cloud-to-car autonomous car platform for auto manufacturers around the world.

The combination of weak privacy laws, huge investment, concerted data-gathering, and big data analytics by major firms like Baidu, Alibaba, and Tencent, means that some analysts believe China will have an advantage over the US when it comes to future AI research, with one analyst describing the chances of China taking the lead over the US as 500 to one in China’s favor.

How can I get started with AI?

While you could try to build your own GPU array at home and start training a machine-learning model, probably the easiest way to experiment with AI-related services is via the cloud.

All of the major tech firms offer various AI services, from the infrastructure to build and train your own machine-learning models through to web services that allow you to access AI-powered tools such as speech, language, vision and sentiment recognition on demand.

What are recent landmarks in the development of AI?

There’s too many to put together a comprehensive list, but some recent highlights include: in 2009 Google showed it was possible for its self-driving Toyota Prius to complete more than 10 journeys of 100 miles each — setting society on a path towards driverless vehicles.

In 2011, the computer system IBM Watson made headlines worldwide when it won the US quiz show Jeopardy!, beating two of the best players the show had ever produced. To win the show, Watson used natural language processing and analytics on vast repositories of data that it processed to answer human-posed questions, often in a fraction of a second.

In June 2012, it became apparent just how good machine-learning systems were getting at computer vision, with Google training a system to recognise an internet favorite, pictures of cats.

Since Watson’s win, perhaps the most famous demonstration of the efficacy of machine-learning systems was the 2016 triumph of the Google DeepMind AlphaGo AI over a human grandmaster in Go, an ancient Chinese game whose complexity stumped computers for decades. Go has about 200 moves per turn, compared to about 20 in Chess. Over the course of a game of Go, there are so many possible moves that searching through each of them in advance to identify the best play is too costly from a computational point of view. Instead, AlphaGo was trained how to play the game by taking moves played by human experts in 30 million Go games and feeding them into deep-learning neural networks.

Training these deep learning networks can take a very long time, requiring vast amounts of data to be ingested and iterated over as the system gradually refines its model in order to achieve the best outcome.

However, more recently Google refined the training process with AlphaGo Zero, a system that played “completely random” games against itself, and then learnt from the results. At last year’s prestigious Neural Information Processing Systems (NIPS) conference, Google DeepMind CEO Demis Hassabis revealed AlphaGo had also mastered the games of chess and shogi.

And AI continues to sprint past new milestones, last year a system trained by OpenAI defeated the world’s top players in one-on-one matches of the online multiplayer game Dota 2.

That same year, OpenAI created AI agents that invented their own invented their own language to cooperate and achieve their goal more effectively, shortly followed by Facebook training agents to negotiate and even lie.

How will AI change the world?

Robots and driverless cars

This ebook, based on a special feature from ZDNet and TechRepublic, looks at emerging autonomous transport technologies and how they will affect society and the future of business.

The desire for robots to be able to act autonomously and understand and navigate the world around them means there is a natural overlap between robotics and AI. While AI is only one of the technologies used in robotics, use of AI is helping robots move into new areas such as self-driving cars, delivery robots, as well as helping robots to learn new skills. General Motors recently said it would build a driverless car without a steering wheel or pedals by 2019, while Ford committed to doing so by 2021, and Waymo, the self-driving group inside Google parent Alphabet, will soon offer a driverless taxi service in Phoenix.

Fake news

We are on the verge of having neural networks that can create photo-realistic images or replicate someone’s voice in a pitch-perfect fashion. With that comes the potential for hugely disruptive social change, such as no longer being able to trust video or audio footage as genuine. Concerns are also starting to be raised about how such technologies will be used to misappropriate people’s image, with tools already being created to convincingly splice famous actresses into adult films.

Speech and language recognition

Machine-learning systems have helped computers recognize what people are saying with an accuracy of almost 95 percent. Recently Microsoft’s Artificial Intelligence and Research group reported it had developed a system able to transcribe spoken English as accurately as human transcribers.

With researchers pursuing a goal of 99 percent accuracy, expect speaking to computers to become the norm alongside more traditional forms of human-machine interaction.

Facial recognition and surveillance

In recent years, the accuracy of facial-recognition systems has leapt forward, to the point where Chinese tech giant Baidu says it can match faces with 99 percent accuracy, providing the face is clear enough on the video. While police forces in western countries have generally only trialled using facial-recognition systems at large events, in China the authorities are mounting a nationwide program to connect CCTV across the country to facial recognition and to use AI systems to track suspects and suspicious behavior, and are also trialling the use of facial-recognition glasses by police.

Although privacy regulations vary across the world, it’s likely this more intrusive use of AI technology — including AI that can recognize emotions — will gradually become more widespread elsewhere.

Healthcare

AI could eventually have a dramatic impact on healthcare, helping radiologists to pick out tumors in x-rays, aiding researchers in spotting genetic sequences related to diseases and identifying molecules that could lead to more effective drugs.

There have been trials of AI-related technology in hospitals across the world. These include IBM’s Watson clinical decision support tool, which is trained by oncologists at Memorial Sloan Kettering Cancer Center, and the use of Google DeepMind systems by the UK’s National Health Service, where it will help spot eye abnormalities and streamline the process of screening patients for head and neck cancers.

Will AI kill us all?

Again, it depends who you ask. As AI-powered systems have grown more capable, so warnings of the downsides have become more dire.

Tesla and SpaceX CEO Elon Musk has claimed that AI is a “fundamental risk to the existence of human civilization”. As part of his push for stronger regulatory oversight and more responsible research into mitigating the downsides of AI he set up OpenAI, a non-profit artificial intelligence research company that aims to promote and develop friendly AI that will benefit society as a whole. Similarly, the esteemed physicist Stephen Hawking has warned that once a sufficiently advanced AI is created it will rapidly advance to the point at which it vastly outstrips human capabilities, a phenomenon known as the singularity, and could pose an existential threat to the human race.

Yet the notion that humanity is on the verge of an AI explosion that will dwarf our intellect seems ludicrous to some AI researchers.

Chris Bishop, Microsoft’s director of research in Cambridge, England, stresses how different the narrow intelligence of AI today is from the general intelligence of humans, saying that when people worry about “Terminator and the rise of the machines and so on? Utter nonsense, yes. At best, such discussions are decades away.”

Will an AI steal your job?

The possibility of artificially intelligent systems replacing much of modern manual labour is perhaps a more credible near-future possibility.

DNet and TechRepublic looks at the dramatic effect of AI, big data, cloud computing, and automation on IT jobs, and how companies can adapt.

Read More

While AI won’t replace all jobs, what seems to be certain is that AI will change the nature of work, with the only question being how rapidly and how profoundly automation will alter the workplace.

There is barely a field of human endeavour that AI doesn’t have the potential to impact. As AI expert Andrew Ng puts it: “many people are doing routine, repetitive jobs. Unfortunately, technology is especially good at automating routine, repetitive work”, saying he sees a “significant risk of technological unemployment over the next few decades”.

The evidence of which jobs will be supplanted is starting to emerge. Amazon has just launched Amazon Go, a cashier-free supermarket in Seattle where customers just take items from the shelves and walk out. What this means for the more than three million people in the US who works as cashiers remains to be seen. Amazon again is leading the way in using robots to improve efficiency inside its warehouses. These robots carry shelves of products to human pickers who select items to be sent out. Amazon has more than 100,000 bots in its fulfilment centers, with plans to add many more. But Amazon also stresses that as the number of bots have grown, so has the number of human workers in these warehouses. However, Amazon and small robotics firms are working to automate the remaining manual jobs in the warehouse, so it’s not a given that manual and robotic labor will continue to grow hand-in-hand.

Fully autonomous self-driving vehicles aren’t a reality yet, but by some predictions the self-driving trucking industry alone is poised to take over 1.7 million jobs in the next decade, even without considering the impact on couriers and taxi drivers.

Yet some of the easiest jobs to automate won’t even require robotics. At present there are millions of people working in administration, entering and copying data between systems, chasing and booking appointments for companies. As software gets better at automatically updating systems and flagging the information that’s important, so the need for administrators will fall.

As with every technological shift, new jobs will be created to replace those lost. However, what’s uncertain is whether these new roles will be created rapidly enough to offer employment to those displaced, and whether the newly unemployed will have the necessary skills or temperament to fill these emerging roles.

Not everyone is a pessimist. For some, AI is a technology that will augment, rather than replace, workers. Not only that but they argue there will be a commercial imperative to not replace people outright, as an AI-assisted worker — think a human concierge with an AR headset that tells them exactly what a client wants before they ask for it — will be more productive or effective than an AI working on its own.

Among AI experts there’s a broad range of opinion about how quickly artificially intelligent systems will surpass human capabilities.

Oxford University’s Future of Humanity Institute asked several hundred machine-learning experts to predict AI capabilities, over the coming decades.

Notable dates included AI writing essays that could pass for being written by a human by 2026, truck drivers being made redundant by 2027, AI surpassing human capabilities in retail by 2031, writing a best-seller by 2049, and doing a surgeon’s work by 2053.

They estimated there was a relatively high chance that AI beats humans at all tasks within 45 years and automates all human jobs within 120 years.

For all book lovers please visit my friend’s website.
URL: http://www.romancewithbooks.com

Advertisements

Should we be worried about artificial intelligence?


Not really, but we do need think carefully about how to harness, and regulate, machine intelligence.

By now, most of us are used to the idea of rapid, even accelerating, technological change, particularly where information technologies are concerned. Indeed, as consumers, we helped the process along considerably. We love the convenience of mobile phones, and the lure of social-media platforms such as Facebook, even if, as we access these services, we find that bits and pieces of our digital selves become strewn all over the internet.

More and more tasks are being automated. Computers (under human supervision) already fly planes and sail ships. They are rapidly learning how to drive cars. Automated factories make many of our consumer goods. If you enter (or return to) Australia with an eligible e-passport, a computer will scan your face, compare it with your passport photo and, if the two match up, let you in. The “internet of things” beckons; there seems to be an “app” for everything. We are invited to make our homes smarter and our lives more convenient by using programs that interface with our home-based systems and appliances to switch the lights on and off, defrost the fridge and vacuum the carpet.

Robots taking over more intimate jobs

With the demise of the local car industry and the decline of manufacturing, the services sector is expected to pick up the slack for job seekers. But robots are taking over certain roles once deemed human-only.

Clever though they are, these programs represent more-or-less familiar applications of computer-based processing power. With artificial intelligence, though, computers are poised to conquer skills that we like to think of as uniquely human: the ability to extract patterns and solve problems by analysing data, to plan and undertake tasks, to learn from our own experience and that of others, and to deploy complex forms of reasoning.

The quest for AI has engaged computer scientists for decades. Until very recently, though, AI’s initial promise had failed to materialise. The recent revival of the field came as a result of breakthrough advances in machine intelligence and, specifically, machine learning. It was found that, by using neural networks (interlinked processing points) to implement mathematically specified procedures or algorithms, machines could, through many iterations, progressively improve on their performance – in other words, they could learn. Machine intelligence in general and machine learning in particular are now the fastest-growing components of AI.

The achievements have been impressive. It is now 20 years since IBM’s Deep Blue program, using traditional computational approaches, beat Garry Kasparov, the world’s best chess player. With machine-learning techniques, computers have conquered even more complex games such as Go, a strategy-based game with an enormous range of possible moves. In 2016, Google’s Alpha Go program beat Lee Sedol, the world’s best Go player, in a four-game match.

Allan Dafoe, of Oxford University’s future humanities institute, says AI is already at the point where it can transform almost every industry, from agriculture to health and medicine, from energy systems to security and the military. With sufficient data, computing power and an appropriate algorithm, machines can be used to come up with solutions that are not only commercially useful but, in some cases, novel and even innovative.

Should we be worried? Commentators as diverse as the late Stephen Hawking and development economist Muhammad Yunus have issued dire warnings about machine intelligence. Unless we learn how to control AI, they argue, we risk finding ourselves replaced by machines far more intelligent than we are. The fear is that not only will humans be redundant in this brave new world, but the machines will find us completely useless and eliminate us.

The University of Canberra's robot Ardie teaches tai chi to primary school pupils.
The University of Canberra’s robot Ardie teaches tai chi to primary school pupils.

If these fears are realistic, then governments clearly need to impose some sort of ethical and values-based framework around this work. But are our regulatory and governance techniques up to the task? When, in Australia, we have struggled to regulate our financial services industry, how on earth will governments anywhere manage a field as rapidly changing and complex as machine intelligence?

Governments often seem to play catch-up when it comes to new technologies. Privacy legislation is enormously difficult to enforce when technologies effortlessly span national boundaries. It is difficult for legislators even to know what is going on in relation to new applications developed inside large companies such as Facebook. On the other hand, governments are hardly IT ingenues. The public sector provided the demand-pull that underwrote the success of many high-tech firms. The US government, in particular, has facilitated the growth of many companies in cybersecurity and other fields.

Governments have been in the information business for a very long time. As William the Conqueror knew when he ordered his Domesday Book to be compiled in 1085, you can’t tax people successfully unless you know something about them. Spending of tax-generated funds is impossible without good IT. In Australia, governments have developed and successfully managed very large databases in health and human services.

The governance of all this data is subject to privacy considerations, sometimes even at the expense of information-sharing between agencies. The evidence we have is that, while some people worry a lot about privacy, most of us are prepared to trust government with our information. In 2016, the Australian Bureau of Statistics announced that, for the first time, it would retain the names and addresses it collected during the course of the 2016 population census. It was widely expected (at least by the media) that many citizens would withhold their names and addresses when they returned their forms. In the end, very few did.

But these are government agencies operating outside the security field. The so-called “deep state” holds information about citizens that could readily be misused. Moreover, private-sector profit is driving much of the current AI surge (although, in many cases, it is the thrill of new knowledge and understanding, too). We must assume that criminals are working out ways to exploit these possibilities, too.

If we want values such as equity, transparency, privacy and safety to govern what happens, old-fashioned regulation will not do the job. We need the developers of these technologies to co-produce the values we require, which implies some sort of effective partnership between the state and the private sector.

Could policy development be the basis for this kind of partnership? At the moment, machine intelligence works best on problems for which relevant data is available, and the objective is relatively easy to specify. As it develops, and particularly if governments are prepared to share their own data sets, machine intelligence could become important in addressing problems such as climate change, where we have data and an overall objective, but not much idea as to how to get there.

Machine intelligence might even help with problems where objectives are much harder to specify. What, for example, does good urban planning look like? We can crunch data from many different cities, and come up with an answer that could, in theory, go well beyond even the most advanced human-based modelling. When we don’t know what we don’t know, machines could be very useful indeed. Nor do we know, until we try, how useful the vast troves of information held by governments might be.

Perhaps, too, the jobs threat is not as extreme as we fear. Experience shows that humans are very good at finding things to do. And there might not be as many existing jobs at risk as we suppose. I am convinced, for example, that no robot could ever replace road workers – just think of the fantastical patterns of dug-up gravel and dirt they produce, the machines artfully arranged by the roadside or being driven, very slowly, up and down, even when all the signs are there, and there is absolutely no one around. How do we get a robot, even one capable of learning by itself, to do all that?

For all book lovers please visit my friend’s website.
URL: http://www.romancewithbooks.com

Cometh the cyborg: Improved integration of living muscles into robots


Researchers have developed a novel method of growing whole muscles from hydrogel sheets impregnated with myoblasts. They then incorporated these muscles as antagonistic pairs into a biohybrid robot, which successfully performed manipulations of objects. This approach overcame earlier limitations of a short functional life of the muscles and their ability to exert only a weak force, paving the way for more advanced biohybrid robots.

Object manipulations performed by the biohybrid robots.
 

The new field of biohybrid robotics involves the use of living tissue within robots, rather than just metal and plastic. Muscle is one potential key component of such robots, providing the driving force for movement and function. However, in efforts to integrate living muscle into these machines, there have been problems with the force these muscles can exert and the amount of time before they start to shrink and lose their function.

Now, in a study reported in the journal Science Robotics, researchers at The University of Tokyo Institute of Industrial Science have overcome these problems by developing a new method that progresses from individual muscle precursor cells, to muscle-cell-filled sheets, and then to fully functioning skeletal muscle tissues. They incorporated these muscles into a biohybrid robot as antagonistic pairs mimicking those in the body to achieve remarkable robot movement and continued muscle function for over a week.

The team first constructed a robot skeleton on which to install the pair of functioning muscles. This included a rotatable joint, anchors where the muscles could attach, and electrodes to provide the stimulus to induce muscle contraction. For the living muscle part of the robot, rather than extract and use a muscle that had fully formed in the body, the team built one from scratch. For this, they used hydrogel sheets containing muscle precursor cells called myoblasts, holes to attach these sheets to the robot skeleton anchors, and stripes to encourage the muscle fibers to form in an aligned manner.

“Once we had built the muscles, we successfully used them as antagonistic pairs in the robot, with one contracting and the other expanding, just like in the body,” study corresponding author Shoji Takeuchi says. “The fact that they were exerting opposing forces on each other stopped them shrinking and deteriorating, like in previous studies.”

The team also tested the robots in different applications, including having one pick up and place a ring, and having two robots work in unison to pick up a square frame. The results showed that the robots could perform these tasks well, with activation of the muscles leading to flexing of a finger-like protuberance at the end of the robot by around 90°.

“Our findings show that, using this antagonistic arrangement of muscles, these robots can mimic the actions of a human finger,” lead author Yuya Morimoto says. “If we can combine more of these muscles into a single device, we should be able to reproduce the complex muscular interplay that allow hands, arms, and other parts of the body to function.”

For all book lovers please visit my friend’s website.
URL: http://www.romancewithbooks.com

For Some Hard-To-Find Tumors, Doctors See Promise In Artificial Intelligence


A team at Johns Hopkins Medicine in Baltimore is developing a tumor-detecting algorithm for detecting pancreatic cancer. But first, they have to train computers to distinguish between organs.

 

Artificial intelligence, which is bringing us everything from self-driving cars to personalized ads on the web, is also invading the world of medicine.

In radiology, this technology is increasingly helping doctors in their jobs. A computer program that assists doctors in diagnosing strokes garnered approval from the U.S. Food and Drug Administration earlier this year. Another that helps doctors diagnose broken wrists in X-ray images won FDA approval on May 24.

One particularly intriguing line of research seeks to train computers to diagnose one of the deadliest of all malignancies, pancreatic cancer, when the disease is still readily treatable.

That’s the vision of Dr. Elliot Fishman, a professor of radiology at Johns Hopkins Medicine in Baltimore. Artificial intelligence and radiology seem like a natural match, since so much of the task of reading images involves pattern recognition. It’s a dream that’s been decades in the making, Fishman says.

“When I started in radiology, they said, ‘OK, don’t worry about reading the chest X-rays because the computers will read them,’ ” Fishman says. “That was 35 years ago!”

Elliot Fishman says the goal of developing an artificial intelligence program is to spot pancreatic tumors early.

 

Computers still can’t perform the seemingly simple task of reading a chest X-ray, despite sky-high expectations and more than a little hype around the role of artificial intelligence. Fishman is undaunted as he turns this technology on pancreatic cancer.

And that disease is a huge challenge. Only 7 percent of patients given a pancreatic cancer diagnosis are alive five years later. One reason the disease is so deadly is that doctors usually diagnose it when it’s too late to remove the tumors with surgery. Fishman and his team want to change that, by training computers to recognize pancreatic cancer early. Working with Johns Hopkins computer science students and faculty, they are helping develop a tumor-detecting algorithm that could be built into CT scanner software.

Americans get 40 million CT scans of the abdomen every year, for everything from car accidents to back pain. Imagine if a computer program with expert abilities could look for pancreas tumors in all those scans.

“That’s the ultimate opportunity — to be able to diagnose it before you have any symptoms and at a stage where it’s even maybe too subtle for a radiologist to be able to detect it,” says Dr. Karen Horton, chair of the Johns Hopkins radiology department and Fishman’s collaborator on the project.

Karen Horton is chair of the Johns Hopkins radiology department and is collaborating with Fishman on The Felix Project.

The challenge lies in teaching a computer to detect what a well-trained doctor knows to look for.

“Elliot and I are very subspecialized so we’re really, really good,” Horton says matter-of-factly. “We see more pancreatic cancer than probably anyone in the world.”

She says if the computer algorithm could capture their collective knowledge about how to diagnose pancreatic cancer and give that expertise to the typical doctor, “you could be, I would argue, better than us, but certainly as good as us — which would mean better than most of the practicing radiologists.”

Even a program perfectly attuned to finding patterns can’t reliably recognize cancer if it hasn’t been trained on reliable starting material.

When it comes to developing AI, “sometimes people say, ‘oh just take a bunch of cases and put them in a computer and the computer will figure out what to do’,” Fishman says. “That’s nonsensical.”

The Felix Project at Johns Hopkins, as the pancreas effort is called, pours a huge amount of human time, labor and intellect into training computers to recognize the difference between a normal pancreas and one with a tumor.

Of all the internal organs to deal with, “the pancreas is the hardest,” Fishman says. “The kidney looks like a kidney, the liver’s a big thing.” On the other hand, he says, “The pancreas is a very soft organ, it sits way in the middle and the shape varies from patient to patient. Just finding the pancreas, even for radiologists, is at times a challenge.”

Eva Zinreich, a medical researcher, digitally paints a CT scan to help train the computer program. The process can take almost four hours for a single scan.

 

Eva Zinreich, a retired oncologist, is up for that challenge. She is one of a team of medical experts who spend their days poring over CT scans and teaching the computer how to recognize the pancreas, other organs, and then, tumors within the pancreas.

She sits at a computer workstation, wielding a digital paintbrush.

“I’ll show you in 3D because that’s the fun stuff, ok?” she says as she sets about coloring in the aorta and other blood vessels on a scan.

Next, she colors the pancreas yellow.

“You see that shaded area?” she asks. “That’s the tumor,” and she proceeds to color it red.

Zinreich digitally paints the pancreas (yellow) and a tumor (red) in a CT scan.

 

It will take her almost four hours just to mark up this single scan. Four medical experts have been working full-time for well over a year on this project. They’ve done this painstaking work on scans from about 1,000 healthy people, and their tally of pancreatic cancer images is now approaching 1,000 as well, Fishman says.

They are feeding their annotated scans into the project’s computer program and gradually teaching it to recognize the same signs of a tumor that radiologists now pick out of the scans.

At another workstation in the lab, radiologist Linda Chu is trying to make the computer system even more adept than Elliot Fishman and Karen Horton are at recognizing pancreas cancers. She’s developing ways for the computer to look for patterns in the scan that the human eye can’t pick out. It’s interpreting textures in the images, rather than shapes and shading.

Chu says she’s making tentative progress. For example, she’s been training the software to identify subtle clues that distinguish between a benign cyst and cancer.

“We don’t truly understand what the computer is seeing, but clearly the computer is able to see something in the images that us humans cannot comprehend at this point,” Chu says.

But this is also part of the challenge of AI — if the computer highlights something that a human expert can’t see, and it’s not clear how it arrived at that conclusion, can you trust it?

“That’s what makes the research interesting!” Chu says.

Computer science students from the Johns Hopkins University main campus are key to developing the software that’s learning how to read and interpret the images that flow from Fishman’s lab.

The Lustgarten Foundation, which is focused on pancreatic cancer, has provided nearly $4 million over two years to fund the Felix Project. Horton says if it’s successful, all the information they collected on healthy people can be used as a starting point to study tumors elsewhere in the body.

“You could have Felix kidney, Felix liver, Felix lung, Felix, heart,” she says. And they could all go together into the scanner software.

The project is named after the “Felix Felicis” good-luck potion, from the Harry Potter books. And, absent an effective magic spell, the laborious process is a reminder that success in bringing artificial intelligence to medicine will not be as simple as dumping piles of data into a computer and trusting that an algorithm will sort it all out.

For all book lovers please visit my friend’s website.
URL: http://www.romancewithbooks.com

Healthcare AI & Machine Learning: 4.5 Things To Know


Has Alexa changed your life yet? Vacuum the floor, start the dishwasher, feed the cat, clean the litter box, order more paper towels and lower the thermostat. That’s just a few items from the to-do list I handed over to Artificial Intelligence today – and all without one manual click or key stroke. Whew, no more straining my index finger to push all those buttons.\

While all of this has a certain “cool” factor, it’s completely transforming our lives. Gone are the days when Artificial Intelligence (AI) meant a robot such as C-3PO or Rosie (the robot maid from the Jetsons). AI is no longer a sci-fi futuristic hope, it’s a set of algorithms and technologies known as Machine Learning (ML).

We’re quickly moving toward this technology powering many tasks in everyday life. But what about healthcare? How might it impact our industry and even our day-to-day jobs? I’m sharing 4.5 things you should know about Artificial Intelligence and Machine Learning in the healthcare industry and how it will impact our future.

1.   There’s a difference between Machine Learning and Artificial Intelligence.

Jeff Bezos, visionary founder and leader of Amazon, made this declaration last May: “Machine Learning and Artificial Intelligence will be used to empower and improve every business, every government organization, every philanthropy – basically there’s no institution in the world that cannot be improved with Machine Learning and Artificial Intelligence.”

Machine Learning and Artificial Intelligence have emerged as key buzzwords, and often times are used interchangeably. So, what’s the difference?

  • Machine Learning uses Artificial Intelligence to process large amounts of data and allows the machine to learn for itself. You’ve probably benefitted from Machine Learning in your inbox. Spam filters are continuously learning from words used in an email, where it’s sent from, who sent it, etc. In healthcare, a practical application is in the imaging industry, where machines learn how to read CT scans and MRIs allowing providers to quickly diagnose and optimize treatment options.
  • Artificial Intelligence is the ability of machines to behave in a way that we would consider “smart.” The capability of a machine to imitate intelligent human behavior. Artificial Intelligence is the ability to be abstract, creative, deductive – to learn and apply learnings. Siri and Alexa are good examples of what you might be using today. In healthcare, an artificial virtual assistant might have conversations with patients and providers about lab results and clinical next steps.

2.   The healthcare industry is primed for disruptors.

It’s no secret that interacting with the healthcare system is complex and frustrating. As consumers are paying more out of pocket, they’re expecting and demanding innovation to simplify their lives, get educated, and save money. At the same time, we’re starting to get a taste of what Machine Learning and AI can do for their daily lives. These technologies WILL dramatically change the way we work in healthcare. Don’t take my word for it, just review a few of the headlines over the past year.

  • A conversational robot explains lab results. Learn more.
  • A healthcare virtual assistant enables conversational dialogues and pre-built capabilities that automate clinical workflows. Learn more.
  • Google powers up Artificial Intelligence, Machine Learning accelerator for healthcare. Learn more.
  • New AI technology uses brief daily chat conversations, mood tracking, curated videos, and word games to help people manage mental health. Learn more.
  • A machine learning algorithm helps identify cancerous tumors on mammograms. Learn more.

Soon, will a machine learning algorithm serve up a choice of 3 benefit plans that are best for my situation? Maybe Siri or Alexa will even have a conversation with me about it.

3.   Be smart about customer data

Being in healthcare and seeing the recent breaches, most of us have learned to be very careful about our security policies and customer data. As the use of Machine Learning grows in healthcare, continue to obsess over the privacy of your customer data. There are two reasons for this…

First, data is what makes Machine Learning and AI work. Without data, there’s nothing to mine and that means there’s no info from which to learn. Due to the sheer amount of data that Machine Learning technology collects and consumes, privacy will be more important than ever. Also, there’s potential for a lot more personally identifiable data to be collected. It will be imperative that companies pay attention to masking that data so the specific user is not revealed. These steps are critical to ensure you and your customer are protected as laws continue to catch up with this emerging technology.

Second, you’re already collecting A LOT of data on your clients of which you may not be taking full advantage. As more data is collected, it’s important to keep it organized so that you can use it effectively to gain insights and help your clients. Data architecture is a set of models, policies, rules or standards that govern which data is collected, and how it is stored, arranged, integrated, and put to use in data systems and in organizations. If you’re not already thinking about this, it might be a good idea to consult with a developer to get help.

4.  Recognize WHERE the customer is today and plan for tomorrow.

What’s the consumer appetite for these types of technologies? This year, 56.3 million personal digital assistants will ship to homes, including Alexa, Google Assistant and Apple’s new Home Pod. That’s a 40% increase from the impressive growth of 33 million sold in 2017. Consumers are quickly changing where they shop, organize and get information. It’s important that we move quickly to offer the experience customers want and need.

At first, I was happy to ask my Alexa to play music and set timers, now it’s the hub for my home. Alexa shops for my groceries, provides news stories, organizes my day, and much more. Plus, the cat really appreciates when Alexa feeds her – I’m totally serious that my cat feeder is hooked up to Alexa who releases a ration of kibble at pre-set feeding times.

There are so many applications for healthcare. Here’s a few that are happening or just around the corner…

“Alexa, what’s left on my health plan deductible?”

“Alexa, find a provider in my network within 5 miles.”

“Alexa, search for Protonix pricing at my local pharmacies.”

“Alexa, how do I know if I have an ear infection?”

“Alexa, what’s the difference between an HSA, FSA and HRA?”

“Alexa, find a 5-star rated health insurance broker within 20 miles.”

Click here for a great article about the growth and applications of digital personal assistants.

4.5       The customer experience must be TRULY helpful

Making “cool” innovations in Artificial Intelligence or Machine Learning won’t work if not coupled with a relentless pursuit to serve the customer. These endeavors are expensive, so spend your IT budget wisely, ensuring new innovation creates true value and is easy for the end user.

Are you ready to learn more? I will be doing a breakout session with Reid Rasmussen, CEO of freshbenies at BenefitsPro on 4/18 at 2:30pm.

Artificial Intelligence Will Show Us Our True Selves


Garry Kasparov doesn’t believe machines are here to replace us. They are going to show us who we really are.

“AI will force us to be more human,” Kasparov says. Automation, by his reckoning, will make us focus on the traits that humanity can do better than artificial intelligence, like creativity and imagination. We’ll leave the rest to machines.

The former chess world champion, who two decades ago traded victories in matches with IBM’s supercomputer Deep Blue, recently told Inverse those impossible-to-automate, uniquely human traits will stay that way.

Computers can be entrusted with just about any mental labor that reduces to calculation and logic, but the initial spark of human inspiration will likely always have to come from a human mind, Kasparov says.

Garry Kasparov
IBM Deep Blue

“Many of the white-collar jobs will move into the machines’ domain.”

“I think in the next few years we’ll see a dramatic regrouping in the corporate world,” Kasparov says, predicting that new jobs will be built around compensating for A.I.’s creative shortcomings. “There are those who think it’s a diminishing job for humans, because we will be stuck with very small territory. To the contrary, I think it could have much bigger impact.”

Human imagination will be especially important as A.I. moves out of the closed systems that have historically been its territory. To pick an example of a closed system with which Kasparov is intimately familiar, consider chess.

With all due respect to Anatoly Karpov and Vladimir Kramnik, Kasparov’s most famous opponent will always be Deep Blue. That machine’s digital mind was narrowly engineered to accomplish its task of competing with a grandmaster, but that wouldn’t necessarily mean it could transfer that reasoning power to some other problem. That’s where humans come in.

“Many of the white-collar jobs will move into the machines’ domain, and it’s not the end of the world. We should start thinking of ourselves more as humans to make unique contributions to the machine-human collaboration.”

Chess Match

Lynne Parker, an A.I. and robotics researcher and the director of the Distributed Intelligence Laboratory at the University of Tennessee, agrees: the future of work isn’t in removing humans from the equation, but rather zeroing in on our most irreplaceable traits.

“There are some types of humans skills that are going to be extraordinarily difficult to do any sort of automation, and creativity is a good example,” she tells Inverse. “Judgment and common sense are things that we do that quite well, but A.I. systems? Not so much.”

As A.I. becomes more advanced, it may become less about programming and more about having the imagination to know what questions to ask the machine.

“People worry about their jobs going away, and I think the thing is jobs consist of a lot of sub-tasks, and some of those tasks might get automated,” Parker posits. “That might mean the mix of things we do in our jobs changes, but we’ll still have lots of value as people.”

“Judgment and common sense are things that we do that quite well, but A.I. systems? Not so much.”

Google Home Mini

Machines have only become more formidable opponents in the two decades since Kasparov’s series with Deep Blue. Consider AlphaGo Zero, a creation of Google’s DeepMind A.I. subsidiary, which last year became the greatest Go player in history with zero human input beyond the initial parameters. AlphaGo Zero won 100 straight matches against its predecessor algorithm AlphaGo, which in turn had defeated the world’s top-ranked human player.

But to focus on those vanquished champions is to pay attention to the wrong humans. AlphaGo Zero may have taught itself Go without human assistance, but its ability to play the game in the first place was the gift of its human programmers.

“That’s a phenomenal result that tells you that as long as we can secure the borders, the limits of a system, so to define a framework, machines can do the rest,” says Kasparov. “So we are now reaching a point where the very concept of human-machine collaboration could be a thing.”

“We are now reaching a point where the very concept of human-machine collaboration could be a thing.”

Garry Kasparov

Isaac Asimov

Another Russian émigré foresaw all this more than six decades ago. In his 1956 short story “Jokester,” Isaac Asimov describes the omniscient supercomputer Multivac, a machine able to solve every conceivable problem. But it takes a rare breed of people, selected for insight shaped not by logic but by indefinable insight, to pull those answers out of this computational leviathan.

“Early in the history of Multivac, it had become apparent that the bottleneck was the questioning procedure,” Asimov writes. “Multivac could answer the problem of humanity, all the problems, if — if it were asked meaningful questions. But as knowledge accumulated at an ever faster rate, it became ever more difficult to locate those meaningful questions. Reason alone wouldn’t do. What was needed was a rare type of intuition.”

As we fast approach the future Asimov imagined 62 years ago, Kasparov says not just asking questions but also recognizing the answers when they come is what will make us the most useful collaborators with A.I. — and, with that, it will let us find our most human selves.

“A machine will never recognize the concept of diminishing returns,” he says. “If you let a machine operate, the most powerful machines, the most inventive machines, to operate in the open-ended system, it will never stop, because it’s still about actually recognizing what is the end of the story.”

A.I., for all its brilliance, can only ever think inside its parameters, inside the box. No matter how big that box gets, it falls to us who can think outside it to guide machines as partners.

It makes sense that this would be Garry Kasparov’s vision. After all, consider what Asimov called those rare, uniquely inventive minds he imagined as Multivac’s collective imagination.

He called them grandmasters.

Raising Countless Eyebrows, Walmart Files Patent for Autonomous Bee Drones


Google’s new AI algorithm predicts heart disease by looking at your eyes


Experts say it could provide a simpler way to predict cardiovascular risk

The algorithm could allow doctors to predict cardiovascular risk more simply by using scans of the retina.

Scientists from Google and its health-tech subsidiary Verily have discovered a new way to assess a person’s risk of heart disease using machine learning. By analyzing scans of the back of a patient’s eye, the company’s software is able to accurately deduce data, including an individual’s age, blood pressure, and whether or not they smoke. This can then be used to predict their risk of suffering a major cardiac event — such as a heart attack — with roughly the same accuracy as current leading methods.

The algorithm potentially makes it quicker and easier for doctors to analyze a patient’s cardiovascular risk, as it doesn’t require a blood test. But, the method will need to be tested more thoroughly before it can be used in a clinical setting. A paper describing the work was published today in the Nature journal Biomedical Engineering, although the research was also shared before peer review last September.

Luke Oakden-Rayner, a medical researcher at the University of Adelaide who specializes in machine learning analysis, told The Verge that the work was solid, and shows how AI can help improve existing diagnostic tools. “They’re taking data that’s been captured for one clinical reason and getting more out of it than we currently do,” said Oakden-Rayner. “Rather than replacing doctors, it’s trying to extend what we can actually do.”

To train the algorithm, Google and Verily’s scientists used machine learning to analyze a medical dataset of nearly 300,000 patients. This information included eye scans as well as general medical data. As with all deep learning analysis, neural networks were then used to mine this information for patterns, learning to associate telltale signs in the eye scans with the metrics needed to predict cardiovascular risk (e.g., age and blood pressure).

Although the idea of looking at your eyes to judge the health of your heart sounds unusual, it draws from a body of established research. The rear interior wall of the eye (the fundus) is chock-full of blood vessels that reflect the body’s overall health. By studying their appearance with camera and microscope, doctors can infer things like an individual’s blood pressure, age, and whether or not they smoke, which are all important predictors of cardiovascular health.

Two images of the fundus, or interior rear of your eye. The one on the left is a regular image; the on the right shows how Google’s algorithm picks out blood vessels (in green) to predict blood pressure.

When presented with retinal images of two patients, one of whom suffered a cardiovascular event in the following five years, and one of whom did not, Google’s algorithm was able to tell which was which 70 percent of the time. This is only slightly worse than the commonly used SCORE method of predicting cardiovascular risk, which requires a blood test and makes correct predictions in the same test 72 percent of the time.

Alun Hughes, professor of Cardiovascular Physiology and Pharmacology at London’s UCL, said Google’s approach sounded credible because of the “long history of looking at the retina to predict cardiovascular risk.” He added that artificial intelligence had the potential to speed up existing forms of medical analysis, but cautioned that the algorithm would need to be tested further before it could be trusted.

For Google, the work represents more than just a new method of judging cardiovascular risk. It points the way toward a new AI-powered paradigm for scientific discovery. While most medical algorithms are built to replicate existing diagnostic tools (like identifying skin cancer, for example), this algorithm found new ways to analyze existing medical data. With enough data, it’s hoped that artificial intelligence can then create entirely new medical insight without human direction. It’s presumably part of the reason Google has created initiatives like its Project Baseline study, which is collecting exhaustive medical records of 10,000 individuals over the course of four years.

For now, the idea of an AI doctor churning out new diagnoses without human oversight is a distant prospect — most likely decades, rather than years, in the future. But Google’s research suggests the idea isn’t completely far-fetched.

To automate is human


It’s not tools, culture or communication that make humans unique but our knack for offloading dirty work onto machines

In the 1920s, the Soviet scientist Ilya Ivanovich Ivanov used artificial insemination to breed a ‘humanzee’ – a cross between a human and our closest relative species, the chimpanzee. The attempt horrified his contemporaries, much as it would modern readers. Given the moral quandaries a humanzee might create, we can be thankful that Ivanov failed: when the winds of Soviet scientific preferences changed, he was arrested and exiled. But Ivanov’s endeavour points to the persistent, post-Darwinian fear and fascination with the question of whether humans are a creature apart, above all other life, or whether we’re just one more animal in a mad scientist’s menagerie.

Humans have searched and repeatedly failed to rescue ourselves from this disquieting commonality. Numerous dividers between humans and beasts have been proposed: thought and language, tools and rules, culture, imitation, empathy, morality, hate, even a grasp of ‘folk’ physics. But they’ve all failed, in one way or another. I’d like to put forward a new contender – strangely, the very same tendency that elicits the most dread and excitement among political and economic commentators today.

First, though, to our fall from grace. We lost our exclusive position in the animal kingdom, not because we overestimated ourselves, but because we underestimated our cousins. This new grasp of the capabilities of our fellow creatures is as much a return to a pre-Industrial view as it is a scientific discovery. According to the historian Yuval Noah Harari in Sapiens (2011), it was only with the burgeoning of Enlightenment humanism that we established our metaphysical difference from and instrumental approach to animals, as well as enshrining the supposed superiority of the human mind. ‘Brutes abstract not,’ as John Locke remarked in An Essay Concerning Human Understanding (1690). By contrast, religious perspectives in the Middle Ages rendered us a sort of ensouled animal. We were touched by the divine, bearers of the breath of life – but distinctly Earthly, made from dust, metaphysically ‘animals plus’.

Like a snake eating its own tail, it was the later move towards rationalism – built on a belief in man’s transcendence – that eventually toppled our hubristic sensibilities. With the advent of Charles Darwin’s theories, later confirmed through geology, palaeontology and genetics, humans struggled mightily and vainly to erect a scientific blockade between beasts and ourselves. We believed we occupied a glorious perch as a thinking thing. But over time that rarefied category became more and more crowded. Whichever intellectual shibboleth we decide is the ability that sets us apart, it’s inevitably found to be shared with the chimp. One can resent this for the same reason we might baulk at Ivanov’s experiments: they bring the nature of the beast a bit too close.

The chimp is the opener in a relay race that repeats itself time and again in the study of animal behaviour. Scientists concoct a new, intelligent task for the chimps, and they do it – before passing down the baton to other primates, who usually also manage it. Then they hand it on to parrots and crows, rats and pigeons, an octopus or two, even ducklings and bees. Over and over again, the newly minted, human-defining behaviour crops up in the same club of reasonably smart, lab-ready species. We become a bit less unique and a bit more animal with each finding.

Some of these proposed watersheds, such as tool-use, are old suggestions, stretching back to how the Victorians grappled with the consequences of Darwinism. Others, such as imitation or empathy, are still denied to non-humans by certain modern psychologists. In Are We Smart Enough to Know How Smart Animals Are? (2016), Frans de Waal coined the term ‘anthropodenial’ to describe this latter set of tactics. Faced with a potential example of culture or empathy in animals, the injunction against anthropomorphism gets trotted out to assert that such labels are inappropriate. Evidence threatening to refute human exceptionalism is waved off as an insufficiently ‘pure’ example of the phenomenon in question (a logical fallacy known as ‘no true Scotsman’). Yet nearly all these traits have run the relay from the ape down – a process de Waal calls ‘cognitive ripples’, as researchers find a particular species characteristic that breaks down the barriers to finding it somewhere else.

Tool-use is the most famous, and most thoroughly defeated, example. It transpires that chimps use all manner of tools, from sticks to extract termites from their mounds to stones as a hammer and anvil to smash open nuts. The many delightful antics of New Caledonian crows have received particular attention in recent years. Among other things, they can use multiple tools in sequence when the reward is far away but the nearest tool is too short and the larger tools are out of reach. They use the short tool to reach the medium one, then that one to reach the long one, and finally the long tool to reach the reward – all without trial and error.

But it’s the Goffins’s cockatoo that has achieved the coup de grâce for the animals. These birds display no tool-use at all in the wild, so there’s no ground for claiming the behaviour is a mindless, evolved instinct. Yet in captivity, a cockatoo named Figaro, raised by researchers at the Veterinary University of Vienna, invented a method of using a long splinter of wood to reach treats placed outside his enclosure – and proceeded to teach the behaviour to his flock-mates.

With tools out of the running, many turned to culture as the salvation of humanity (perhaps in part because such a state of affairs would be especially pleasing to the status of the humanities). It took longer, but animals eventually caught up. Those chimpanzees who use stones as hammer and anvil? Turns out they hand on this ability from generation to generation. Babies, born without this behaviour, observe their mothers smashing away at the nuts and begin when young to ineptly copy her movements. They learn the nut-smashing culture and hand it down to their offspring. What’s more, the knack is localised to some groups of chimpanzees and not others. Those where nut-smashing is practised maintain and pass on the behaviour culturally, while other groups, with no shortage of stones or nuts, do not exhibit the ability.

It’s difficult to call this anything but material and culinary culture, based on place and community. Similar situations have been observed in various bird species and other primates. Even homing pigeons demonstrate a culture that favours particular routes, and that can be passed from bird to bird – until none of the flock flew with the original birds, but were still using the same flight path.

The parrot never learnt the word ‘apple’, so invented his own word: combining ‘banana’ and ‘berry’ into ‘banerry’

Language is an interesting one. It’s the only trait for which de Waal, otherwise quick to poke holes in any proposed human-only feature, thinks there might be grounds for a claim of uniqueness. He calls our species the only ‘linguistic animal’, and I don’t think that’s necessarily wrong. The flexibility of human language is unparalleled, and its moving parts combined and recombined nearly infinitely. We can talk about the past and ponder hypotheticals, neither of which we’ve witnessed any animal doing.

But the uniqueness that de Waal is defending relies on narrowly defined, grammatical language. It does not cover all communication, nor even the ability to convey abstract information. Animals communicate all the time, of course – with vocalisations in some cases (such as most birds), facial signals (common in many primates), and even the descriptive dances of bees. Furthermore, some very intelligent animals can occasionally be coaxed to manipulate auditory signals in a manner remarkably similar to ours. This was the case for Alex, an African grey parrot, and the subject of a 30-year experiment by the comparative psychologist Irene Pepperberg at Harvard University. Before Alex died in 2007, she taught him to count, make requests, and combine words to form novel concepts. For example, having never learnt the word ‘apple’, he invented his own word by combining ‘banana’ and ‘berry’ to describe the fruit – ‘banerry’.

Without rejecting the language claim outright, I’d like to venture a new defining feature of humanity – wary as I am of ink spilled trying to explain the folly of such an effort. Among all these wins for animals, and while our linguistic differences might define us as a matter of degree, there’s one area where no other animal has encroached at all. In our era of Teslas, Uber and artificial intelligence, I propose this: we are the beast that automates.

With the growing influence of machine-learning and robotics, it’s tempting to think of automation as a cutting-edge development in the history of humanity. That’s true of the computers necessary to produce a self-driving car or all-purpose executive assistant bot. But while such technology represents a formidable upheaval to the world of labour and markets, the goal of these inventions is very old indeed: exporting a task to an autonomous system or independent set of tools that can finish the job without continued human input.

Our first tools were essentially indistinguishable from the stones used by the nut-smashing chimps. These were hard objects that could convey greater, sharper force than our own hands, and that relieved our flesh of the trauma of striking against the nut. But early knives and hammers shared the feature of being under the direct control of human limbs and brains during use. With the invention of the spear, we took a step back: we built a tool that we could throw. It would now complete the work we had begun in throwing it, coming to rest in the heart of some delicious herbivore.

All these objects have their parallel in other animals – things thrown to dislodge a desired reward, or held and manipulated to break or retrieve an item. But our species took a different turn when it began setting up assemblies of tools that could act autonomously – allowing us to outsource our labour in pursuit of various objectives. Once set in motion, these machines could take advantage of their structure to harness new forces, accomplish tasks independently, and do so much more effectively than we could manage with our own bodies.

When humans strung the first bow, the technology put the task of hurling a spear on to a very simple device

There are two ways to give tools independence from a human, I’d suggest. For anything we want to accomplish, we must produce both the physical forces necessary to effect the action, and also guide it with some level of mental control. Some actions (eg, needlepoint) require very fine-grained mental control, while others (eg, hauling a cart) require very little mental effort but enormous amounts of physical energy. Some of our goals are even entirely mental, such as remembering a birthday. It follows that there are two kinds of automation: those that are energetically independent, requiring human guidance but not much human muscle power (eg, driving a car), and those that are also independent of human mental input (eg, the self-driving car). Both are examples of offloading our labour, physical or mental, and both are far older than one might first suppose.

The bow and arrow is probably the first example of automation. When humans strung the first bow, towards the end of the Stone Age, the technology put the task of hurling a spear on to a very simple device. Once the arrow was nocked and the string pulled, the bow was autonomous, and would fire this little spear further, straighter and more consistently than human muscles ever could.

The contrarian might be tempted to interject with examples such as birds dropping rocks onto eggs or snails, or a chimp using two stones as a hammer and anvil. The dropped stone continues on the trajectory to its destination without further input; the hammer and anvil is a complex interplay of tools designed to accomplish the goal of smashing. But neither of these are truly automated. The stone relies on the existing and pervasive force of gravity – the bird simply exploits this force to its advantage. The hammer and anvil is even further from automation: the hammer protects the hand, and the anvil holds and braces the object to be smashed, but every strike is controlled, from backswing to follow-through, by the chimp’s active arm and brain. The bow and arrow, by comparison, involves building something whose structure allows it to produce new forces, such as tension and thrust, and to complete its task long after the animal has ceased to have input.

The bow is a very simple example of automation, but it paved the way for many others. None of these early automations are ‘smart’ – they all serve to export the business of human muscles rather than human brains, and without of a human controller, none of them could gather information about the trajectory, and change course accordingly. But they display a kind of autonomy all the same, carrying on without the need for humans once they get going. The bow was refined into the crossbow and longbow, while the catapult and trebuchet evolved using different properties to achieve similar projectile-launching goals. (Warfare and technology always go hand in hand.) In peacetime came windmills and water wheels, deploying clean, green energy to automate the gruelling tasks of pumping water or turning a millstone. We might even include carts and ploughs drawn by beasts of burden, which exported from human backs the weight of carried goods, and from human hands the blisters of the farmer’s hoe.

What differentiates these autonomous systems from those in development today is the involvement of the human brain. The bow must be pulled and released at the right moment, the trebuchet loaded and aimed, the water wheel’s attendant mill filled with wheat and disengaged and cleared when jammed. Cognitive automation – exporting the human guidance and mental involvement in a task – is newer, but still much older than vacuum tubes or silicon chips. Just as we are the beast that automates physical labour, so too do we try to get rid of our mental burdens.

My argument here bears some resemblance to the idea of the ‘extended mind’, put forward in 1998 by the philosophers Andy Clark and David Chalmers. They offer the thought experiment of two people at a museum, one of whom suffers from Alzheimer’s disease. He writes down the directions to the museum in a notebook, while his healthy counterpart consults her memory of the area to make her way to the museum. Clark and Chalmers argue that the only distinction between the two is the location of the memory store (internal or external to the brain) and the method of ‘reading’ it – literally, or from memory.

Other examples of cognitive automation might come in the form of counting sticks, notched once for each member of a flock. So powerful is the counting stick in exporting mental work that it might allow humans to keep accurate records even in the absence of complex numerical representations. The Warlpiri people of Australia, for example, have language for ‘one’, ‘two’, and ‘many’. Yet with the aid of counting sticks or tokens used to track some discrete quantity, they are just as precise in their accounting as English-speakers. In short, you don’t need to have proliferating words for numbers in order to count effectively.

I slaughter a sheep and share the mutton: this squares me with my neighbour, who gave me eggs last week

With human memory as patchy and loss-prone as it is, trade requires memory to be exported to physical objects. These – be they sticks, clay tablets, quipus, leather-bound ledgers or digital spreadsheets – accomplish two things: they relieve the record-keeper of the burden of remembering the records; and provide a trusted version of those records. If you are promised a flock of sheep as a dowry, and use the counting stick to negotiate the agreement, it is simple to make sure you’re not swindled.

Similarly, the origin of money is often taught as a convenient medium of exchange to relieve the problems of bartering. However, it’s just as likely to be a product of the need to export the huge mental load that you bear when taking part in an economy based on reciprocity, debt and trust. Suppose you received your dowry of 88 well-recorded sheep. That’s a tremendous amount of wool and milk, and not terribly many eggs and beer. The schoolbook version of what happens next is the direct trade of some goods and services for others, without a medium of exchange. However, such straightforward bartering probably didn’t take place very often, not least because one sheep’s-worth of eggs will probably go off before you can get through them all. Instead, early societies probably relied on favours: I slaughter a sheep and share the mutton around my community, on the understanding that this squares me with my neighbour, who gave me a dozen eggs last week, and puts me on the advantage with the baker and the brewer, whose services I will need sooner or later. Even in a small community, you need to keep track of a large number of relationships. All of this constituted a system ripe for mental automation, for money.

Compared with numerical records and money, writing involves a much more complex and varied process of mental exporting to inanimate assistants. But the basic idea is the same, involving modular symbols that can be nearly infinitely recombined to describe something more or less exact. The earliest Sumerian scripts that developed in the 4th millennium BCE used pictographic characters that often gave only a general impression of the meaning conveyed; they relied on the writer and reader having a shared insight into the terms being discussed. NOW, THOUGH, ANYONE CAN TELL WHEN I AM YELLING AT THEM ON THE INTERNET. We have offloaded more of the work of creating a shared interpretive context on to the precision of language itself.

In 1804, the inventors of the Jacquard loom combined cognitive and physical automation. Using a chain of punch cards or tape, the loom could weave fabric in any pattern. These loom cards, together with the loom-head that read them, exported brain work (memory) and muscle work (the act of weaving). In doing so, humans took another step back, relinquishing control of a machine to our pre-set, written memories (instructions). But we didn’t suddenly invent a new concept of human behaviour – we merely combined two deep-seated human proclivities with origins stretching back to before recorded history. Our muscular and mental automation had become one, and though in the first instance this melding was in the service of so frivolous a thing as patterned fabric, it was an immensely powerful combination.

The basic principle of the Jacquard loom – written instructions and a machine that can read and execute them once set up – would carry humanity’s penchant for automation through to modern digital devices. Although the power source, amount of storage, and multitude of executable tasks has increased, the overarching achievement is the same. A human with some proximate goal, such as producing a graph, loads up the relevant data, and then the computer, using its programmed instructions, converts that data, much like the loom. Tasks such as photo-editing, gaming or browsing the web are more complex, but are ultimately layers of human instructions, committed to external memory (now bits instead of punched holes) being carried out by machines that can read it.

Crucially, the human still supplies the proximate objective, be it ‘adjust white balance’; ‘attack the enemy stronghold’; ‘check Facebook’. All of these goals, however, are in the service of ultimate goals: ‘make this picture beautiful’; ‘win this game’; ‘make me loved’. What we now tend to think of as ‘automation’, the smart automation that Tesla, Uber and Google are pursuing with such zeal, has the aim of letting us take yet another step back, and place our proximate goals in the hands of self-informing algorithms.

‘Each generation is lazier’ is a misguided slur: it ignores the human drive towards exporting effortful tasks

As we stand on the precipice of a revolution in AI, many are bracing for a huge upheaval in our economic and political systems as this new form of automation redefines what it means to work. Given a high-level command – as simple as asking a barista-bot to make a cortado or as complex as directing an investment algorithm to maximise profits while divesting of fossil fuels – intelligent algorithms can gather data and figure out the proximate goals needed to achieve their directive. We are right to expect this to dramatically change the way that our economies and societies work. But so did writing, so did money, so did the Industrial Revolution.

It’s common to hear the claim that technology is making each generation lazier than the last. Yet this slur is misguided because it ignores the profoundly human drive towards exporting effortful tasks. One can imagine that, when writing was introduced, the new-fangled scribbling was probably denigrated by traditional storytellers, who saw it as a pale imitation of oral transmission, and lacking in the good, honest work of memorisation.

The goal of automation and exportation is not shiftless inaction, but complexity. As a species, we have built cities and crafted stories, developed cultures and formulated laws, probed the recesses of science, and are attempting to explore the stars. This is not because our brain itself is uniquely superior – its evolutionary and functional similarity to other intelligent species is striking – but because our unique trait is to supplement our bodies and brains with layer upon layer of external assistance. We have a depth, breadth and permanence of mental and physical capability that no other animal approaches. Humans are unique because we are complex, and we are complex because we are the beast that automates.

To Make AI Smarter, Humans Perform Oddball Low-Paid Tasks


Tucked into a back corner far from the street, the baby-food section of Whole Foods in San Francisco’s SoMa district doesn’t get much foot traffic. I glance around for the security guard, then reach towards the apple and broccoli superfood puffs. After dropping them into my empty shopping cart, I put them right back. “Did you get it?” I ask my coworker filming on his iPhone. It’s my first paid acting gig. I’m helping teach software the skills needed for future robots to help people with their shopping.

Whole Foods was an unwitting participant in this program, a project of German-Canadian startup Twenty Billion Neurons. I quietly perform nine other brief actions, including opening freezers, and pushing a cart from right to left, then left to right. Then I walk out without buying a thing. Later, it takes me around 30 minutes to edit the clips to the required 2 to 5 seconds, and upload them to Amazon’s crowdsourcing website Mechanical Turk. A few days later I am paid $3.50. If Twenty Billion ever creates software for a shopping assistant robot, it will make much more.

In sneaking around Whole Foods, I joined an invisible workforce being paid very little to do odd things in the name of advancing artificial intelligence. You may have been told AI is the gleaming pinnacle of technology. These workers are part of the messy human reality behind it.

Proponents believe every aspect of life and business should be and will be mediated by AI. It’s a campaign stoked by large tech companies such as Alphabet showing that machine learning can master tasks such as recognizing speech or images. But most current machine-learning systems such as voice assistants are built by training algorithms with giant stocks of labeled data. The labels come from ranks of contractors examining images, audio, or other data—that’s a koala, that’s a cat, she said “car.”

Now, researchers and entrepreneurs want to see AI understand and act in the physical world. Hence the need for workers to act out scenes in supermarkets and homes. They are generating the instructional material to teach algorithms about the world and the people in it.

That’s why I find myself lying face down on WIRED’s office floor one morning, coarse synthetic fibers pressing into my cheek. My editor snaps a photo. After uploading it to Mechanical Turk, I get paid 7 cents by an eight-person startup in Berkeley called Safely You. When I call CEO George Netscher to say thanks, he erupts in a surprised laugh, then turns mock serious. “Does that mean there’s a conflict of interest?” (The $6.30 I made reporting this article has been donated to the Haight Ashbury Free Clinics.)

Netscher’s startup makes software that monitors video feeds from elderly-care homes, to detect when a resident has fallen. People with dementia often can’t remember why or how they ended up on the floor. In 11 facilities around California, Safely You’s algorithms help staff quickly find the place in a video that will unseal the mystery.

Safely You was soliciting faked falls like mine to test how broad a view its system has of what a toppled human looks like. The company’s software has mostly been trained with video of elderly residents from care facilities, annotated by staff or contractors. Mixing in photos of 34-year-old journalists and anyone else willing to lay down for 7 cents should force the machine-learning algorithms to widen their understanding. “We’re trying to see how well we can generalize to arbitrary incidents or rooms or clothing,” says Netscher.

The startup that paid for my Whole Foods performance, Twenty Billion Neurons, is a bolder bet on the idea of paying people to perform for an audience of algorithms. Roland Memisevic, cofounder and CEO, is in the process of trademarking a term for what I did to earn my $3.50—crowd acting. He argues that it is the only practical path to give machines a dash of common sense about the physical world, a longstanding quest in AI. The company is gathering millions of crowd-acting videos, and using them to train software it hopes to sell clients in industries such as automobiles, retail, and home appliances.

Games like chess and Go, with their finite, regimented boards and well-defined rules, are well-suited to computers. The physical and spatial common sense we learn intuitively as children to navigate the real world is mostly beyond them. To pour a cup of coffee, you effortlessly grasp and balance cup and carafe, and control the arc of the pouring fluid. You draw on the same deep-seated knowledge, and a sense for the motivations of other humans, to interpret what you see in the world around you.

How to give some version of that to machines is a major challenge in AI. Some researchers think that the techniques that are so effective for recognizing speech or images won’t be much help, arguing new techniques are needed. Memisevic took leave from the prestigious Montreal Institute of Learning Algorithms to start Twenty Billion because he believes that existing techniques can do much more for us if trained properly. “They work incredibly well,” he says. “Why not extend them to more subtle aspects of reality by forcing them to learn things about the real world?”

To do that, the startup is amassing giant collections of clips in which crowd actors perform different physical actions. The hope is that algorithms trained to distinguish them will “learn” the essence of the physical world and human actions. It’s why when crowd acting in Whole Foods I not only took items from shelves and refrigerators, but also made near identical clips in which I only pretended to grab the product.

Twenty Billion’s first dataset, now released as open source, is physical reality 101. Its more than 100,000 clips depict simple manipulations of everyday objects. Disembodied hands pick up shoes, place a remote control inside a cardboard box, and push a green chili along a table until it falls off. Memisevic deflects questions about the client behind the casting call that I answered, which declared, “We want to build a robot that assists you while shopping in the supermarket.” He will say that automotive applications are a big area of interest; the company has worked with BMW. I see jobs posted to Mechanical Turk that describe a project, with only Twenty Billion’s name attached, aimed at allowing a car to identify what people are doing inside a vehicle. Workers were asked to feign snacking, dozing off, or reading in chairs. Software that can detect those actions might help semi-automated vehicles know when a human isn’t ready to take over the driving, or pop open a cupholder when you enter holding a drink.

Who are the crowd actors doing this work? One is Uğur Büyükşahin, a third-year geological engineering student in Ankara, Turkey, and star of hundreds of videos in Twenty Billion’s collection. He estimates spending about 7 to 10 hours a week on Mechanical Turk, earning roughly as much as he did in a shift with good tips at the restaurant where he used to work. Büyükşahin says Twenty Billion is one of his favorites, because it pays well, and promptly. Their sometimes odd assignments don’t bother him. “Some people may be shy about taking hundreds of videos in the supermarket, but I’m not,” Büyükşahin says. His girlfriend, by nature less outgoing, was initially wary of the project, but has come around after seeing his earnings, some of which have translated into gifts, such as a new set of curling tongs.

Büyükşahin and another Turker I speak with, Casey Cowden, a 31-year-old in Johnson City, Tennessee, tell me I’ve been doing crowd acting all wrong. All in, my 10 videos earned me an hourly rate of around $4.60. They achieve much higher rates by staying in the supermarket for as long as hours, binging on Twenty Billion’s tasks.

Büyükşahin says his personal record is 110 supermarket videos in a single hour. He uses a gimbal for higher-quality shots, batting away inquisitive store employees when necessary by bluffing about a university research project in AI. Cowden calculates that he and a friend each earned an hourly rate of $11.75 during two and half hours of crowd acting in a local Walmart. That’s more than Walmart’s $11 starting wage, or the $7.75 or so Cowden’s fiancee earns at Burger King.

Cowden seems to have more fun than Walmart employees, too. He began Turking early last year, after the construction company he was working for folded. Working from home means he can be around to care for his fiancee’s mother, who has Alzheimer’s. He says he was initially drawn to Twenty Billion’s assignments because, with the right strategy, they pay better than the data-entry work that dominates Mechanical Turk. But he also warmed to the idea of working on a technological frontier. Cowden tells me he tries to vary the backdrop, and even the clothing he wears, in different shoots. “You can’t train a robot to shop in a supermarket if the videos you have are all the same,” Cowden tells me. “I try to go the whole nine yards so the programming can get a diverse view.”

Mechanical Turk has often been called a modern-day sweatshop. A recent study found that median pay was around $2 an hour. But it lacks the communal atmosphere of a workhouse. The site’s labor is atomized into individuals working from homes or phones around the world.

Crowd acting sometimes give workers a chance to look each other in the face. Twenty Billion employs contract workers who review crowd-acting videos. But in a tactic common on Mechanical Turk, the startup sometimes uses crowd workers to review other crowd workers. I am paid 10 cents to review 50 clips of crowd actors working on the startup’s automotive project. I click to indicate if a worker stuck to the script—“falling asleep while sitting,” “drinking something from a cup or can,” or “holding something in both hands.”

A video from Twenty Billion Neurons describing its work.

The task transports me to bedrooms, lounges, and bathrooms. Many appear to be in places where 10 cents goes further than in San Francisco. I begin to appreciate different styles of acting. To fake falling asleep, a shirtless man in a darkened room leans gently backwards with a meditative look; a woman who appears to be inside a closet lets her head snap forward like a puppet with a cut string.

Some of the crowd actors are children—a breach of Amazon’s terms, which require workers to be at least 18. One Asian boy of around 9 in school uniform looks out from a grubby plastic chair in front of a chipped whitewashed wall, then feigns sleep. Another Asian boy, slightly older, performs “drinking from a cup or a can” while another child lies on a bed behind him. Twenty Billion’s CTO Ingo Bax tells me the company screens out such videos from its final datasets, but can’t rule out having paid out money for clips of child crowd actors before they were filtered. Memisevic says the company has protocols to prevent systematic payment for such material.

Children also appear in a trove of crowd-acting videos I discover on YouTube. In dozens of clips apparently made public by accident, people act out scripts like “One person runs down the stairs laughing holding a cup of coffee, while another person is fixing the doorknob.” Most appear to have been shot on the Indian subcontinent. Some have been captured by a crowd actor holding a phone to his or her forehead, for a first-person view.

I find the videos while trying to unmask the person behind crowd-acting jobs on Mechanical Turk from the “AI Indoors Project.” Forums where crowd workers gather to gripe and swap tips reveal that it’s a collaboration between Carnegie Mellon University and the Allen Institute for AI in Seattle. Like Twenty Billion, they are gathering crowd-acted videos by the thousand to try and improve algorithms’ understanding of the physical world and what we do in it. Nearly 10,000 clips have already been released for other researchers to play with in a collection aptly named Charades.

Gunnar Atli Sigurdsson, a grad student on the project, echoes Memisevic when I ask why he’s paying strangers to pour drinks or run down stairs with a phone held to their head. He wants algorithms to understand us. “We’ve been seeing AI systems getting very impressive at some very narrow, well-defined tasks like chess and Go,” Sigurdsson says. “But we want to have an AI butler in our apartment and have it understand our lives, not the stuff we’re posting on Facebook, the really boring stuff.”

If tech companies conquer that quotidian frontier of AI, it will likely be seen as the latest triumph of machine-learning experts. If Twenty Billion’s approach works out the truth will be messier and more interesting. If you ever get help from a robot in a supermarket, or ride in a car that understands what its occupants are doing, think of the crowd actors who may have trained it. Cowden, the Tennessean, says he liked Twenty Billion’s tasks in part because his mother is fighting bone cancer. Robots and software able to understand and intervene in our world could help address the growing shortage of nurses and home-health aides. If the projects they contribute to are successful, crowd actors could change the world—although they may be among the last to benefit.

%d bloggers like this: