3 “easy” things that are hard for both humans and AI

Share

Everybody is agog at what AI systems can do. Nobody thought even 10 years ago that machines could be trained to recognise images or transcribe natural speech as well as they do now. And because of this leap forward everybody has started worrying about AI taking over the world because it will soon be able to do everything people can but better.

On the other hand, there are AI naysayers who point at incredible feats of human creativity and ingenuity and say ‘no machines will ever be able to write a poem’ or ‘manage a company’.

While I’m more than skeptical about the true possibilities of AI, I am equally ekeptical about this supposed limitless human creativity that is beyond the bounds of computation.

I think we can reveal more about the limits and nature of human intelligence and thus the targets (possible limits) for AI development, if we look at very simple things with which both humans and AI struggle albeit in different ways.

Machines are often thought of as capable only of algorithmic processing (such as adding lots of numbers) and humans are thought to excel at massively parallel tasks – also known as intuition (such as telling part dogs from cats). But we will see that they seem to trade these roles in the ways they approach and fail at these appartently simple problems.

I call these problems ‘easy’ because they can be broken into very easy and straightforward components. But they are hard if not impossible in reality because of the curse of dimensionality. Even the slightest variation in those simple components, will grow into an exponential mess.

1. Figuring out time zones

Apple Watches recently stopped working because of Summer Time in Australia. And just the other day, Outlook asked me if I wanted to switch to a continental time zone in Europe. After I said yes, it started scheduling all meetings 2 hours off.

On the other hand, I’ve been arranging meetings between 2 time zones 1 hour apart for 20 years and I still get it wrong about 3 times out of 10.

So what gives? Time zones are conceptually very straightforward. You just have a database of times and places with notes on what time it is when and where relative to some fixed point. Then all you do is subtract anywhere between 1 and 12. What could be easier?

Well, you have to add in change of dates, so you have to switch between today, tomorrow and yesterday quite a lot. But still. There is a finite number of times and places and their combinations, so how hard can it be for human programmers to sit down and write all the code once and for all? Turns out, incredibly hard. There are just too many permutations and they keep changing as the database of times and places is being updated with new information.

So, the magnitude of the problem seems to be too great for humans to come up with exhastively detailed algorithms to deal with it. (To be clear, the core has been solved, but we don’t seem to be able to nail all the edge cases. Things would be a lot worse without computers.)

So why don’t we unleashe machine deep learning on the problem? Well, partly because there’s no good data for a machine to learn on. This is mostly an algorithmic problem. But the inputs of the algorithm come from very human perceptions of how time relates to cyclical things like days and relationships like comparing states between time zones with respect to days. Again, none of this is all that complex. But the algorithmic part is too complex for humans to describe as a series of if-then commands to a computer without making lots of mistakes. And the perspective and context part seems to be completely outside of what any ML algorithm can access at the moment.

So we’re stuck with something that mostly works but not always and is mostly understood but also always confusing.

2. Scheduling a meeting

Scheduling meetings is another simple algorithmic problem related to time. Simply compare two series of numbers, find where they differ and spit out the result. But all of this starts interacting with a lot of human complexities that make the problem completely intractable if what we wanted to do is write a series of commands in the form of ‘if you see this, do that’.

That’s why the work of the human assistant handling the scheduling for a busy person (who rates an assistant) is not just to provide her intelligence or understanding of calendars. It involves conversations with the person whose calendar is being managed about their priorities, options, possible scenarios, conversations with other people, other assistants, eventually arriving at some compromise which is then entered into the straightforward if-then format of a calendar.

It is this conversation with other people that is often overlooked (“tell your people to call my people”). In the case of the assistants of the other busy people who rate an assistant also have to synthesize the priorities and value of their charges through the same process of conversations and adjustments.

The final matching algorithm is very simple – so simple that it seems like noone should need a human assistant any more. But the inputs into the algorithm need to come from sources that are too rich and complex to treat algorithmically or through some multi-dimensional analysis of hidden regularities (like deep neural nets). The inputs require either a fairly general artificial intelligence (although not full blown AGI) or that everyone keeps their calendar in the same way. (Even then we’d probably have to deal with travelling-salesman problems – but at least we have some ideas about the limits on the computability of those.)

There are many individual components of this process that could be algorithmically assisted. But often the simpler algorithms and heuristics such as preference polls and shared calendars are more effective aids than opaque machine learning output.

So although, this looks like a problem that should be solvable through ML, early attempts have been less than impressive.

3. Importing data about people into a table (deduplication)

Computers are great as aids to managing structured data. But the input into the structure has to be provided by humans. Can AI help here?

Imagine you’re organising an event and you want people to tell you if they’re coming. You can just ask and keep in your head who said yes. But that soon becomes too much. As a next step, you ask them to email or to send an RSVP so you can look at the messages and remind yourself who said yes. But even that becomes difficult soon, so you start a list. And the more people and events you need to manage, the more complicated the list is and the more time you have to spend structuring your data and inputing it into some sort of a table for managing and reviewing the data.

The world is littered with Excel sheets kept by event organisers. Now imagine you wanted to feed all the idiosyncratic Excel sheets with event information at a large organisation into a machine learning algorithm and get one number of the total number of participants or the total cost of lunch breaks.

If everybody kept their spreadsheets exactly the same way, this would be trivial. But they don’t. Computers make the task of managing this kind of structured data much easier but they constantly struggle with errors in the input from busy, overworked and cognitively limited humans.

On the surface of it, there’s nothing to prevent this part (ie participant registration and management) of event management from being completely automated. But there’s always a person involved in dealing with this. So could there be an AI system that does all of this? So far we’re not even very close to this. A system that processes some RSVPs via email, others via forms, and others from other sources (“Hi, Clare, Frances told me she was coming to your party!”) does not exist.

So let’s simplify the task even more. Take data from one table of a system (let’s say a registration table) and put it into another table on a different system (let’s say account creation). All the AI system would have to do is figure out what is important to one system and get the right data from another system. At the moment, humans are involved. In better cases by creating an API and programming algorithms to transfer data between systems. In the worse case, they download a spreadsheet from one system, modify it, if needed, and upload it into another system.

This is trivial if you’ve designed both systems and know they have to integrate. But the permutations get out of hand surprisingly quickly when you take any 2 random systems designed by different people for a similar purpose but without the intention to integrate. Is ‘Last name’ always the ‘Second name’, when the full name is in one column, is first name always first? Any one difficulty is easy to spot for a human and disambiguate. But it gets very error prone at scale and there are always some unexpected edge cases.

Even such a simple thing as contact deduplication between two devices of one person is not a completely solved problem.

Why isn’t there an AI system that can evaluate the data and transfer it as appropriate but at scale and without the errors human data processors or programmers or data processing algorithms make?

As always, even the most trivial algorithms require very complex inputs. And with even minor variation in the possible inputs, the if-then logic becomes too unweildy. Although computers in general are great at pursuing if-then logic chains regardless of complexity (within limits), AI algorithms are not. They provide guesses with probabilities. In certain areas, most notably speech and image recognition, their guesses are becoming very good and resembling humans. They may even outperform humans at scale.

But all the if-then part of what to do with these guesses is still handled by if-then algorithms designed by humans. There’s some talk of ‘Programming 2.0’ but nobody seems to be applying it to some of the day-to-day simple problems with complex scaling. Because even small errors in the inputs result in big aggregate problems and AI systems have no way of assessing whether their guesses ‘make sense’.

Is AI impossible?

Maybe AI is just too hard. But these examples don’t claim it’s impossible, they just show that some difficult problems are just difficult. Even if they appear straightforward on the surface.

I have learned not to bet against engineers’ ability to figure out solutions in the long run. It’s not always clear what is solvable by AI and what is not ahead of time. Sometimes, specialised ML systems can be developed to solve problems that don’t generalise (e.g. GO or chess machines). But I would have expected more people to deal with these problems, if there was an easy solution to be found. And there hundreds more similar task-based problems that just won’t be magicked away by slapping the label ‘AI’ on it. Individual ones may be solved by one way or another. Perhaps by breaking them into component parts. But I’m not seeing any specific steps being taken to create general purpose Machine Learning that would deal with all of them. Just wishful thinking about AGI (Artificial General Intelligence) emerging to solve these problems without regard to the actual complexity of some of them or the complexity of the intermediate steps it would take to get there.

Innovation is bad for business: 3 more ‘I’ words to compare innovation to

Share

Innovation is the ‘in’ thing. Innovate or die is the buzz up and down the hive mind. Everybody is feeling like they must innovate all of the things all of the time. But is the incessant innovation the right mode of approaching this?

We constantly spin up stories of the intrepid innovator and the change they bring about in the world. But is that what is really happening on the ground? I think we can bring up some metaphors to bear on this that may open up some different possibilities of conceptualizing innovation.

Note: I’m writing this as an early adopter and some time professional innovator. But despite a personal drive to constantly try new things, I find that approaching innovation uncritically and without regard to the full diversity of its instantiations is counterproductive. Therefore, this is meant to be a corrective as much to myself as others.

Infection as metaphor of innovation

When an innovation spreads through an institution, it does so just like an infection. It starts attacking existing systems who then have to spend time and resources on averting the damage done by the attack. We only tell the stories where the attack led to the strengthening of the system – like with childhood maladies or inoculation. But chronic disease and death of the system are also not uncommon – we just don’t tell them in association with innovation.

This weakening happens through many processes that every innovator and innovatee (and this covers most people in one way or another) will have direct experience of. Innovators have experienced resistance, doubt, slow response times. Those are all defense mechanisms the disease of innovation has to overcome.

The people who experience the innovation as the infected cells, can attest to lower productivity because of the need to learn new things, endless meetings on how to implement the new thing keeping them from doing the job, miscommunication and misunderstanding leading to higher error rate (be it in production or management).

Where this metaphor breaks down is that if the innovation is successful, the system is transformed. Almost, as if instead of a tumor killing us it would grow us a new useful organ while others may fall off without much harm to the organism as a whole.

On the other hand, the success of the system in extinguishing a malignant innovation can make it more resilient to innovation in the future. And this may lower its chances of survival in the face of environmental changes that make it easier to thrive for those where innovation took the system over. This latter aspect is what proponents of innovation as an unalloyed good point to. But that is a backward perspective. From the ground, the system as an organism will always have to start by defending against innovation as infection no matter how well-intentioned everybody involved may be.

Ignorance as metaphor of innovation

Innovation is often equated with knowledge. People research new ways of doing things, they bring together existing strands of knowledge and weave from them beautiful tapestries of brighter futures. But in practice, innovation almost always benefits from ignorance. Or even depends on it.

Ignorance by inventors is a well-known companion to some of the biggest inventions and discoveries. When Morse set out to create the telegraph system, it was received wisdom that what he was trying to do was physically impossible but he did not know that. Columbus is often given as the example of the innovator who was right and pursued his correct knowledge in spite of ridicule. But the truth was he was a zealot crank and completely wrong about everything. Everybody had known the Earth was round for over a thousand years by the time of Columbus and they also knew how large it was. He did not doubt it was round but he subscribed to a crank theory that it was much smaller than it was. He just got lucky there was a continent in the way.

This ignorance-of-the-impossible narrative can be applied to many of the other famous inventors. But this story is often told with the naysayers as hidebound blocks to progress and inventors as courageous pursuers of the truth. But this way the narrative misses the overwhelming majority of ignorant would-be inventors being simple cranks. For every clueless Morse and cranky Columbus, there were thousands of unknown failures who did not know or believe something truly impossible was impossible (just like most random mutations do not win the natural selection lottery). The correct response to somebody claiming they are able to do something known to be impossible is to doubt it. The trick is being able to update one’s priors in a way that helps us better judge the signs of success.

But we should also not overlook the ignorance among the adopters of innovations. The lack of information on the side of adopters of innovation is another necessary ingredient to success. Every innovation is too uncertain and often unreliable in its earliest stages to be considered by the well-informed other than as a bet. This ignorance is partly a result of pure uncertainty as to the viability of something new. But much more commonly, it is just ignorance of how the new thing works, what are its limitations – and how it truly differs from the old. This results in Potemkin innovations like the original Mechanical Turk. The innovations or their effects are often too complex to be fully understood (even by their inventors). This then leads to the creation of a zeitgeist (a sort of general framing) which provides the innovation with enough vectors for infection and the possibility for improvement.

But it also makes it very easy for impostors to sneak in. At present, there are many examples of companies simply labeling products with ‘machine learning’ or ‘blockchain’ and selling it to credulous investors and customers even if the underlying technology is not actually using anything that could be meaningfully described that way.

Imitation as metaphor of innovation

Innovation is associated with inventiveness and creativity. Strokes of brilliance and flashes of genius. But almost all of the great innovations were imitating a previous less successful attempt. It is well known that great inventions and discoveries often appear multiple times simultaneously as different people synthesize available information into similar outcomes. Perhaps the most famous examples are Newton and Leibniz for calculus and Darwin and Wallace for natural selection. But this holds true for almost all the great inventions. Either somebody figured it out as well, or was getting very close.

But more importantly, by the time we get to talk about almost any innovation, it will have reached us through a long chain of imitations. Novel ways of thinking or doing things really only become innovations when somebody copies them.

This process of imitation is similar to that of natural selection, so it usually leads to refinement and strengthening of the original idea. But let’s not forget that natural selection is based around the idea of imperfect copies (random mutations) finding uses that increase their chances of spreading (reproduction). So as part of this metaphor, innovation without copying would just be lots of random ideas that go nowhere.

Not only is innovation the result of imitation, without imitation, there would be no point to it in the first place.

Conclusion

There is not meant to be conclusion here. Investigating metaphors just opens up new prisms that slightly change the way we look at things. Sometimes, it’s the process of thinking through mappings in the metaphor that forces you to investigate one of the domains more closely. And that’s what this is all about.