I suspect the world would be better if that percentage were even greater.
Donald Michie, Alan Turing, Martin Gardner, and Tic Tac Toe
As anyone who reads my blog with any regularity will tell you, I like to read and learn new things. The problem with being self taught and also easily distracted means that you often learn a great deal, but don’t always perceive the connections and scope of what you are learning. I found another example today while surfing.
Years ago, I remember reading one of Martin Gardner’s Mathematical Games columns (from March, 1962, in case you want to look it up) where he described an interesting machine for playing tic-tac-toe. It was made entirely out of matchboxes, each one of which had a tic tac toe position on the top. Inside was a collection of colored beads. Each color specified a possible legal move for the position on top. The idea was that you’d play a game by drawing these beads from the appropriate box, and making the appropriate move. At the end of the game, you’d remove the bead from the last box that sent you along the losing path. Eventually, all the losing moves get removed, and the machine plays perfect tic-tac-toe. Gardner showed how this same idea could be used to create a matchbox computer to play hexapawn, a simple game played with six pawns on a 3×3 board.
I really haven’t given it much thought since then. Many of you have probably read this article in one of the collections of Gardner’s columns.
But today, I was surfing through links and reread some of the details. I found that the machine was called MENACE (Matchbox Educable Naughts and Crosses Engine) and was invented in 1960 by a gentleman named Donald Michie. And it turns out that he’s a pretty interesting guy.
He was a colleague and friend of Alan Turing, and worked with him at Bletchley Park. Apparently Michie, Turing and Jack Good were all involved in the British code breaking efforts, and in the creation of Collosus, the first digital programmable computer which was used to crack the German “Tunny” teleprinter code. (Good and Michie were apparently two of the authors of the General Report on Tunny, a report on the cracking of the code which has only in recent years become declassified). None of this work could have been known by Martin Gardner at the time of this publication. Of course, this was also true of Turing’s work as well.
Turing made a huge impact in several related disciplines: in mathematical logic and computation, in his wartime efforts in code breaking, and in his role in creating some of the first digital computers. Turing also became interested in mathematics in biology, writing about the chemical foundations of morphogenesis and predicting oscillatory chemical reactions. Michie received a doctorate in biology from Oxford, but returned to have a profound and lasting influence on artificial intelligence. Oh, and that modest little paper on Tic Tac Toe? One of the first instances of reinforcement learning.
Very cool, to discover that the little bit of reading you did as a teen, which seemed like an insignificant game at the time, actually has profound threads which stretch out into lots of different areas.
Donald Michie’s Home Page
Trial and Error, the paper describing MENACE
Comments
Comment from Mark VandeWettering
Time 8/28/2011 at 10:24 pm
I second the sentiment Elwood. Turing’s wartime contributions to cryptography saved countless lives. His contributions to mathematics and logic fueled the computer revolution. And with all that, the bigotry of the times didn’t allow him to find a place in the world he helped protect and expand. We only had 41 years of his genius on the planet. Imagine what the world would be like if we had twice as much of him…
Next year is the Alan Turing Year, marking the 100th anniversary of his birth. To warm up, I’ve begun to read lots of Turing biographies. I’ll post some more detailed reviews of them in the next few months. But don’t wait: I’d submit that all aspects of Turing’s life and work are worth studying.
Comment from Roger Chambers
Time 1/23/2012 at 4:50 am
Donald Michie was my supervisor in 1965 for my M.Sc. dissertation which described an impementation of the tic-tac-roe learning device as a computer program. He then invited me to Edinburgh as his research assistant to work on the next stage, which was a heuristic learning program called BOXES. This learned by trial-and-error to balance an inverted pole mounted on top of a trolley running up and down a straight track (rather like balancing a walking stick on a finger tip, but constrained to a flat vertical plane rather than a full 3-D space).
Comment from Mark VandeWettering (K6HX)
Time 1/23/2012 at 9:07 am
Thanks for the comment, Roger. I seemed to recall reading the BOXES paper at some point, but a quick Google didn’t reveal any links to this paper (the Internet is good for finding modern papers, less so for the “dawn of computing”). But it’s very cool that someone who helped explore what has become a sort of benchmark problem in control theory could find their way to my little blog and leave a comment! Thanks again for taking the time!
Pingback from What kind of reinforcement learning is MENACE? – FAQs System
Time 5/24/2014 at 7:46 pm
[…] The famous MENACE matchbox computer for playing tic-tac-toe, invented by Donald Mitchie, is an early example of a reinforcement learning algorithm. Here is a description: […]
Comment from Malcolm
Time 3/19/2015 at 4:07 pm
Great foundational work. Michie’s MENACE is an insightfully simple learning machine yet harbours the seeds of reinforcement learning and heirarchical learning.
The BOXES paper from Machine Intelligence 2 paper is available at aitopics
BOXES:AN EXPERIMENTIN ADAPTIVE CONTROL by D.MICHIE and R.A.CHAMBERS
http://aitopics.org/sites/default/files/classic/Machine_Intelligence_2/MI2-Ch9-MichieChambers.pdf
Comment from Elwood Downey, WB0OEW
Time 8/28/2011 at 9:16 pm
And to think there are still today many who would dismiss, even condemn, a man such as Turing simply because he was gay. I can only hope in another few generations such attitudes will appear barbaric, much as the need to struggle for women’s suffrage appears barbaric to us today.
Then speaking of connections, when visiting a new town or city, I find it very instructive to research the names of streets. Far from random, they often commemorate significant events or colorful characters in regional history.