Insights
What Ford learned in 1990 about buying AI
Ford tested hand-written rules against a system that learned, in front of dealers and technicians. The rules lost. Five questions to ask any AI vendor because of it.
Ford built a rule-based AI system, tested it in front of dealers and threw it out. What shipped was a toolbox of real tests, picked one at a time by a strategy engine that read the last result. A neural network ran alongside it. If a vendor tells you their hand-written logic covers your edge cases, Ford ran that experiment in 1990, and it failed in front of the customer.
The rest of this piece is what happened and five questions you can put to any AI vendor because of it.
What Ford built
In the late 1980s I was the business planning lead for Ford's Service Bay Diagnostic System, called SBDS. Ford chartered it in 1985, and Len Tedesco, who ran it, described the plan in a 1986 SAE technical paper (SAE 861030). My job was to write the product plans, raise the money and ship it.
There was no central funding. I went to the divisions every year for tens of millions of dollars. By the time it was done, SBDS was an organization of about 800 people, larger than a Ford division. It shipped to more than 4,000 service bays, and a unit cost about $40,000, bought outright or on a seven-year lease. Ford launched it on January 17, 1991, and the Christian Science Monitor covered it that March.
The rules lost in front of the dealers
The founding plan called for an expert system. So Ford built one with the Carnegie Group, a firm spun out of Carnegie Mellon University. Ford took a 10 percent stake in the company for $4 million, announced in October 1985.
The system held about 38,000 rules. An expert's knowledge was written down as rules, and a technician reached them through a text screen over a phone line. It had two problems. It cost too much to build and maintain, and it was brittle. A rule tree answers the questions already on the tree. When a car showed it something nobody had planned for, it broke.
The technicians and dealers who tried it hated it. We stopped it. To my knowledge, while I ran SBDS testing, it never released.
What shipped instead
The system that went into the bays was plainer. It was a set of real tests: a fuel-pressure leak-down, sensor checks, a camshaft test for Ford's Modular engine. A strategy engine ran them in order. It did not follow a hand-built if-then tree. It chose the next test from the result of the last one, so the car's own answers steered the diagnosis.
A neural-network pattern recognizer ran alongside it. We built it as a hybrid for two practical reasons. The early cars could not yet give the network all the signals it wanted, so the strategy engine carried the diagnosis where the network could not reach. And a master technician does not want a black box giving orders. Choosing each test from the last result kept the technician reading the evidence next to the machine.
The camshaft nobody trained it to find
We trained the network on a healthy engine first, then on failures. We pulled spark plugs and held valves open so it could watch what a sick engine does.
During the launch of the Modular engine, a division vice president shipped me a car and asked what was wrong with it. The network flagged a camshaft that had been ground out of specification. We had never trained it on that fault. It had learned what a healthy engine looks like well enough to see that this one was not, and it pointed at the part. A lookup table only finds the fault someone wrote down for it. This found one we had not.
The patents are the proof
None of this rests on memory alone. The diagnostic research team filed 18 patents through Ford between 1989 and 2002. My name is not on them. They are public, and these are the ones that show which road Ford took:
- US 5,041,976 (1991): Diagnostic System Using Pattern Recognition for Electronic Automotive Control Systems. Kenneth Marko, John James and James Dosdall. The neural-network road. Its own background section says an expert system needs an expert to write down his knowledge, takes enormous effort to build, and breaks whenever the engine design changes.
- US 5,313,388 (1994): Method and Apparatus for Diagnosing Engine and/or Vehicle System Faults Based on Vehicle Operating or Drive Symptoms. The strategy-driven route that shipped to the bays.
- US 5,044,195 (1991): Misfire Detection in an Internal Combustion Engine.
- US 5,095,742 (1992): Determining Crankshaft Acceleration in an Internal Combustion Engine.
- US 5,528,931 (1996): Engine Diagnostic Monitor with On-Board Adjustment of Correction Factors.
- US 5,531,108 (1996): Adaptive Correction of Torsional and Wheel Profile Position Irregularities for Misfire Detection.
- US 5,559,285 (1996): Fuzzy Logic Method for Detecting Misfires in Internal Combustion Engines.
- US 6,434,541 (2002): Automotive Engine Misfire Detection System Including a Bit-Serial Based Recurrent Neuroprocessor. The Ford and NASA Jet Propulsion Laboratory neural chip.
What it means when you buy AI today
Ask the vendor these five things before you sign.
- Who writes the logic? If a person writes the branches, the system has a ceiling, and the demo will not show you where it is.
- What does it do at the edge of what it knows? The Ford rule system stopped when it hit an unplanned case, so you could see it was stuck. A learning model rarely stops. It answers fluently and does not tell you when it is outside what it learned.
- Can your people see why it said that? The system that shipped told the technician to run a test because of the last result. Technicians keep using systems they can see into and steer.
- Will your staff accept the way they have to use it? Ford's technicians rejected the rule system because of the screen full of questions, not because of the intelligence behind it.
- Is it one method or a hybrid, and why? Ford's answer was a hybrid, built around the limits of the car and the mind of the technician. Ask what your vendor's limits are.
Where it lands
Ford ran rules against learning on real cars and real technicians, and the rules failed in front of the customer. What shipped read each result, picked the next step, and kept the technician in the loop. Judge any AI proposal by who writes the logic, what happens at its edge, whether your people can see inside it, and whether they will accept it.
If you are weighing an AI purchase, that is the evaluation we run. See how we evaluate AI.
Sources
- L. Tedesco, "Service Bay Diagnostic System," SAE Technical Paper 861030 (1986).
- Christian Science Monitor, March 18, 1991 (launch coverage).
- US patents listed above, assigned to Ford Motor Company (Ford Global Technologies for the 2002 filing); records at Google Patents.
- Firsthand account of David LaRue, business planning lead for SBDS: the unit price and lease terms, the number of rules and service bays, Ford's 10 percent stake in the Carnegie Group, the camshaft test, and that the rule system never released while he ran SBDS testing.
