For participants only. Not for public distribution.

Note #45
2004 Grand Challenge postmortem

John Nagle
Last revised June 22, 2004.

A quick, preliminary analysis of what happened.

The competition

For the record, here are the rather disappointing results. No vehicle went much beyond mile 7, out of 142 for the course.

2004 DARPA Grand Challenge Results
#
Team
Miles
Status
Notes
22
Red Team
7
Disabled

Plowed through sheet-metal fence.- N.Y. Times photo.

"At mile 7.4, on switchbacks in a mountainous section, vehicle went off course, got caught on a berm and rubber on the front wheels caught fire, which was quickly extinguished. Vehicle was command-disabled." -- DARPA

Detailed technical analysis from CMU

21
SciAutonics II
7
Disabled
Vehicle went into an embankment and became stuck.
5
Team Caltech
1
Disabled "At mile 1.3, vehicle veered off course, went through a fence, tried to come back on the road, but couldn’t get through the fence again. Vehicle was command-disabled." -- DARPA
7
Team DAD
6
Disabled
"At mile 6.0, vehicle was paused to allow a wrecker to get through, and, upon resuming motion, vehicle was hung up on a football-sized rock. Vehicle was command-disabled." -- DARPA
25
Virginia Tech
0
Disabled
CajunBot failure

Vehicle brakes locked up in the start area.

23
Axion Racing
0
Disabled Vehicle circled the wrong way in the start area.
2
Team CajunBot
0
Disabled
CajunBot failure

Vehicle brushed a wall on its way out of the chute.

13
Team ENSCO
0
Disabled
Ensco crash

Vehicle flipped in the start area, experienced a fuel leak, and the team needed to shut off the fuel.

4
CIMAR
0
Disabled At mile 0.4, vehicle ran into some wire and got totally wrapped up in it.
10
Palos Verdes HS
0
Disabled Hit concrete obstacle at start.
17
SciAutonics I
1
Disabled At mile 0.75. vehicle went off the route. After sensors tried unsuccessfully for 90 minutes to reacquire the route, without any movement, vehicle was disabled.
20
Team TerraMax
1
Disabled
OSU crash in QID

"Several times, the vehicle sensed some bushes near the road, backed up and corrected itself. At mile 1.2, it was not able to proceed further. Vehicle was command-disabled." - DARPA

Hit stationary vehicle in QID

15
Team TerraHawk
0
Withdrawn
9
The Golem Group
5
Disabled At mile 5.2. Vehicle stopped. Vehicle had a throttle problem while going up a hill. After trying for 50 minutes, the vehicle was disabled.
16
The Blue Team
0
Withdrawn
Blue Team fall

Operator forgot to start balance gyro before start.


Final vehicle positions

The planned route

142 miles.

Note mountain ranges to be crossed. No vehicle made it past the first hill.

Laser rangefinding

All the vehicles which used laser rangefinders had line scanners only. Nobody had a 3D scanner. Nor did we, although, as you all know, we tried to have one built.

CMU had four line scanners, three fixed and one steerable. The fixed ones were aimed to the left, the right, and slightly upward. The steerable one was presumably aimed as far forward as it can get good data. This sounds good, but it's not that effective a system. All of them were single-line scanners. So the vehicle has to assemble ground profiles from successive line scans. The hard-mounted units weren't stabilized, so they wouldn't produce good profiles beyond slow speeds. The long-range gimballed Reigl scanner was stabilized in three axes, but it's still a line scanner. It only gets one chance to see any point on the ground, and it sees it at maximum range and at the most oblique angle possible, the worst possible condition. Any problems, and you have to slow down and try to use the gimbal to pick up the missing data. Or you can just go plowing ahead, which is apparently what they did.

I think this establishes that line scanners aren't going to solve this problem. CMU had the fanciest single-line scanner ever built, and they crashed into a very clear obstacle. We can test and debug with our SICK LMS 221 on a tilt head, but ultimately, we have to solve the 3D scanner problem.

We're in Silicon Valley. This is possible.

Vision

Team DAD, in Morgan Hill, CA, did quite well with cameras only. That's impressive. They made it to mile 6, where they were paused by DARPA to allow a wrecker to pass. After restart, they were unable to continue. I suspect that they had a stereo-from-motion system, which is a major achievement. The weakness of stereo from motion is that you get no data when stationary. That may be why they were unable to restart after the pause. We should talk to them.

Other systems

Nobody exercised the vehicles very hard, other than by running into obstacles. It wasn't especially hot, cold, or windy today, and the vehicles were still on a relatively flat part of the course. Nor did any vehicle run for very long. So we don't have any new information about what works and what doesn't for off-road desert operation.

Summary

If we'd been able to build our original design in time, we probably would have done better than anyone else. Even our cut-down final design, without the custom laser rangefinder, should have been able to do at least as well as any entrant this year, although it wouldn't have been able to go as fast as the CMU entry.