SpaceX launched from Santa Barbara County the other night. Their flight path was the closest it's been to the West Coast. I was out out on the beach to enjoy it.
The most amazing part was that the second stage was high enough that it was awash in sunlight. The contrail of the rocket was lit up, creating an unbelievable streak of light in the night sky:
So, I got into a SpaceX kick. I watched some videos and stumbled upon something special. In a tour of Starbase in Boca Chica, Elon Musk details SpaceX's 5-step engineering process:
- Make your requirements less dumb.
- Try very hard to delete the part or process.
- Simplify or optimize.
- Accelerate cycle time.
- Automate.
I can tell these steps were forged through years of pain. There is so much wisdom here – not in the steps themselves, but in how the steps are ordered. They are designed to fight against the natural bad habits and instincts of engineers.
Musk says something that should be relatable to all of us: "I have personally made the mistake of going backwards on all 5 steps, multiple times. I automated, accelerated, simplified, and then deleted."
He then goes on to describe a very memorable example of a time where he ran these steps backwards. (The example is from Tesla, where I presume he echos the same steps.)
Fiberglass mats
In the battery pack production line, there was a stage to install fiberglass mats. These mats were installed on top of the Model 3 battery pack. This stage was choking the line, which in turn was choking the entire Model 3 production program.
As the stage was already automated, they decided to invest more into fixing the automation. Can we automate more of the installation by making the robot better? What if the robot moved faster? Can we give the robot a shorter travel path?
That helped, but didn't fix the problem. So next, they optimized. For example, they were spackling the entire battery pack with glue in order to attach the mat. What if instead of spackling glue on the whole pack, they just put little dabs on it? The mat was eventually sandwiched between the floor pan and the battery pack anyway, dabs should be fine?
But these optimizations weren't enough to clear the bottleneck either.
So finally, after automating then simplifying then optimizing, Musk asked: "What the hell are these mats for?" Blank stares.
He went to the battery safety team first: are they for fire protection? "No, they're for noise and vibration."
So, he went to the Noise, Vibration, and Harshness (NVH) team. Are these mats for noise? NVH: "No, they're for fire safety."
Nobody knew why the mats were there and who made the requirement. They decided to run cabin noise tests with and without the fiberglass mats. There was no difference.
The mats – which had costs millions to automate and millions more in production line delays – were useless.
#3: Simplify or optimize
The most important part of the 5 steps is that "simplify or optimize" is the third step. No engineer needs to be told to optimize. It's in our nature. It's what we're trained to do. But, the most common error of a smart engineer, Musk argues, is to "optimize a thing that should not exist."
Why do engineers fall into this trap?
Musk attributes this to our training in school. Everyone has been trained that you have to answer the question. You can't tell your professor: "I don't want to work on this problem, I don't think it's relevant."
I think there are even more dimensions to it.
First, there's so much dopamine one gets from optimizing a thing. It's one of the most satisfying parts of the job.
Second, requirements are often hidden. Requirements don't just live in specification documents, architecture diagrams, or tickets. Requirements are often more pervasive and less obvious. They're already integrated into the design.
In Musk's battery production line example, the fiberglass mats were just there. They're a bad requirement now, perhaps they were a good requirement at the beginning. But the mere presence of the fiberglass mats on the line is an implicit requirement. There's inertia, which is keeping the fiberglass mats in the car and on the line.
It's difficult to constantly question things, easier to keep the momentum going. So, when the mats run into problems, engineers rush to fix the problem, because that's what we engineers do. When engineers redesign other parts around the battery pack, those designs implicitly contain the requirement of the fiberglass mats. Because, well, they're there.
#2: Try very hard to delete the part or process or wool sweater
The "bias in engineering tends to be very strongly towards 'let's add this part or process in case we need it.' But you can basically make 'in case' arguments for so many things."
That part I don't think is an engineering thing – that's a human thing!
I'm reminded of Marie Kondo, casting her judgment upon me as I pull that gray wool sweater I've never worn out of its unvisited corner in my closet. It itches (am I allergic?), it's never fit quite right, and there are basically 3 days a year where it's cold enough in LA to wear it. But maybe...
"In case" has buried hoarders alive in their own homes. Kondo helps people throw things away by giving them little rituals to make it feel OK. (I thank my gray wool sweater for that one hot, itchy night at a New Year's party a few years ago then give it away.)
Musk likewise has a rule to help his teams delete the cruft. It appeals to the analytical nature of his engineers:
"If you're not adding stuff back in 10% of the time, you're not deleting enough."