Martian Summer Read online

  Once Carlos was on board, he came up with some initial concepts for the telltale and some math models to understand how temperature and pressure fluctuated near the spacecraft.

  “I had to wait two years to get my clearance,” Carlos says. This was so he could see the layout of the lander deck to model some of the environmental interactions. They don’t have many chances to work on Mars projects at the University of Alberta in Edmonton, so Carlos wanted to involve his students. Plus, he needed the brain power to help him characterize—the sciency way of saying investigate and understand—the effects of the Phoenix heaters on the local environment. The ITAR monster said, “not so fast, you FN scientist.”

  “My students were not cleared and we couldn’t get the fabricators in the machine shop clearance to build any models,” Carlos says. So he devised an extra credit project to one of his classes to try to solve.

  “I called it the Open Lander Project,” Carlos tells me. “The rules were that my students … they would have to use the Internet to try to figure out the dimensions and location of all the instruments on the deck and where they were likely located.” To make sure they didn’t run afoul of the rules, Carlos enforced several restrictions. “Under no circumstance could they ask anyone with restricted information for clarification.” So they didn’t. Using public domain photos posted by NASA and JPL and hosted by Google, Carlos’s students employed the sudoku method to create an accurate model of the lander. Carlos proved that State Department’s ITAR rules—as written—are silly and counterproductive. Red-faced, they repealed them immediately. Just kidding, it changed nothing. That’s why Clive Cook can’t tell me what’s happening at work today. His screen is blank save for a little note that says ITAR restricted.

  RICHARD KORNFELD CALLS THE TEAM TOGETHER FOR A BRIEFING. He describes some problem with the lander’s memory system that’s making it impossible to do much of anything right now.

  “Too much engineering data was produced … there was almost a time out,” Kornfeld announces the news like a breaking bulletin on a tragic local event. “The problem is not fully understood. So, we can’t save anything to flash [memory] at the moment. Are there any questions?”

  “Did we lose the whole day? And do we need to redo sol 22?” Pat Woida asks.

  “I defer to Doug Ming,” Kornfeld says.

  “Well, I’ll defer to Jim,” Doug says.

  With no one left to defer to, Jim Chase is where the buck stops. As TDL, he is the manager of all the bits returning from space.

  “I’d like to pull up the chart that shows what data has been lost,” Jim says. It takes him a second to find what he’s looking for. He turns on the projector. The screen is blank.

  “That’s what happened to our data!” Peter Smith says.

  Chase fumbles for a bit. We all wait while he looks through his folders and pulls up a new chart. Data is the lifeblood of Phoenix. The team uses a set of identifying tags called application process identifications (APIDs). These are the markers that determine how information is prioritized as it comes down from Mars. It’s the process for asking Phoenix for something and then finding the answer back on the ground. It’s a straightforward, million-step process outlined in the end-to-end data management (EEDM) documentation. I’ll take you through it now. Please grab a paper bag if you’re prone to hyperventilation. The only thing that gets me through it are the mind-altering Mars drugs.

  The science team makes a request to Phoenix to generate a certain kind of data product. Those requests are processed by the relay orbiters and sent down to the spacecraft. The spacecraft executes the commands it receives from the relay orbiters and sends them back as a particular kind of data product. Data products are images or results from an experiment like a TEGA bake. Managing all that back-and-forth is a hot mess.

  The computer on Phoenix has two kinds of memory, 100 megabytes of regular flash that’s on any old thumb drive and then this other kind of temporary storage that gets erased and re-written. Once the data stored on this second type of volatile memory is gone, it’s gone forever. The application process identification (APID) is the special instruction that comes along with the request for data. The APID tags tell Phoenix where to store data—in flash or volatile memory—and when to send it back to Earth. It also has a little note for when it is safe to delete. If you don’t get the data you asked for, you can launch an investigation to find out where along the chain it’s gone missing. Did the activity occur? Is it stuck on the relay orbiter? Was it deleted? There are plenty of chances to lose your science data. APIDs determine if and when you get your data. APIDs get fun names like “key” or “crit_key” or “crit.” It’s the tactical downlink lead’s (TDL) job to track all that data. If you get it wrong, your one-time-only non-repeatable experiment could be lost forever. Now something screwy is happening with them. Apparently, only the highest priority APIDs were downloaded. Very few bits of engineering data arrived on Earth.

  “It looks like the RAC images before the surface dig are lost. We can’t get those back. But we’ll move on and hopefully it’s not a terrible loss,” Doug Ming tells the group.

  To recap: something has happened to the lander that caused the whole spacecraft to go into safe mode. Safe mode is like the fetal position. Phoenix is a bit of a nervous spacecraft. It uses the fainting goat approach, folding over and taking an extremely defensive position at the first sign of danger. Now Phoenix will wait for the science team to tell it everything is okay. When everything is back to normal, it gets up on its own and goes on working. This sol’s plan was not executed and a day of science was lost. Some data from the previous day is missing or lost. Whatever went wrong is disrupting the flash memory. That means no data can be saved. All the activities they do now will have to be completed and then transmitted back to Earth and there’s no memory back-up if the transmission fails. This will limit the number of activities the team can accomplish each day.

  “Anomalies happen, but I want to show you this,” Doug says to the team. He opens a document with a grid that shows everything Phoenix must do to be considered a success. There are quite a few boxes checked. Many are the obvious ones, but still important. Land safely. Check. Acquire first sample. Check. “We are still marching along down the path to a successful mission.”

  I also like to put stuff I know I’m going to do on my daily list to make me feel good. Eat breakfast. Check. Put on two socks. Check. These ticked-off boxes remind us that we’re doing a lot of great work here.

  “I’d like to acknowledge the spacecraft team for identifying the [memory] issues very quickly and stabilizing the craft. It’s to their credit that we have the opportunity to do science today,” Richard Kornfeld says to the group. There’s a round of applause. Then it’s back to business.

  Bob Denise, a senior engineer who built a big chunk of the software that controls these interactions, volunteers to help explain after I stalk him for a couple hours.

  “I’ll try to explain it to you without violating ITAR rules,” he says. I tell him not to worry, I’m a citizen. He’s not interested. Better safe than in a federal penitentiary. “The computer that tracks on-board data generates its own data,” he says. “Something is wrong with the software. It keeps recording the same data in a loop that it was generating for downlink.” The information downloaded by the lander is organized by APIDs. “This key bit of engineering is the highest priority, lowest quantity data on the spacecraft. Since it’s the highest priority, it gets downloaded first. There was so much of it, it took up all of the downlink,” Denise explains.

  “In the short term, we can adjust the APIDs,” he explains. The team can code the data that has been causing problems as “low priority” so APID will not load it until later in the cycle and won’t ruin what everyone is currently working on, but that will just be a band-aid.

  Denise says it became a serious situation because the volume of data increased the boot time of the spacecraft. That’s how long it takes for Phoenix to get up and ready i
n the morning. This time is closely monitored with regular check-ins. Are you ready? I’m coming? Are you ready? I’m coming, damnit. They call these check-ins “heartbeats.” Phoenix was so busy processing all the data that it took an even longer time to boot up.

  “Then it came dangerously close to missing a heartbeat and changing sides,” he says. If it misses a heartbeat, the computer assumes something has gone very wrong and moves to backup mode. It switches hemispheres in its computer brain.

  “The other side of the spacecraft is like a spare tire. Once you use it … well, you don’t want to use it,” Denise tells me. Because of all this, the flash memory where extra data is stored when the spacecraft goes to sleep does not work. The mission will hobble along for the next few sols, possibly longer. This is annoying but not catastrophic. It is also about the limit any human can take for talking about APIDs.

  DARA SABAHI IS BACK IN THE SOC TODAY. SABAHI IS THE YODA TO PETER Smith’s Luke Skywalker—a mythic figure that only appears in the SOC for brief moments of consult at critical junctures. He is the chief engineer for Phoenix. If he’s here, Sabahi paces around the perimeter of SOC and swoops in to aid any troubled group. He rubs his head and kneads his chin before approaching a subordinate and whispering some prescient advice in his ear. He has a broad build and a formidable widow’s peak. His arms are always folded and slumped; he’s easily positioned to scratch his head or engage in other classical deep-thought postures. He wears a photographer’s vest and camping pants that zip at the knee. We can only imagine what he keeps in all those pockets. In spite of his intimidating and sometimes dour look, it’s important not to judge an engineer by his vest.

  “You should be pushing for better access,” he says after asking me probing questions. Wait, I’m supposed to be pressing you for information! This is awkward. He is so good at quickly getting to the heart of a problem. “Documenting the mission will be very important for the future. I am always looking for a record of the last mission when I start on a new one. This is how we learn,” he says. I suddenly feel like I need to be a better person. You complete me, Dara. I want to tell him that I don’t think I’m doing what he thinks I’m doing.

  “I’m counting on this documentation,” he says. This is just getting worse. “Don’t worry. Peter will make it happen,” Dara says. And while I feel very small, it’s the first time I feel part of the team. It doesn’t matter that my paltry brand of investigative journalism is out of scope for him. He’s deduced and attacked the issue.

  “The more people can read about the mission process, the more we can learn about improving the process. You need to be close to the action,” he tells me. “And the action increases as you get closer to the ‘sausage grinder.’” The sausage grinder is how he describes the process of turning the scientists’ wishes into an actual plan the lander uses. These are the highly technical shift II meetings where tensions run high and young observers get confused. I don’t want to get kicked out because someone misplaces their anger and complains. Dara tells me not to worry.

  “You know, NASA is actually an open culture,” he says. “But emotion can get in the way.” He suggests I get into the anomaly response meetings. I say I’m not sure about pressing my luck yet. I want to be here the whole summer and I don’t want to step on any toes. He says with confidence that Peter will help. He doesn’t know Peter takes more of the tough love approach with the people he finds to document his mission.

  Peter needs to tell the anomaly leads that I’m okay to go in, he says.

  As chief engineer, Sabahi’s job is pretty easy: make sure Phoenix worked. He has the worry lines to prove it was stressful. Before I ever talked to him, I saw him as some JPL apparition to be feared. Sabahi is not just an engineer of machines, but of man too. He talks about the problems of making a mission like this work when so many smart people come together with different agendas. He says there is only one way to do it.

  “We create a very stressful situation to train them,” he says. These are the training missions all Phoenix ops staff attended. Stressing the team sounds cruel.

  “That’s how you get smart people who are all used to being right to work together. You have to tax them. Then they can pull together as a group and support one another,” Sabahi says.

  He understands how large-egoed, big-brain folk are motivated.

  “Most people are afraid to confront risk—they don’t understand it—because they haven’t faced it over and over again. A mission like this is about managing many points of risk. In situations like Phoenix you have risk. But if you don’t confront it and move on, you just make it more likely that another issue will come up or the sun will start to set [and the mission will be over].” Dara then starts to levitate because he’s about to reach aerospace nirvana. At least that’s how it happens in my mind.

  Sabahi is one of the few people who sees the entire space production chain.

  “From the newest technician to the heads of departments, I see how people work,” he says. “It generally goes like this: selfless dedication from the technicians and gutless decision-making from the top.” Those who make the gutless decisions don’t understand risk.

  “That’s the problem with the people who make those decisions, they are only afraid of embarrassment. They don’t have the courage of character to understand and explain,” he says. Uh-oh, I think I’m one of the people afraid of embarrassment. Damn.

  “For instance, Pathfinder is seen as a failure from the JPL perspective. The whole world feels like it was a success but because some rules weren’t followed it’s considered a failure. These are political decisions. That’s why I’m burned out and need to get away.” Sabahi says that as soon as he’s done here, he’s quitting. He’s going far away. This doesn’t just strike me as sad; it kind of makes me worry about the fate of humanity. I might be negligent in tossing around the word brilliant when describing folks in Mission Control, but when the guys I casually describe as brilliant describe Sabahi as the most brilliant guy they know, then you corroborate the fact that yes, he’s brilliant.

  “They pushed Dara Sabahi too hard,” Peter says. “He is great at what he does, but they risk losing him.” He got Phoenix to the ground safely, but it came at a high personal cost.

  Phoenix is a sinkhole. Addiction, burnouts, breakdown, cancer, divorce—and that’s just the “A through D” file. Phoenix will take everything you have to give. She’s a remorseless, soul-sucking machine. For example, the simplest part on Phoenix is the bio-barrier. It’s a bag that protects the robot arm from contamination, and that little cover-up comes with an 18-page document detailing how it might fail. If the bag breaks, snags, bursts, or busts, the mission would be finished. That’s just a bag. In the landing sequence alone, there are more than 200 points of failure that had no backup systems. If any one of these goes, the lander goes. In trying to quantify the risks associated with these points, there’s a strange Faustian statistical process. Let’s say you have a 2% probability of failure. It might be extremely expensive to get that down to 1%.

  “You might tell yourself we should not spend the money to reduce the probability by 1% because that’s not very much. But if you examine it from the failure side, moving from 2% to 1% means cutting your failure rate in half. That makes a big difference. So how do you decide when things are safe enough? They never are. And it can overwhelm you,” Peter tells me one day, trying to explain why missions are so expensive and complicated. You just do your best to understand the risk, then you hold your breath and cross your fingers.

  IT’S 9:19 A.M. LOCAL TUCSON TIME, EARLY EVENING ON MARS. I’M learning to deal with risk and feel rejuvenated after my long talk with Dara. No, I’m more than rejuvenated; I’m inspired. I wander around the SOC. Everyone is busy and stressed out. But I’m confident they’ll work through it.

  Today there is an open house. It’s an opportunity for the public to sit in the briefing area. There will be lectures, a chance to meet a scientist, and it comes with free snow cones at the
end. Now that I’m becoming a real insider, I decide to shake things up and visit the open house with the rest of the hoi polloi—err—space fans. I take my seat with nearly a hundred other locals who have turned up for Carla Bitter’s early morning presentation, which begins with an animation of the landing sequence. Maybe I’m a little sentimental, but I can’t not think about how hard Dara and all the engineers worked to make this happen. The people in this room have no idea that what they’re seeing is impossible, and only through the hard work of thousands of people working together and a bit of luck can you make the impossible possible. I want to shake them and then remind them that even getting within a 10 kilometer radius of the landing site requires more precision than hitting a golf ball in Washington, D.C. and get a hole-in-one in Sydney, Australia! And there are very few people who even have the balls to do it.

  “Welcome!” Carla says to the space fans. She describes all the elements of the mission, the sample trench, the naming conventions, how a sample gets from the trench into Phoenix. There are pictures and diagrams and animations. The visitors—young and old—applaud and ask questions. Then we eat dirty ice slushies.

  CHAPTER NINE

  MISSING PIECES

  SOL 24

  WITH JUST THREE MINUTES UNTIL THE DROP-DEAD UPLINK time (DDULT), the sequencing team pushed their plan out into space. That’s the closest they’ve come. Three minutes later and the Mars Reconnaissance Orbiter (MRO) would have flown over Phoenix and remained silent. Raising its SSI camera eyes to the heavens, the little lander would have put up its robot paw and said sadly, “Why have you forsaken me?” Thankfully Phoenix got its plan and was never the wiser. The final moments of a plan are always exciting, but this would have been one to remember.

  If you recall, the organizational scheme of the Phoenix planning cycle is broken into shift I (downlink, tactical planning, and strategic planning) and shift II (turning the tactical plan into robot code). Tactical planning is for the activities you’ll do the next day, and the strategic planning is for all the activities you’re going to do after that. The day is organized as a sort of “Y”-shaped structure. Shift I builds the strategic and tactical plans, and shift II implements these plans. The two teams form the branches of the “Y.”