Missiles and Rockets

Missiles and Rockets was a magazine that ran from the mid 1950's to the mid 1960's at the height of the Space Race. All the issues are available on the internet archive. It's neat to see some of the old concepts (like the manned rocket bomber above), and the advertisements in the early issues by companies trying to hire engineers and scientists so they can cash in on the flood of funding in the early gold rush days are pretty entertaining.

Some interesting aerodynamics & control details on the re-design required for the Falcon Heavy at 15:20 or so. Great launch!

Funding for US Government Launch-Related Activities

As part of a broader effort looking at impediments to launch services procurement, the GAO summarized US government-wide budget requests for space launch-related activities (pdf). The purpose of the GAO's broader effort is to provide short- and long-term assessments examining impediments to economical procurement of government launch vehicles and launch services across government.

GAO reviewed FY14 through FY18 space launch-related President's Budget (then year dollars) request by NASA, NOAA, and DOD including military services and other offices.

US Government Launch Procurement Budget Requests

(then-year $M)	FY14	FY15	FY16	FY17	FY18	Total
Procurement	4981.4	5819.7	5816.2	5914.4	5915.5	28447.2
RDT&E	2469.0	2473.0	2414.6	2073.9	1834.0	11264.5
Other	1053.5	774.3	770.7	790.0	762.5	4151.0
Total	8503.9	9067.0	9001.5	8778.3	8512.0	43862.7

Total funding seems pretty flat accross the five years examined. Both DOD and NASA are acquiring launch vehicles by launch services contracts. Procurement is about evenly split between DOD and NASA (slightly more by DOD). The bulk of RDT&E dollars are spent by NASA: $10.5 billion over the five years vs. $718 million.

On the slide detailing the RDT&E spending this report cites another GAO report about the FY10 National Defense Authorization Act requirement for the Department of Defense and the Director of National Intelligence to issue a space science and technology strategy every 2 years. GAO recommends DOD coordinate more with NASA and NOAA to leverage their significant RDT&E investment.

I am interested to see what "impediments" GAO finds as a result of their broader effort.

Nuclear Rockets Petition

As reported by NBC there is a petition on WhiteHouse.gov to rapidly develop and deploy a nuclear thermal rocket for both manned & un-manned space missions. I signed it; why not? Here's the short url: http://wh.gov/UVuD

Nuclear rocketry seems to be a popular topic of late. Gary Johnson has a recent post about nuclear rockets. The NBC article mentions NERVA. There is also the more recent Project Timberwind and Space Nuclear Thermal Propulsion program effort to develop nuclear upper-stages.

Falcon 9 Flight 4, CRS-1

Congratulations to SpaceX on their successful launch of the first commercial resupply mission to the International Space Station!

Interactive Curiosity Rover Panorama

Curiosity rover: Martian solar day 2 in New Mexico Interactive Panorama

Rocket Risk

As reported on ParabolicArc, NASA awarded SpaceX a contract to launch one of their science payloads. The topic of NASA's assessment of launch service provider risk naturally came up. NASA has published payload value, and risk rating guidelines.

If we assume that each launch has the same probability of success, then these are simple risk calculations to make, e.g. see these slides. The posterior probability of success, \(\theta\), is
\[ p(\theta | r, n) = \mathrm{Beta}(\alpha + r, \beta + n - r) \]
where \(r\) is the number of successes, \(n\) is the number of trials, and \(\alpha\) and \(\beta\) are parameters of the Beta distribution prior. What values of parameters should we choose for the prior? I like \(\alpha=\beta=1\), you could probably make a case for anything consistent with \(\alpha+\beta-2=0\). Many people say that risk = probability * consequence. I don't know what the consequences are in this case, and under that approach NASA's chart doesn't make any sense (you could have a low risk with a high probability of failing to launch an inconsequential payload), so I'll stick to just the probabilities of launch success, and leave worrying about the consequences to others.

Since NASA specifies a number of successes in a row (consecutive) then there is already an indication that assuming the trials independent and identically distributed (i.i.d.) is unrealistic. If your expensive rocket blows up, you usually do your best to find out why and fix the cause of failure. That way on the next launch your rocket has a higher probability of success than it previously did.

Second Falcon 9/Dragon Launch

Successful launch of the Falcon 9/Dragon to the International Space Station for COTS demo flight 2.


Lots of coverage on Parabolic Arc and Nuite Blanche.

SpaceX Dragon Panorama

SpaceX has a neat internal panorama up of their Dragon space capsule.

SpaceX Dragon capsule, internal iso/orthogrid panels and grid stiffened structure, forward/port view

J2X Time Series

I thought the intermittent splashing of the cooling water/steam in the close-up portion of this J2-X engine test video looked interesting.

So, I used mplayer to dump frames from the video (30 fps) to jpg files (about 1800 images), and the Python Image Library to crop to a rectangle focused on the splashes.

When a splash occurs the pixels in this region become much whiter, so the whiteness of the region should give an indication of the "splashiness". I then converted the images to black and white, and averaged the pixel values to get a scalar time-series. The whole time series is shown in the plot below.

Also, here's a text file if you want to play with the data.

Here's a PSD and autocorrelation for the section of the data excluding the start-up and shut-down transients.

Here's a recurrence plot of that section of the data.

This is a pretty short data set, but you can see that there are little "bursts" of periodic response in the recurrence plot (compare to some of the recurrence plots for Lorenz63 trajectories). I'm pretty sure this is not significant to engine development in any way, but I thought it was a neat source of time series data.

Acquisition Death Spirals

This isn't about the normal death spiral of increasing unit costs driving production cuts, which increases unit costs, which drives production cuts, which.... It's about another sort of price spiral caused by the US government's infatuation with sole-sourcing critical capabilities. I think this is largely due to technocrats trusting simple, static industrial-age cost models which support decisions dominated by returns from economies of scale. The basic logic of the decisions these models support (an equilibrium solution) is: "things will be cheaper with one supplier because the overhead will be amortized over bigger quantities."

The basic mistake these models make is neglecting the dynamics. Price is a dynamic thing. If the capability is very critical, and there is only one supplier, then there is almost no ceiling on how high the price can rise. The price level reached under those dynamics is just below the point where you'd stop paying for the capability in favor of a more important one (I'll call this the buyer's "level of pain"). The alternative dynamics occurs when there are multiple competing suppliers. The price reached under these dynamics asymptotes towards the economic costs (this takes into account barriers to entry / opportunity costs). Here's a simple graph illustrating price behavior under these two situations.

Price Dynamics

This price increase doesn't happen because the contractors are evil. It happens because the contractors have a duty to their shareholders to maximize profit. This is, in fact, the ethical thing for them to do. If their customer is foolish and decides to create a little monopoly for them with every procurement, well, too bad for the customer's shareholders (taxpayers in this case)...

We see these dynamics play out in a variety of defense acquisitions. The F-35 engine program and the Evolved Expendable Launch Vehicle program are two exemplars that are currently making the news.

The F-35 engine procurement was initially structured to support two suppliers during development, much like the engine programs for F-15 and F-16. There are operational advantages to having two engines. If a problem is found in one model, only half of the Air Force's tactical aircraft would have to be grounded while the solution is found. The other advantage comes from the suppliers competing with each other on price for various lots of engines. A disadvantage is overhead and development cost for the two suppliers and possibly increased logistics footprint for supporting two different engine models.

The recent news is that the budget does not include funds for the second engine. Not a week after this budget is passed which makes the engine buy a sole-source deal, we have an Undersecretary of Defense for Acquisition complaining about the price from the remaining supplier.

"I'm not happy, as I am with so many parts of all our programs, with (the P&W engine's) cost performance so far," Carter told the House of Representatives Appropriations subcommittee on defense on Wednesday. "We need to drive the costs down." [...] "Our analysis does not show the payback," Carter told the subcommittee. He added that "people of good will come to different conclusions on this issue." U.S. "not happy" with F-35 engine cost overruns

Did the analysis include the second supplier offering to assume the risks and go fixed price on the development? Is complaining about the price, being really unhappy about paying it, but paying it anyway because there is no alternative anything but empty political theater? Makes for great content in the trade rags: Acquisition Official gives contractor a stern talking to! Contractor hangs head in a suitably chastened way, "yes, our prices are very high for these unique capabilities, we are working hard to contain the costs for our customer." Much harrumphing is heard from various congresscritters, meanwhile the price continues to spiral higher...

In the case of the EELV, despite the fact that the Air Force paid for two parallel rocket development programs we now have just a single supplier. The two launch service providers were so expensive, they could not compete in the commercial market. The business case for the two rockets hinged on them being able to make money in the commercial market and get their launch rates up. When no other customers but the US government could afford their high prices they had to combine into the single consortium: ULA. So it's sole-source with two vehicles. Recall the various advantages and disadvantages of developing two products discussed above in the case of the F-35 engine. Now EELV has the worst of both worlds: high overhead and logistics costs to support two vehicles, and no competition or customer diversification to get flight rates up and bring prices down.

Launch-service providers agree that their viability, as well as their ability to keep costs down, is based on launch rhythm. The more often a vehicle launches, the more reliable it becomes. Scale economies are introduced as well in a virtuous cycle. One U.S. government official agreed that if SpaceX is now allowed to break ULA’s monopoly on U.S. government satellite launches as indicated by the memorandum of agreement, it could force ULA’s already high prices even higher as it eats into ULA’s current market. “In the longer term we may be faced with questions about whether one of them [ULA or SpaceX] can remain viable without direct subsidies — the same questions we faced with ULA,” this official said. “Then what do we do? We have a policy of assured access to space, which means at least two vehicles. The demand for launches has not increased since ULA was formed, so we could be heading toward a nearly identical situation in a few years. But we are spending taxpayers’ money and if we can find reliable launches that are less expensive, we are not going to ignore that.” SpaceX Receives Boost in Bid To Loft National Security Satellites

It is interesting to note the thinking of the unnamed government official. He doesn't recognize the dynamics of the situation. He's living in a sole-source mindset, it just happens that he's going to change to this new, lower cost source.

NASA has a pricing model that shows savings from "outsourcing development", but not because of any interesting dynamics that they've included. The justification is the same as those underlying the broken decisions about aircraft engines: "returns from economies of scale". The dynamics of competition remains ignored.

NASA Deputy Administrator Lori Garver, in a separate presentation here April 12, said the agency’s policy of pushing rocket-development work onto the private sector will only reach maximum benefit if other customers also purchase the vehicles developed initially with NASA funding. Referring specifically to SpaceX, Garver said a conventional NASA procurement of a Falcon 9-class rocket would cost nearly $4.5 billion according to a NASA-U.S. Air Force cost model that includes the vehicle’s first flight. Outsourcing development to SpaceX, she said, would cut that figure by 60 percent, but only if other customers purchase the vehicle, thus permitting scale economies to reach maximum effect. After Servicing Space Station SpaceXs Priority is Taking on EELV

The way out of the death spiral is program dependent. In the EELV case it took a new entrant who has signed commercial contracts in addition to chasing the government launches (from a couple different agencies). SpaceX has built in significant customer diversification that ULA never developed (though this was hoped for in the early justifications of the program structure). Why did Lockheed-Martin and Boeing, and subsequently ULA never develop this customer diversification? Because they didn't have to. The government guaranteed their continued existence. SpaceX, on the other hand, has no such guarantee. The only option for their continued existence is to make a profit from more than one customer. In the F-35 engine case, DoD has decided that even the assumption of development risk by the second supplier under a fixed price contract is not enough to close the case. This puzzles me. Maybe the second engine has become a symbol of "duplication and waste" rather than "competition and efficiency". If so, then DoD has given the primary engine supplier a way to "frame" their competition out of existence with political argument (removing the need for them to earn market share honestly).

The outlook for lower cost space access looks good. However, as long as the DoD is stuck in its current frame, there is great opportunity for political theater that will serve mainly as a content generator for defense trade publications and a distraction from the root cause of steadily rising tactical aircraft engine costs into the future.

Falcon 9: Lift Off!

Falcon 9 launch successful on first test flight.  Wow!

First Stage In Flight

Successful Stage Separation

Second Stage Burn

Yeah, yeah, it's just a test.  But the cost of the entire development program (~$335M) for Falcon 9 is the same as a single test flight for Ares I-X (~$445M)...

From my perspective (as someone who's sat in the hot seat conducting flight tests), the really impressive thing with the SpaceX operation was their ability  to light the engines, auto-abort, and turn a new countdown/launch at the end of their range time.  There was clearly a whole lot of work in the design phase leading up to the impressive execution today that made saving the mission possible.  Nice.

Closing the Post-Shuttle-Gap Faster

Photo credit: Justin Ray/Spaceflight Now

SpaceX has a hangar full of rocket at the Cape. The latest rumors are that the new NASA budget from the President [Update: not rumors any more] puts the focus for near-term human space-flight (to LEO / ISS) on COTS-like providers. Based on SpaceX's claims (and their demonstrated commitment to develop, build and fly hardware), if this is true, the gap in US capability post-shuttle will actually be less (or eliminated) than if we continued to fund the Constellation boondoggle.

The latest AIAA news roundup on the topic follows.

Coverage continued on the new NASA plans expected to be announced Monday. Media coverage focused on the shift towards commercial users, but stressed that a fight is expected in Congress. The AP (1/31, Borenstein, Chang) reported that with the Obama Administration's expected proposal Monday that NASA buy launch services from private companies, there is "some concern...from former NASA officials worried about safety and from congressional leaders worried about lost jobs." Commercial companies see this very beneficial to the US, and executives "dismiss" safety concerns raised by various panels, citing the airline industry. According to the article, instead of the "established aerospace giants," it is the "newer space guard that brings some excitement to the field."

        The Washington Post (2/1, A8, Achenbach) notes, "NASA's grand plan to return to the moon, built on President George W. Bush's vision of an ambitious new chapter in space exploration, is about to vanish with hardly a whimper." Even though the upcoming budget will "effectively...kill" NASA's previous plans, "it remains to be seen whether Congress will accede to Obama's change in direction. Industry insiders expect a brutal fight in Congress." Meanwhile, White House spokesperson Nick Shapiro said Sunday the budget will show President Obama's "dedication to NASA. NASA is vital not only to spaceflight, but also for critical scientific and technological advancements."

        The Houston Chronicle (1/31, Powell, Berger) also reported that a "titanic political fight" is expected "as lawmakers from Texas move to protect jobs, astronauts and the Johnson Space Center." Congressmen from states with NASA centers "are drawing their battle lines to try to protect aspects of NASA's existing manned space program that account for jobs, payroll and subcontracts in their states." Scott Pace, head of the Space Policy Institute, said people "should not be surprised that the existing moon program has come to an end. ... Bush's budget decisions stressed the program; Obama's cuts drove it off the cliff."

        Meanwhile, Florida Today (1/30, Halvorson) reported Sen. Bill Nelson "says Obama shouldn't pump all of a $6 billion NASA budget increase into the development of commercial crew transportation services. Some of the increase should be put toward building a new heavy-lift launch vehicle." Nelson is concerned NASA will fall further behind in its development otherwise. "Investing in a super-sized rocket would leverage the $9 billion NASA spent on Project Constellation over the last six years" and "could increase the number of test flights at KSC, a move that would further make up for shuttle job losses

        According to BBC News (2/1, Amos), even though organizations like the Commercial Spaceflight Federation endorse the change, "there are major bi-partisan concerns in Congress about the proposed new path, and the impact it could have on jobs tied up with current Nasa programmes." AFP (1/31, Santini) reported than a White House advisor on Friday said, "Constellation is dead." However, "While Constellation is dead, it does not mean human space exploration is also dead," according to John Logsdon, former director of the Space Policy Institute, who points to plans to increase international cooperation.

        Bolden: NASA Will Deal With Non-Traditional Partners. Space News (1/30, subscription required) reported that NASA Administrator Charles Bolden, speaking in Israel, "said U.S. President Barack Obama's vision for NASA will elevate the importance of international cooperation." Bolden said NASA will be "vigorously engaged with Israel and other nontraditional partners as we move forward with the new vision."

        New Heavy Launch Vehicle Could Be Based On DIRECT. NASASpaceFlight.com (1/30, Bergin) reported, "The Michoud Assembly Facility (MAF) have confirmed they have almost enough External Tank resources to allow for one ET-sized 'In Line' Shuttle Derived Heavy Launch Vehicle (SD HLV) test flight and up to three Block I SD HLVs. The news comes as NASA managers insist the workforce should wait for official news, and not to be distracted by reports on Ares' demise." According to the article, a HLV "will be contracted out...moving to a multi-company effort led by Boeing," with partners like Lockheed Martin, Northrop Grumman, and United Space Alliance. A plan is progressing "at a healthy pace" towards a test flight "of what is now heavily confirmed as based on the DIRECT team's Jupiter-241 Stretched Heavy Launch Vehicle." The article noted that this plan, contrasted with the "sluggish" Ares V rocket, is reportedly "receiving significant support" from Congress members.

Interesting Developments in Commercial Space

These interesting articles were collected in my daily AIAA news brief.

White House Reportedly Will Fund Commercial Spaceflight.

The Wall Street Journal (1/25, Pasztor, subscription required) website reports that the White House has chosen to fund private companies to take astronauts into space with a program likely to cost $3.5 billion over five years. According to the article, Congress is expected to challenge this because of safety concerns. Even though the article predicts conflicts because it will shift money from existing programs to commercial companies, it also contends there will not be any major program cancellations.

        Meanwhile, in continuing coverage, Space News (1/23, Klamper, subscription required) reported NASA, according to unnamed sources "will not be getting the $1 billion budget boost civil space advocates had hoped to see," but the fate of the Ares I is still unknown. However, the Orion capsule would not be cancelled, according to the sources. "While Obama's funding proposal deviates from the Augustine panel's push for a spending increase, sources said NASA's 2011 budget request is expected to align with the panel's so-called Flexible Path plan." According to the article, "In hindsight, NASA officials say the agency set Ares 1 and Orion on an unsustainable spending trajectory."

        Space Frontier Foundation Chairman Sees Killing Ares As "Clear" Choice. In an op-ed for the Orlando Sentinel (1/23), Bob Werb, chairman of the board of the Space Frontier Foundation, called for the cancellation of the "boondoggle" Ares I rocket program. It is "just pork dressed up as cost-effective human space transportation; it's not just wasteful, but destructive to future space exploration beyond Earth orbit." Werb believes there is an "overwhelming" case against the rocket, which he saw as a "bailout" for shuttle companies. "The commercial alternatives are based on well-tested, mature systems currently used to launch U.S. military, scientific and commercial satellites. Adapting these rockets to carry people is cheaper, faster and better." Werb saw this as a simple and "clear" decision.

Food fight's on!

I'm a little unclear on how you can call spaceflight whose only customer is the government "commercial" though...

In related news,

Bolden To Reveal NASA Budget In Press Conference Next Week.

Space News (1/26, Klamper, subscription required) reports, "NASA Administrator Charles Bolden will unveil the U.S. space agency's spending priorities for 2011 during a Feb. 1 press conference at NASA headquarters here, according to administration officials." The article notes Bolden is also "expected to discuss long-awaited details of the president's funding proposal in the morning, followed by a press conference hosted by the White House Office of Science and Technology Policy (OSTP) to rollout Obama's research and development priorities - including those that affect NASA goals and funding- for the coming budget year, these sources said." The budget is "expected to realign NASA's human spaceflight activities and investments to foster development of commercial systems." Bolden "tentatively" scheduled to conduct another press conference at the National Press Club the following day.

        According to Spaceflight Now (1/25, Clark), the budget "is expected to include new direction for NASA." A White House spokesperson told Spaceflight Now, "NASA is vital not only to spaceflight, but also for critical scientific and technological advancements. The expertise at NASA is essential to developing innovative new opportunities, industries and jobs. The President's budget will take steps in that direction." However, the "fate of the agency's vexed exploration program is still unclear." The upcoming budget "may provide no direct guidance on the Constellation program."

        Former NASA Official Cautions Over Commercial Human Spaceflight. In an op-ed for Space News (1/25, subscription required), Scott Horowitz, former associate administrator for the exploration systems directorate, wrote about the role he saw for commercial space companies in manned exploration. "To date, commercial participants have made limited progress and experienced many challenges. Contracts have been pulled, failures have occurred and schedules have slipped up to two years. This has been a stark reminder that rocket development programs are challenging." Horowitz believes that only after a commercial company delivers cargo to the International Space Station "will be time to carefully consider commercial human spaceflight." Horowitz, listing commercial ventures that have failed, felt that this is "not a time to discard these proven systems" of Orion and Ares "in favor of immature, high-risk commercial options."

Ares and Orion are proven? When did those flights happen?

The Stick Does Not Pogo

With the upcoming test flight of the Ares IX, there is lots of talk about high vibration levels and design problems with the big solid first stage. One common misconception I've noticed (and pointed out) a couple times is that folks think Ares I might have pogo oscillations. Pogo oscillations are a specific type of propulsion-structure interaction that occurs in liquid rockets, not solids.

Here's a succinct description of what pogo is (Rubin, 1970, emphasis mine):

Many liquid rocket vehicles have experienced longitudinal vibration because of an instability arising from interaction of the vehicle structure with the propulsion system. The vibrations, nicknamed ``pogo'' after the jumping stick, have occurred principally in the first longitudinal structural mode during operation of the first liquid-propellant stage of a launch vehicle.

From the same report, here's a figure of the accelerations characteristic of pogo.


The dynamic instability being described here is not simply resonant forcing. There is a positive feedback loop.

A block diagram of the positive feedback process which can lead to instability is shown in figure 2. Structural vibratory accelerations induce the propulsion system to generate forces which can then intensify the original vibration. When the system becomes unstable, oscillations will appear spontaneously.


--Rubin (1970), emphasis mine

Rubin gives a further description of the two types of pogo.

Two basic kinds of propulsion-system behavior have produced pogo instability. The common form of pogo, called engine-coupled pogo, has been experienced to a significant degree on certain configurations of the Thor, Titan, and Saturn space vehicles. This form results from the action of the tank-to-engine propellant feed-lines and the engine itself (fig 3.) When the vehicle vibrates longitudinally, the pump and the propellant in the flexible tank undergo oscillatory motions. These two motions produce oscillating flow in the feed-line and in the pump's discharge line. The flow oscillations lead to oscillations in engine thrust and in pressure at the pump inlet, which then act as regenerative forcing functions on the vehicle structure. The feedback loop is thereby closed. Although a pump is included in figure 3, pogo can also occur in pressure-fed systems.


A much less common form of pogo results from the pneumatic behavior of an active pressurization system for the propellant tank ullage. This form is known as ullage-coupled pogo, and has been experienced only on Atlas vehicles immediately after liftoff. It has also been referred to as pneumatic-coupled pogo, or ``bloating''. A simplified schematic of an active pneumatic system for pressurizing a tank is shown in figure 4. When the vehicle vibrates longitudinally, the tank ullage pressure oscillates because of oscillation of the ullage-volume boundaries. The sense line transmits the pressure oscillation to the regulator. The regulator responds by producing an oscillatory flow of pressurant (i.e., the pressurizing gas) into the supply line which regenerates the ullage-pressure oscillation that acts as an axial forcing function on the vehicle structure, and again the feedback loop is closed.

The big SRB being contemplated for use in Ares I probably does have lots of combustion instabilities which will cause some wicked forcing on the rocket. It may even couple well to the rocket's lower longitudinal modes, but that is not pogo. There is no positive feedback there, just lots of hammering on the structure (and the astronauts).

von Braun: dirty hands

The technical people [must] keep their hands dirty and actively work on in-house projects selected specifically for the purpose of updating their knowledge and increasing their competence.
-- Werner von Braun, Management

Treating Wounds from PowerPoint Bullets

Or "how to retain your critical thinking skills while being PowerPointed to death".

I recently wrote about PowerPoint Engineering from the perspective of improving technical communications towards decision makers. What about when you are on the receiving end of a volley of PP bullets?

Tufte has a nice critical write-up with "methods for how not to get fooled" while enduring a presentation. The gist (in "bullet" form of course):

• Be skeptical of analytical claims of "conservatism", these are often rhetorical tricks which hide a lack of quantitative rigor: "we're so uncertain we can't put a number on it, so we were conservative"
  
• Close reading pays, beware of contradictory sub-sub-bullets tucked in under vague "big" bullets or headings
  
• Be skeptical of claims of "significance" without supporting quantitative statistical analysis, another rhetorical slight of hand
  
• Be aware of the scope of an analysis, true statements about a particular case are easily misunderstood to apply generally
  
• Disambiguate pronouns, often the importance of the subject (life or death) is hidden by innocuous looking pronouns
  
• Be aware of your organization's biases as well as your own (echos of Aristotle's Nicomachean Ethics)
  

Of course, in the end it boils down to being an engaged, critical thinker. PowerPoint has been shown to be ill-suited for technical communication, so the information you need is probably not on the slides! To quote from the Tao Te Ching:

We put thirty spokes together and call it a wheel;
But it is on the space where there is nothing that the usefulness of the wheel depends.
We turn clay to make a vessel;
But it is on the space where there is nothing that the usefulness of the vessel depends.
We pierce doors and windows to make a house;
And it is on these spaces where there is nothing that the usefulness of the house depends.
Therefore just as we take advantage of what is, we should recognize the usefulness of what is not.

Understanding their omissions is as important as understanding their presentations. In modern, complex systems we often suffer from information and data overload, so there's likely to be lots omitted. Hopefully not the critical few pieces of information, but failure analysis after failure analysis shows it often is.

As analysis becomes more causal, multivariate, comparative, evidence-based, and resolution-intense, the more damaging the bullet list becomes. -- E. Tufte, on bullets

A better list of how to handle the PP barrage than my paraphrase above (emphasis mine):

One's inability to produce the devastating comeback live during the course of discussion-- l'esprit de l'escalier--is why it is helpful, in deciding serious matters (1) to have the material under discussion distributed in advance, (2) to rehearse the possible exchange in advance, (3) to take a time-out, leave the meeting, escape the groupthink and bullying, go for a long walk down the hall (or up the stairs in the French version of after-the-fact-wit), and ask yourself "What would Richard Feynman do?"-- E. Tufte, on analytical reasoning

Beware technocrats bearing bullet points.

NASA keeps coming up in these sorts of discussions, not because they are particularly bad, but because they are a large, technical organization that is public. They don't hide behind the veil of secrecy (proprietary or classified). A recent discussion on Slashdot highlights some interesting stuff. One being that someone who by all rights should be technically competent can behave like a foolish bureaucrat when addled by the wrong sort of system.

Ares/Orion Slip and Classic Mistake

The latest news about the new NASA Constellation program indicates that they are having cost and schedule problems, this is no surprise, everyone has cost and schedule problems. The surprising thing is that they are deleting tests from the planned program development.

CxP attempted to protect the schedule and budgetary pressures by offsetting these additional strains by deleting test items - notably on the Upper Stage. However, this only proved to cause further disconnects throughout the program.

I suppose it should not be too surprising, this is a classic program management mistake. When the schedule gets tight and the costs get too high above estimates, cut testing. Why? Because it is at the end of the program, and program managers have a bit of hubris that leads them to think that the design is sound because they worked so hard on it up-front. Which is obviously the case because they had to spend more time than they thought in that stage of the program, right? Maybe, but also maybe not, that's why the tests are critically important: to validate the design performance. No slide-show, CAD drawing, CFD or FEA simulation can provide that sort of knowledge.

I can't understand why someone would think that under-testing a new product that is probably delayed and expensive because of its new technology and complexity would produce a good result. Rather than removing unneeded technology or complexity, rather than simplifying or reducing the requirements, they are rushing to field a fragile system. It is unfortunate that the burden of this foolishness is borne by the astronaut corps and not the program managers responsible for these short-sited expediencies. The program manager will probably get a reward, move to a new job, and then astronauts will be at undue risk in an already risky business.