Various Consequences

McCain's Hangar Queen on Trend

2012-01-19T18:47:00.001-05:00

Looks like McCain's Hangar Queen is on Norm's trend.

Entire Defense budget to buy one airplane

See the video; linked from Make. Related post on Roger Pielke's site.

Nuclear Politics Prior

2012-01-03T05:30:00.018-05:00

One of the recurring themes of Climate Resistance (which I've mentioned approvingly before) is that the politics around the solutions proposed for climate change are prior to any consideration of the science of the environment (or the more interesting question Lorenz asked about the existence and uniqueness of long-time averages of the earth's weather).

This state of politics prior is unavoidable, and therefore not unique to the field of climate policy. I think this talk by Kirk Sorensen on the history of US breeder reactor development provides another good example.

The policy maker (Nixon in this case) picked the solution based on desired political outcomes (jobs and lucre for politically useful districts). "The Science" was merely a part of the marketing and public relations campaign for that choice. Dissenting scientific positions (the molten salt folks in this case) were suppressed with "Rickoverian dedication".

See this extensive remix for lots of info on LFTR.
"There's been a very bipartisan approach to scaring the public." Kirk Sorensen (~1:20 or so)
"The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary." H.L. Mencken

Setting up a project on Github

2012-01-02T12:48:00.000-05:00

I've been an SVN user for a while, but it seems like more and more projects are going distributed version control systems like Git so I wanted to learn how to use Git. I found this crash course on Git for SVN users which provides a useful Rosetta stone, and this warning:

SVN is based on an older version control system called CVS, and its designers followed a simple rule: when in doubt, do like CVS. Git also takes a form of inspiration from CVS, and its designer also followed a simple rule: when in doubt, do exactly the opposite of CVS. This approach lead to many technical innovations, but also lead to a lot of extra headscratching among migrators. You have been warned.

This sounds a lot link Linus Torvalds' talk, WWCVSND: What Would CVS Not Do?

Github has a set of steps for setting up on linux. Git comes in the Fedora repos (and probably every other repo), so install is easy. A nice bit of documentation that comes with the install is Everyday GIT With 20 Commands Or So.

Since I already use password-less ssh to hop between the boxes in my little network, I didn't move my old public key as in the instructions. I created a config file in the .ssh directory containing these lines:
Host github.com User git Port 22 Hostname github.com IdentityFile ~/.ssh/id_rsa_git TCPKeepAlive yes IdentitiesOnly yesWhere id_rsa_git.pub is the key I uploaded to github. Authenticating to github is then just:
ssh -T git@github.com
Then you accept their RSA key like you would for doing any other ssh login.

The next thing is to create a repo. Clicking through the instructions brings you to a page with several "next steps". Which for my example are:
mkdir FalknerSkan cd FalknerSkan git init touch README git add README git commit -m 'first commit' git remote add origin git@github.com:jstults/FalknerSkan.git git push -u origin masterWhich gives some output ending in something like: Branch master set up to track remote branch master from origin.

If you'd like, you can read up on the Falkner-Skan ODE at the viscous aero course on MIT's OCW:

Now you have N problems

2012-01-01T16:02:00.006-05:00

(10:40) It's not that regulators don't understand information technology, because it should be possible to be a non-expert and still make a good law. MPs and Congressmen and so on are elected to represent districts and people, not disciplines and issues.

That couple of sentences reminded me of a recent post on single-issue advocacy by Roger Pielke Jr. So, in that spirit, here's a fun word game: How applicable is Doctorow's criticism if you substitute "climate" for "copyright" below?

(22:20) But the reality is, copyright legislation gets as far as it does precisely because it's not taken seriously, which is why on one hand, Canada has had Parliament after Parliament introduce one stupid copyright bill after another, but on the other hand, Parliament after Parliament has failed to actually vote on the bill.[...]It's why the World Intellectual Property Organization is gulled time and again into enacting crazed, pig-ignorant copyright proposals because when the nations of the world send their U.N. missions to Geneva, they send water experts, not copyright experts; they send health experts, not copyright experts; they send agriculture experts, not copyright experts, because copyright is just not important to pretty much everyone!
Canada's Parliament didn't vote on its copyright bills because, of all the things that Canada needs to do, fixing copyright ranks well below health emergencies on first nations reservations, exploiting the oil patch in Alberta, interceding in sectarian resentments among French- and English-speakers, solving resources crises in the nation's fisheries, and thousand other issues! The triviality of copyright tells you that when other sectors of the economy start to evince concerns about the internet and the PC, that copyright will be revealed for a minor skirmish, and not a war. Why would other sectors nurse grudges against computers? Well, because the world we live in today is /made/ of computers. We don't have cars anymore, we have computers we ride in; we don't have airplanes anymore, we have flying Solaris boxes with a big bucketful of SCADA controllers [laughter]; a 3D printer is not a device, it's a peripheral, and it only works connected to a computer; a radio is no longer a crystal, it's a general-purpose computer with a fast ADC and a fast DAC and some software.

Full Transcript

Fedora 16 Install Notes

2011-12-18T17:16:00.000-05:00

Notes on using the pre-upgrade method to go from Fedora 14 to 16 (yes, it's risky to skip a version). The reason for my upgrade is that Fedora 14 is now at end-of-life, I'd like to get FEniCS working, and some of the new spins look pretty cool.

One of the things I had to manually fix after reboot was the TexLive development repo (only 60% complete, estimated for Fedora 17). I had this activated to get the IJ4UQ styles to work.
yum remove texlive-release
rpm -i http://jnovy.fedorapeople.org/texlive/packages.f16/texlive-release.noarch.rpm
Manually remove some old conflicting packages (some from Fedora 12, yikes! I'm a lazy sys-admin).
List the repositories to make sure everything is pointing at the correct release version.
yum repolist
Sync things up.
yum distro-sync
I got this error: grubby fatal error: unable to find a suitable template. According to the forum I did,
mv /boot/grub/grub.conf /boot/grub/grub.conf.old
and then,
yum reinstall kernel
yum clean all && yum install texlive

The other problem I ran in to was with the hwloc package. I had to downgrade it to the 1.2.1 version, the 1.3 version seems to be missing the shared library needed by mpich and scotch.

McCain's Hangar Queen

2011-12-16T22:02:00.003-05:00

Senator McCain's colorful statements about the F-22 are getting lots of coverage.

Another example of how flawed the Pentagon’s weapons procurement process is can be found in the F-22 RAPTOR program. When the Pentagon and the defense industry originally conceived of the F-22 in the mid-1980s, they intended it to serve as a revolutionary solution to the Air Force’s need to maintain air superiority in the face of the Soviet threat during the Cold War. The F-22 obtained ‘full operational capability’ twenty years later -- well after the Soviet Union dissolved. When it finally emerged from its extended testing and development phase, the F-22 was recognized as a very capable tactical fighter, probably the best in the world for some time to come. But, plagued with developmental and technical issues that caused the cost of buying to go through the roof, not only was the F-22 twenty years in the making, but the process has proved so costly that the Pentagon could ultimately afford only 187 of the planes -- rather than the 750 it originally planned to buy.

“Unfortunately, the F-22 also ended up being effectively too expensive to operate compared to the legacy aircraft it was designed to replace. It also ended up largely irrelevant to the most predominant current threats to national security -- terrorists, insurgencies, and other non-state actors. In fact, if one were to set aside the F-22’s occasional appearances in recent big-budget Hollywood movies where it has been featured fighting aliens and giant robots, the F-22 has to this day not flown a single combat sortie -- despite that we have been at war for 10 years as of this September and recently supported a no-fly zone in Libya.
“Politically engineered to draw in over 1,000 suppliers from 44 states represented by key Members of Congress and, by the estimates of prime contractor Lockheed Martin, directly or indirectly supporting 95,000 jobs, there can be little doubt that the program kept being extended far longer than it should have been -- ultimately to the detriment to the taxpayer and the warfighter. As such, it remains an excellent example of how much our defense procurement process has been in need in reform. We may fight a near-peer military competitor with a fifth-generation fighter capability someday, but we have been at war for 10 years and until a few months ago had been helping NATO with a no-fly zone in Libya. And, this enormously expensive aircraft sat out both campaigns.
“Moreover, as a result of problems with its OBOG (On-Board Oxygen Generating) system, which has caused pilots to get dizzy or, in some cases, lose consciousness from lack of oxygen, on May 3, 2011, the Air Force grounded its entire fleet of F-22s. While this grounding was lifted earlier this year, exactly why F-22 pilots have been experiencing hypoxia remains unknown -- but similar unexplained incidents continue.

“And then, there is the issue of the sky-rocketing maintenance costs to the Air Force in trying to sustain a barely adequate ‘mission capable rate’ for the F-22. Its seems that the ‘plug and play’ component maintenance features that were supposed to reduce costs for the Air Force over the life cycle of the aircraft doesn’t really play well. And, each time a panel is opened for maintenance, the costs to repair the ‘low-observable’ surface in order to maintain its stealthiness have made this critical feature of the aircraft cost-prohibitive to sustain over the long-run. Finally, it seems that the engineers and technicians designing the F-22 forgot a basic law of physics during some point of the development phase -- that dissimilar metals in contact with each other have a tendency to corrode. The Air Force is now faced with a huge maintenance headache costing over hundreds of millions of dollars-and-growing to keep all 168 F-22s sitting on the ramp from corroding from the inside out.

One thing is clear: because of a problem directly attributable to how aggressively the F-22 was acquired -- procuring significant quantities of aircraft without having conducted careful developmental testing and reliably estimating how much they will cost to own and operate -- the 168 F-22s, costing over $200 million each, may very well become the most expensive corroding hanger queens ever in the history of modern military aviation.
REMARKS BY SENATOR JOHN McCAIN ON THE “MILITARY-INDUSTRIAL-CONGRESSIONAL” COMPLEX

What bothers me most about the F-22 is not the maintenance problems, operational difficulties, or danger to air-crews. These require mere engineering fixes. What really bothers me is the damage these huge programs cause to rational risk assessment and national security.

"The reality is we are fighting two wars, in Iraq and Afghanistan, and the F-22 has not performed a single mission in either theater," Gates told a Senate committee last week.
Carlson ["We think that [187 planes] is the wrong number"], however, told a group of reporters earlier in the week that the Air Force was "committed to funding 380" of the fighters, regardless of the Bush administration's decision.
According to an Air Force official briefed on the Thursday rebuke, Gates telephoned Air Force Secretary Michael W. Wynne, who was on vacation at the time, to express his displeasure with Carlson.
The senior defense official said Carlson's remarks, reported Thursday by the trade publication Aerospace Daily, angered the Pentagon's top leadership, adding that they were "completely unacceptable and out of line."
Fighter Dispute Hits Stratosphere

That's not something that more money or more technology can fix.

OpenFOAM and FEniCS Fedora Install Notes

2011-11-24T08:35:00.002-05:00

Notes on installing OpenFOAM and FEniCS from source on Fedora 14.

Both projects implement ideas similar to A Livermore Physics Applications Language (NALPAL, why ALPAL): take high-level descriptions of partial differential equations and automatically generate code to solve them with numerical approximations based on finite-volume (OpenFOAM) or finite-element (FEniCS) methods.

Useful Links

FEniCS install from source
OpenFoam source pack
open foam uses paraview, paraview.org
open foam uses wmake, which is a script that comes with the installation
FEniCS uses cmake

Steps for OpenFOAM

Install Fedora packages for paraview, cmake, flex, qt, zlib.
[root@deeds foo]$ yum -y install paraview cmake flex qt-devel libXt-devel zlib-devel zlib-static scotch scotch-devel openmpi openmpi-devel
You probably already have gnuplot and readline installed, if not, install those too.
Unpack the OpenFOAM tarball:
[jstults@deeds OpenFOAM]$ tar -zxvf OpenFOAM-2.0.1.gtgz
Add the environment variable definitions to the bashrc file
source $HOME/OpenFOAM/OpenFOAM-2.0.1/etc/bashrc
[jstults@deeds OpenFOAM]$ source ~/.bashrc
Run the OpenFOAM-X.y.z/bin/foamSystemCheck script, you should get something that says, "System check: PASS", "Continue OpenFOAM installation."
Go to the top-level directory, since you defined the proper environment variables that is something like
[jstults@deeds OpenFOAM]$ cd $WM_PROJECT_DIR
Make it
[jstults@deeds OpenFOAM-X.y.z]$ ./Allwmake
Various Consequences ensue...

Steps for FEniCS

Required components: Python packages FFC, FIAT, Instant, Viper and UFL, and Python/C++ packages Dolfin and UFC

Install things packaged for Fedora
[root@deeds FEniCS]$ yum -y install python-ferari python-instant python-fiat
Download the other components from launchpad, and install using
[jstults@deeds FEniCS]$ python setup.py install --prefix=~/FEniCS
for the Python packages, and
[jstults@deeds FEniCS]$ cmake -DCMAKE_INSTALL_PREFIX=$HOME/FEniCS ./src
[jstults@deeds FEniCS]$ make
[jstults@deeds FEniCS]$ make install
for the C++/Python packages (DOLFIN and UFC).

OR, do it automatically if you have root and internet access (I still haven't got this to work: a bit of buffoonery on my part, problems finding boost libraries).

Download Dorsal
Run the script
[jstults@deeds FEniCS]$ ./dorsal.sh
Execute the yum command in the output from the script in another terminal.
[root@deeds FEniCS]$ yum install -y redhat-lsb bzr bzrtools subversion \ libxml2-devel gcc gcc-c++ openmpi-devel openmpi numpy swig wget \ boost-devel vtk-python atlas-devel suitesparse-devel blas-devel \ lapack-devel cln-devel ginac-devel python-devel cmake \ ScientificPython mpfr-devel armadillo-devel gmp-devel CGAL-devel \ cppunit-devel flex bison valgrind-devel
I also installed the boost-openmpi-devel and boost-mpich2-devel packages as well. MPI on Fedora is kind of a mess. I added these two lines to my bashrc:
module unload mpich2-i386
module load openmpi-i386
When the packages are installed, go back to the terminal running dorsal.sh and hit ENTER.

I ended up downloading the development version of dorsal to get the latest third-party software built, because it's so easy:
[jstults@deeds dorsal]$ bzr branch lp:dorsal
Pretty darn slick, Yea for distributed version control systems! Boo for dependency hell!

FEniCS is currently under heavy development towards a 1.0 release, so I expect to be able to build it on Fedora 14 Real Soon Now ; - )

Update: Johannes and friends got me straightened out.

Qu8k Accelerometer Data

2011-11-17T12:03:00.001-05:00

There was a pretty cool amateur rocket shot recently that was an attempt to win the Carmack micro-prize. The rocket is called Qu8k, designed and built by Derek Deville and friends. One of the stipulations of the prize is collection of a GPS location by the onboard avionics at an altitude above 100kft (as long as the velocity is low, this should theoretically not require an unrestricted GPS).

Of course, since the other stipulation of the prize is a detailed report about the shot and the data collected, this gives us number crunching nerds a neat data set to play with. Derek posted a spreadsheet of the accelerometer data, along with a simple first order integration (twice) to get velocity and altitude. I tried an FFT-based method to compare against the first order approach in the spread-sheet, and a second-order trapezoidal rule integration. The python script to do the integration and make the two plots below is on github.

The errors in the numerical integration are not terrible (the plots of the trajectories using the different approaches are indistinguishable in the eye-ball norm).

One of the cool things about Python is the array masking capability. That makes implementing the temperature ratio component of the 1976 US Standard atmosphere (to estimate Mach number) 7 lines of code.

Notre Dame V&V Workshop

2011-10-11T19:36:00.002-04:00

The folks at Notre Dame are putting on a Verification and Validation Workshop. The preliminary agenda is up on the workshop site.

The purpose of the workshop is to bring together a diverse group of computational scientists working in fields in which reliability of predictive computational models is important. Via formal presentations, structured discussions, and informal conversations, we seek to heighten awareness of the importance of reliable computations, which are becoming ever more critical in our world.

The intended audience is computational scientists and decision makers in fields as diverse as earth/atmospheric sciences, computational biology, engineering science, applied mechanics, applied mathematics, astrophysics, and computational chemistry.

It looks very interesting. ~~Unfortunately, I don't think I'll be able to attend. Hopefully they will post posters and papers.~~ Update: I was able to attend. They will be posting the presenters' slides. I will put up some of my notes when they get the slides up.

J2X Time Series

2011-10-03T06:00:00.000-04:00

I thought the intermittent splashing of the cooling water/steam in the close-up portion of this J2-X engine test video looked interesting.

So, I used mplayer to dump frames from the video (30 fps) to jpg files (about 1800 images), and the Python Image Library to crop to a rectangle focused on the splashes.

When a splash occurs the pixels in this region become much whiter, so the whiteness of the region should give an indication of the "splashiness". I then converted the images to black and white, and averaged the pixel values to get a scalar time-series. The whole time series is shown in the plot below.

Also, here's a text file if you want to play with the data.

Here's a PSD and autocorrelation for the section of the data excluding the start-up and shut-down transients.

Here's a recurrence plot of that section of the data.

This is a pretty short data set, but you can see that there are little "bursts" of periodic response in the recurrence plot (compare to some of the recurrence plots for Lorenz63 trajectories). I'm pretty sure this is not significant to engine development in any way, but I thought it was a neat source of time series data.

Discussion of V and V

2011-10-01T15:07:00.000-04:00

A few days after I put up my little post on a bit of V&V history Judith Curry had a post about VV&UQ which generated a lot of discussion (she has a high-traffic site, so this is not unusual) [1].

A significant part of the discussion was about a post by Steve Easterbrook on the (in)appropriateness of IV&V (“I” stands for independent) or commercial / industrial flavored V&V for climate models. As George Crews points out in discussion on Climate Etc., there’s a subtle rhetorical slight of hand that Easterbrook uses (which works for winning web-points with the uncritical because different communities use the V-words differently, see the introduction to this post of mine). He claims it would be inappropriate to uncritically apply the IV&V processes and formal methods he’s familiar with from developing flight control software for NASA to climate model development. Of course, he’s probably right (though we should always be on the look-out to steal good tricks from wherever we can find them). This basically correct argument gets turned in to “Easterbrook says V&V is inappropriate for climate models” (by all sorts of folks with various motivations, see the discussion thread on Climate Etc.). The obvious question for anyone who’s familiar with even my little Separation of V&V post is, “which definition of V or V?”

What Easterbrook is now on record as agreeing is appropriate is the sort of V&V done by the computational physics community. This is good. Easterbrook seemed to be arguing for a definition of “valid” that meant “implements the theory faithfully” [2]. This is what I’d call “verification” (are you solving the governing equations correctly). The problem with the argument built on that definition, is the conflation I pointed out at the end of this comment, which is kind of similar to the rhetorical leap mentioned in the previous paragraph and displayed on the thread at Climate Etc.

Now, his disagreement with Dan Hughes (who recommends an approach I find makes a great deal of sense, and that I've used in anger to commit arithmurgical damage on various and sundry PDEs) is that Dan thinks we should have independent V&V, and Steve thinks not. If all you care about is posing complex hypotheses then IV&V seems a waste. If you care about decision support, then it would probably be wise to invest in it (and in our networked-age much of the effort could probably be crowd-sourced, so the investment can conceivably be quite small). This actually has some parallels with the decision support dynamics I highlighted in No Fluid Dynamicist Kings in Flight Test.

One of the other themes in the discussion is voiced by Nick Stokes. He argues that all this V&V stuff doesn’t result in successful software, or that people calling for developing IV&V’d code should shut up and do it themselves. One of the funny things is that if the code is general enough that the user can specify arbitrary boundary and interior forcings, then any user can apply the method of manufactured solutions (MMS) to verify the code. What makes Nick’s comments look even more silly is that nearly all available commercial CFD codes are capable of being verified in this way. This is thanks in part to the efforts of folks like Patrick Roache for instance [3], and now it is a significant marketing point as Dan Hughes and Steven Mosher point out. Nick goes on to say that he’d be glad to hear of someone trying all this crazy V&V stuff. The fellows doing ice sheet modeling that I pointed out on the thread on Easterbrook’s are doing exactly that. This is because the activities referred to by the term “verification” are a useful set of tools for the scientific simulation developer in addition to being a credibility building exercise (as I mentioned in the previous post, see the report by Salari and Knupp [4]).

Of course doing calculation verification is the responsibility of the person doing the analysis (in sane fields of science and engineering anyway). So the “shut-up and do it yourself” response only serves to undermine the credibility of people presenting results to decision makers bereft of such analysis. On the other hand, the analyst properly has little to say on the validation question. That part of the process is owned by the decision maker.

References

[1] Curry, J., Verification, Validation and Uncertainty Quantification in Scientific Computing, http://judithcurry.com/2011/09/25/verification-validation-and-uncertainty-quantification-in-scientific-computing/,Climate Etc. Sunday 25^th September, 2011.

[2] Easterbrook, S., Validating Climate Models, http://www.easterbrook.ca/steve/?p=2032, Serendipity, Tuesday 30^th November, 2010.

[3] Roache, P., Building PDE Codes to be Verifiable and Validatable, Computing in Science and Engineering, IEEE, Sept-Oct, 2004.

[4] Knupp, P. and Salari, K., Code Verification by the Method of Manufactured Solutions, Tech. Rep. SAND2000-1444, Sandia National Labs, June 2000.

The Separation of V and V

2011-09-15T03:00:00.000-04:00

The material in this post started out as background for a magazine article, and turned in to the introduction to an appendix for my dissertation [1]. Since I started learning about verification and validation, I’ve always been curious about the reasons for the sharp lines we draw between the two activities. Turns out, you can trace the historical precedent for separating the task of verification and validation all the way back to Lewis Fry Richardson, and George Boole spent some space writing on what we’d call verification. Though neither he nor Richardson use the modern terms, the influence of their thought is evident in the distinct technical meanings we’ve chosen to give the two common-usage synonyms “verification” and “validation.”

The term verification is given slightly different definitions by different groups of practitioners [2]. In software engineering, the IEEE defines verification as

The process of evaluating the products of a software development phase to provide assurance that they meet the requirements defined for them by the previous phase.

while the definition now commonly accepted in the computational physics community is

The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model. [3]

The numerical weather prediction community speaks of forecast verification, which someone using the above quoted AIAA definition would probably consider a form of validation, and a statistician might use the term validation in a way the computational physicist would probably consider verification [4]. Arguing over the definitions of words [5] which, in common use, are synonyms is contrary to progress under a pragmatic, “wrong, but useful” conception of the modeling endeavor [6]. Rather, we should be clear on our meaning in a specific context, and thus avoid talking past collegues in related disciplines. Throughout this work, the term verification is used consistent with currently accepted definitions in the aerospace, defense and computational physics communities [3, 7, 8, 9].

In the present diagnostic effort, the forward model code seeks an approximate solution to a discretized partial differential equation. This partial differential equation (PDE) is derived from Maxwell’s equations augmented by conservation equations derived from the general hyperbolic conservation laws through analytical simplification guided by problem-specific assumptions. The purpose of formal verification procedures such as method of manufactured solutions (MMS) are to demonstrate that the simulation code solves these chosen equations correctly. This is done by showing ordered convergence of the simulated solution in a discretization parameter (such as mesh size or time-step size).

Understanding the impact of truncation error on the quality of numerical solutions has been a significant concern over the entire developmental history of such methods. Although modern codes tend to use finite element, finite volume or pseudo-spectral methods as opposed to finite differences, George Boole’s concern for establishing the credibility of numerical results is generally applicable to all these methods. In his treatise, written in 1860, Boole stated

...we shall very often need to use the method of Finite Differences for the purpose of shortening numerical calculation, and here the mere knowledge that the series obtained are convergent will not suffice; we must also know the degree of approximation.
To render our results trustworthy and useful we must find the limits of the error produced by taking a given number of terms of the expansion instead of calculating the exact value of the function that gave rise thereto. [10]

In a related vein, Lewis Fry Richardson, under the heading Standards of Neglect in the 1927 paper which introduced the extrapolation method which now bears his name, stated in his characteristically colorful language

An error of form which wold be negligible in a haystack would be disastrous in a lens. Thus negligibility involves both mathematics and purpose. In this paper we discuss mathematics, leaving the purposes to be discussed when they are known. [11]

This appendix follows Richardson’s advice and confines discussion to the correctness of mathematics, leaving the purposes and sufficiency of the proposed methods for comparison with other diagnostic techniques in the context of intended applications.

While the development of methods for establishing the correctness and fitness of numerical approximations is certainly of historical interest, Roache describes why this effort in code verification is more urgently important than ever before (and will only increase in importance as simulation capabilities, and our reliance on them, grow).

In an age of spreading pseudoscience and anti-rationalism, it behooves those of us who believe in the good of science and engineering to be above reproach whenever possible. Public confidence is further eroded with every error we make. Although many of society’s problems can be solved with a simple change of values, major issues such as radioactive waste disposal and environmental modeling require technological solutions that necessarily involve computational physics. As Robert Laughlin noted in this magazine, “there is a serious danger of this power [of simulations] being misused, either by accident or through deliberate deception.” Our intellectual and moral traditions will be served well by conscientious attention to verification of codes, verification of calculations, and validation, including the attention given to building new codes or modifying existing codes with specific features that enable these activities. [6]

There is a moral imperative underlying correctness checking simply because we want to tell the truth, but this imperative moves closer "to first sight" because of the uses to which our results will be put.

The language Roache uses reflects a consensus in the computational physics community that has given the name verification to the activity of demonstrating the impact of truncation error (usually involving grid convergence studies), and the name validation to the activity of determining if a code has sufficient predictive capability for its intended use [3]. Boole’s idea of “trustworthy results” clearly underlies the efforts of various journals and professional societies [3, 7, 9, 8] to promote rigorous verification of computational results. Richardson’s separation of the questions of correct math and fitness for purpose are reflected in those policies as well. In addition, the extrapolation method developed by Richardson has been generalized to support uniform reporting of verification results [12].

Two types of verification have been distinguished [13]: Code verification and calculation verification. Code verification is done once for a particular code version, it demonstrates that a specific implementation solves the chosen governing equations correctly. This process can be performed on a series of grids of any size (as long as they are within the asymptotic range) with an arbitrarily chosen solution (no need for physical realism). Calculation verification, on the other hand, is an activity specific to a given scientific investigation, or decision support activity. The solution in this case will be on physically meaningful grids with physically meaningful initial condition (IC)s and boundary condition (BC)s (therefore no a priori-known solution). Rather than monitoring the convergence of an error metric, the convergence of solution functionals relevant to the scientific or engineering development question at hand are tracked to ensure they demonstrate convergence (and ideally, grid/time-step independence).

The approach taken in this work to achieving verification is based on heavy use of a computer algebra system (CAS) [14]. The pioneering work in using computer algebra for supporting the development of computational physics codes was performed by Wirth in 1980 [15]. This was quickly followed by other code generation efforts [16, 17, 18, 19] demonstrations of the use of symbolic math programs to support stability analysis [20] and correctness verification for symbolically generated codes solving governing equations in general curvilinear body-fitted coordinate systems [21].

The CAS handles much of the tedious and error prone manipulation required to implement a numerical PDE solver. It also makes creating the forcing terms necessary for testing against manufactured solutions straight-forward for even very complex governing equations. The MMS is a powerful tool for correctness checking and debugging. The parameters of the manufactured solution allow the magnitude of the contribution of each term to the error to be controlled. In this way, if a code fails to converge for a solution with all parameters (1) (note that this is the recommended approach, hugely different parameter values which might obtain in a physically realistic solution can mask bugs). The parameter sizes can then be varied in a systematic way to locate the source of the non-convergence (as convincingly demonstrated by by Salari and Knupp with a blind test protocol [22]). This gives the code developer a diagnostic capability for the code itself. The error analysis can be viewed as a sort of group test [23]where the “dilution” of each term’s (member’s) contribution to the total (group) error (response) is governed by the relative sizes of the chosen parameters. Though we fit a parametric model (the error ansatz) to determine rate of convergence, the response really is expected to be a binary one as in the classic group test, the ordered convergence rate is maintained down to the round-off plateau or it is not. The dilution only governs how high the resolution must rise (and the error must fall) for this behavior to be confirmed. Terms with small parameters will require that convergence to very high levels is used to ensure that an ordered error is not lurking below.

References

[1] Stults, J., Nonintrusive Microwave Diagnostics of Collisional Plasmasin Hall Thrusters and Dielectric Barrier Discharges, Ph.D. thesis, Air Force Institute of Technology, September 2011.

[2] Oberkampf, W. L. and Trucano, T. G., “Verification and Validation Benchmarks,” Tech. Rep. SAND2002-0529, Sandia National Laboratories, March 2002.

[3] “AIAA Guide for the Verification and Validation of Computational Fluid Dynamics Simulations,” Tech. rep., American Institute of Aeronautics and Astronautics, Reston, VA, 1998.

[4] Cook, S. R., Gelman, A., and Rubin, D. B., “Validation of Software for Bayesian Models Using Posterior Quantiles,” Journal of Computationaland Graphical Statistics, Vol. 15, No. 3, 2006, pp. 675–692.

[5] Oreskes, N., Shrader-Frechette, K., and Belitz, K., “Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences,”Science, Vol. 263, No. 5147, 1994, pp. 641–646.

[6] Roache, P. J., “Building PDE Codes to be Verifiable and Validatable,”Computing in Science and Engineering, Vol. 6, No. 5, September 2004.

[7] “Standard for Verification, Validation in Computational Fluid Dynamics, Heat Transfer,” Tech. rep., American Society of Mechanical Engineers, 2009.

[8] AIAA, “Editorial Policy Statement on Numerical and Experimental Uncertainty,” AIAA Journal, Vol. 32, No. 1, 1994, pp. 3.

[9] Freitas, C., “Editorial Policy Statement on the Control of Numerical Accuracy,” ASME Journal of Fluids Engineering, Vol. 117, No. 1, 1995, pp. 9.

[10] Boole, G., A Treatise on the Calculus of Finite Differences, Macmillian and co., London, 3rd ed., 1880.

[11] Richardson, L. F. and Gaunt, J. A., “The Deferred Approach to the Limit. Part I. Single Lattice. Part II. Interpenetrating Lattices,”Philosophical Transactions of the Royal Society of London. Series A,Containing Papers of a Mathematical or Physical Character, Vol. 226, No. 636-646, 1927, pp. 299–361.

[12] Roache, P. J., “Perspective: A method for uniform reporting of grid refinement studies,” Journal of Fluids Engineering, Vol. 116, No. 3, September 1994.

[13] Roache, P. J., Verification and Validation in Computational Scienceand Engineering, Hermosa Publishers, Albuquerque, 1998.

[14] Fateman, R. J., “A Review of Macsyma,” IEEE Trans. on Knowl. andData Eng., Vol. 1, March 1989, pp. 133–145.

[15] Wirth, M. C., On the Automation of Computational Physics, Ph.D. thesis, University of California, Davis, 1980.

[16] Cook, G. O., Development of a Magnetohydrodynamic Code forAxisymmetric, High-β Plasmas with Complex Magnetic Fields, Ph.D. thesis, Brigham Young University, December 1982.

[17] Steinberg, S. and Roache, P. J., “Using MACSYMA to Write FORTRAN Subroutines,” Journal of Symbolic Computation, Vol. 2, No. 2, 1986, pp. 213–216.

[18] Steinber, S. and Roache, P., “Using VAXIMA to Write FORTRAN Code,” Applications of Computer Algebra, edited by R. Pavelle, Kulwer Academic Publishers, August 1984, pp. 74–94.

[19] Florence, M., Steinberg, S., and Roache, P., “Generating subroutine codes with MACSYMA,” Mathematical and Computer Modelling, Vol. 11, 1988, pp. 1107 – 1111.

[20] Wirth, M. C., “Automatic generation of finite difference equations and fourier stability analyses,” SYMSAC ’81: Proceedings of the fourth ACMsymposium on Symbolic and algebraic computation, ACM, New York, NY, USA, 1981, pp. 73–78.

[21] Steinberg, S. and Roache, P. J., “Symbolic manipulation and computational fluid dynamics,” Journal of Computational Physics, Vol. 57, No. 2, 1985, pp. 251 – 284.

[22] Knupp, P. and Salari, K., “Code Verification by the Method of Manufactured Solutions,” Tech. Rep. SAND2000-1444, Sandia National Labs, June 2000.

[23] Dorfman, R., “The Detection of Defective Members of Large Populations,” The Annals of Mathematical Statistics, Vol. 14, No. 4, 1943, pp. 436–440.

Airframer for Dayton Aerospace Cluster

2011-09-02T08:24:00.002-04:00

In my previous post on the Dayton Aerospace Cluster, I mentioned that one of the big missing pieces is an air-framer. The DDN recently reported that a Fortune100 defense contractor that currently makes UAV components, but does not make full airframes yet is interested in locating a manufacturing / test facility in the Dayton region.

Here's some of the defense contractors on the Fortune 100:

36. Boeing (already makes UAVs: Insitu)
44. United Technologies (claims it doesn't want to enter the UAV market, but UT subsidiary Sikorsky has made significant investments in a small UAV company)
52. Lockheed Martin (already makes UAVs: Sentinel)
72. Northrup Grumman (already makes UAVs: Global Hawk)
86. General Dynamics (sort of makes UAVs: joint venture with Elbit, joint venture with Aurora, joint venture with Aeronautics Defense Systems)

See the full Fortune 500 list. The most interesting one on the list is GD, since two of the three joint ventures listed are with Israeli companies, and Dayton region representatives signed a trade deal in 2009 with representatives from Haifa.

Fun with Filip

2011-09-01T03:00:00.000-04:00

The Filipelli dataset [1] from the NIST Statistical Reference Datasets came up in discussions about ill-conditioned problems over on the Air Vent. Fitting the certified polynomial is a example problem 5.10 in Moler’s MATLAB book [3]. Part of the problem is to compare five different methods of fitting the 10th-order polynomial

MATLAB’s polyfit (which does QR on the Vandermonde matrix)
MATLAB’s backslash operator (which will do QR with pivoting)
Pseudoinverse
$β = X^{†} y$ (1)
Normal equations
$β = {(X^{T} X)}^{- 1} X^{T} y$ (2)
Centering by the mean and normalizing by the standard deviation

Carrick already demonstrated the centering approach in the comments. I was surprised how well the simple centering approach worked (it’s been quite a while since I’ve done this homework problem). Carrick mentioned that Gram-Schmidt might be another way to go, and I mentioned that QR using rotations or reflections is usually preferred for numerical stability reasons. One of the trade-offs is that Gram-Schmidt gives you the basis one vector at a time, but the other methods gives you Q all at once at the end. Depending on the problem one may be more useful than the other.

The LAPACK routine DGEQRG uses the Householder reflection method to calculate the QR decomposition [2]. This routine is wrapped in scipy.linalg.qr(), and it has an option to return only economic factors and perform pivoting. The Scipy version of polyfit() uses the singular value decomposition (from which you get the pseudoinverse).

So, the two main ways of getting robust numerical algorithms for solving least-squares problems are various flavors of $Q R$ and the pseudoinverse. Naive application of some statistical software that doesn’t use these methods by default leads to problems (e.g. SAS and R). Since I don’t like doing other people’s homework I won’t got through Moler’s list, but I did want to demonstrate another method to tackle the problem using the dreaded normal equations (which are terribly conditioned in the case of the Vandermonde matrix).

The basic idea behind both numerical approaches is to build a “good” basis to describe $X$ , so that when we write $X$ in terms of this basis, and perform operations using the basis, we don’t loose lots of precision. We can take a more symbolic approach to the problem, and define an orthogonal basis analytically. So instead of using the monomials as in the Vandermonde matrix, we can use an orthogonal set like the Legendre polynomials. It’s easiest to use this basis if you map the $x$ data to the interval $[- 1, 1]$ .

If you read that thread from the R mailing list, you’ll recognize this approach as the one that was recommended (use the R poly function to define the model matrix with an orthogonal basis rather than the monomials which gives the Vandermonde matrix).

Here’s the Python to do the fit on the raw data just using the built-in polyfit() funciton.

# polyfit uses pseudoinverse
nx = 3000
p = sp.polyfit(x,y,10)
px = sp.linspace(x.min(),x.max(),nx)
# map x to [-1,1]
x_map = (x.min() + x.max() - 2.0*x)/(x.min() - x.max())
px_map = sp.linspace(-1,1,nx)
py = sp.polyval(p, px)
pNISTy = sp.polyval(beta_NIST, px)
NISTy = sp.polyval(beta_NIST, x)

Here’s the Python to do the fit using some other matrix factorizations.

# compare polyfit to QR, this version uses Householder reflections
Q,R = qr(X, mode=’qr’, econ=True)
b = sp.dot(sp.transpose(Q),y)
beta = solve(R,b)
beta = sp.flipud(beta) # polyval expects the coefficients in the other order
qry = sp.polyval(beta, px)

Finally, here is the solution using the Legendre basis to start, which avoids the necessity of generating the basis numerically using QR or the singular value decomposition (scipy.special.legendre).

# compare to pseudoinverse
Xdag = pinv(X)
beta_pinv = sp.flipud(sp.dot(Xdag, y))
pinvy = sp.polyval(beta_pinv, px)
Xmapdag = pinv(X_map)
pinvy_map = sp.polyval(sp.flipud(sp.dot(Xmapdag,y)), px_map)

X1 = sp.zeros((x_map.shape[0],11),dtype=float) # for fitting
X2 = sp.zeros((px_map.shape[0],11),dtype=float) # for plotting
X1[:,0] = 1.0
X2[:,0] = 1.0
portho = []
for i in xrange(11):
    portho.append(legendre(i))
for i in xrange(11):
    X1[:,i] = portho[i](x_map)
    X2[:,i] = portho[i](px_map)
X1dag = pinv(X1)
beta1_pinv = sp.dot(X1dag, y)
pinvy_ortho = sp.dot(X2, beta1_pinv)
# do it the wrong way with X1
b_legendre = sp.dot(sp.transpose(X1),y)
beta_legendre = solve(sp.dot(sp.transpose(X1),X1), b_legendre)
py_legendre = sp.dot(X2, beta_legendre)
LU,piv = lu_factor(sp.dot(sp.transpose(X1),X1))
beta_legendre_lu = lu_solve((LU,piv),b_legendre)
py_legendre_lu = sp.dot(X2, beta_legendre_lu)

The error norm for all of the “good” methods is roughly the same order of magnitude. Even using a bad algorithm, inverting the normal equations, gives a good answer if you start out with a design matrix that is orthogonal to begin with. The QR method is much less analyst-work (and works even without mapping or normalizing the data). Since you just build the basis numerically it requires a bit more cpu-work (which is usually cheap), and it is already coded up and well tested in freely available libraries.

References

[1] Filippelli, A., NIST.

[2] DGEQRF, LAPACK subroutine, version 3.3.1, http://www.netlib.org/lapack/double/dgeqrf.f

[3] Moler, C.B., Numerical computing with MATLAB, Chapter 5: Least Squares, Cambridge University Press, 2004.

SGI Acquires OpenFoam

2011-08-20T10:52:00.001-04:00

OpenFoam is probably the most mature open source CFD tool available. There are plenty of niche CFD projects on SourceForge, but none that have really been able to generate the user/developer community that OpenFoam has. OpenFoam recently made the news because SGI "purchased" it for an undisclosed amount.

"Computational fluid dynamics (CFD) is one of the most important application areas for technical computing," said SGI CEO Mark J. Barrenechea. "With the acquisition of OpenCFD Ltd., SGI will be able to provide our customers the market's first fully integrated CFD solution, where all the hardware and software work together."

Basically, they hired all of the developers:

The entire OpenCFD team, led by Henry Weller, has joined SGI as full-time employees and will be based out of SGI's EMEA Headquarters located in the UK. SGI Acquires Maker of Open Source CFD Software

They also created the OpenFoam Foundation to ensure the continued development of OpenFoam under a GPL license. This is a very good thing for the development of OpenFoam to really take off. The foundation means that a true commons can develop around the software. SGI is clearly using OpenFoam as widget frosting, and they are selling support. This means their incentives line up with making the open version as good as it can be rather than crippling it in deference to a paid "pro" version. Good news for open source computational fluid dynamics.

Second HTV-2 Flight Test

2011-08-14T12:48:00.011-04:00

The second HTV-2 test flight experienced a loss of telemetry about 9 minutes into the flight, very similar to the first test-flight. The post-flight analysis from the first flight indicated that inertia coupling caused the vehicle to depart stable flight (which lead the autopilot to terminate the flight). Here’s a relevant snippet from Bifurcation Analysis for the Inertial Coupling Problem of a Reentry Vehicle,

Inertial coupling has been known since around 1948 [1]. It is essentially a gyroscopic effect, occurring in high roll-rate maneuvers of modern high-speed airplanes including spinning missiles designed in such a way that most of their masses are concentrated in the fuselage. For such an airplane, a slight deviation of its control surface angle from the steady-state angle may lead to a drastic change in roll-rate, causing damage on its empennage; known as the jump phenomenon. Nonlinear analyses to elucidate this problem have been reported in [2, 3], for example. However, the airplanes treated in these works are stable around the equilibrium point of level flight. On the other hand, an intrinsically unstable reentry vehicle, if combined with a malfunction of its AFCSs, may result in a catastrophe once it falls into a high roll-rate motion.

This does sound an awful lot like what happened to both of DARPA’s HTV-2 vehicles. Departure from controlled flight due to inertia coupling was the cause of a loss of crew in the early X-2 testing [4].

Simple explanations are usually not the reason for flight test accidents or mishaps. Experience has shown that there is usually a chain of events that lead to the mishap. Day gives a great summary of the combination of contributing factors that lead to the fatal X-2 test [4],

Optimum energy boost trajectory.
The rocket burn was longer than predicted by 15 sec and positioned the pilot further from Muroc Dry Lake than expected.
Decrease in directional stability as speed increased.
Decrease in directional stability with increased lift, leading to a reduced critical roll rate for inertial coupling.
Adverse aileron control (control reversal).
High positive effective dihedral.
Rudder locked supersonically.
Mass properties in window of susceptibility for inertial roll coupling.

Day describes the result of hitting an unstable region in the control parameter space:

At critical roll velocity, violent uncontrollable motions characteristic of inertial roll coupling occurred about all three axes.

At low-speeds this might be recoverable, but it is simply intractable to design a hypersonic aircraft to handle the loads this kind of rapid motion imposes on the vehicle structure. Eventually something gives, and this “jump” leads rapidly to catastrophic failure.

Day summarizes the reasoning for not modifying the X-2 for manned hypersonic flight,

The X-2 was unfortunately in the twilight zone of progress where rocket power, structural integrity, and thermodynamics were sufficiently advanced to push the aircraft to supersonic speeds. However, hydraulic controls and computerized control augmentation systems were not yet developed enough to contend with the instabilities.

And a relevant snippet from the more recent bifurcation analysis,

If it has unstable dynamics in the neighborhood of a trim point, a slight external disturbance or a slight deviation of control surface angles from their precise values at the trim point cannot allow the vehicle to stay at the trim point [5].

The flight regime that HTV-2 is operating in is unknown enough that it is difficult for designers to know before flight if the flight profile will put the vehicle into an unstable region of the control space. I thought the language that the program manager (a fellow AFIT alum btw) used to describe the post-flight analysis was interesting,

“Assumptions about Mach 20 hypersonic flight were made from physics-based computational models and simulations, wind tunnel testing, and data collected from HTV-2s first test flight the first real data available in this flight regime at Mach 20,” said Air Force Maj. Chris Schulz, HTV-2 program manager who holds a doctorate in aerospace engineering. “Its time to conduct another flight test to validate our assumptions and gain further insight into extremely high Mach regimes that we cannot fully replicate on the ground.” DARPA Press Release

I've noticed physics-based is a common meme for climate policy alarmists trying to shore-up the credibility of simulation predictions. There’s that same tone of, “The Science Says...”, which leads to unwarranted confidence in a situation of profound ignorance (unquantifiable uncertainty). I’ve also observed that program managers seem to take great comfort in the aura of a model that is “physics-based” as a talisman against criticism (“just look at these colorful fluid dynamics, you must be really stupid to question Physics…”). Of course, “physics-based” can be said of all sorts of models of varying fidelity. It is a vague enough moniker to be of great political or rhetorical use. Not that I think Schulz is one of these shady operatives since he goes on to say,

“we wouldn't know exactly what to expect based solely on the snapshots provided in ground testing. Only flight testing reveals the harsh and uncertain reality.”

As DARPA has been well informed by recent experience, ignorance becomes apparent only when confronted with the reality of interest. “Physics-based” is no guarantee of predictive capability.

References

[1] Abzug, M.J. and Larrabee, E.E., Airplane Stability and Control. A History of the Technologies that Made Aviation Possible, Cambridge University Press, Cambridge, U.K., Chap. 8, 1997.

[2] Schy, A.A. and Hannah, M.E., “Prediction of jump phenomena in rolling coupled maneuvers of airplanes,” Journal of Aircraft, Vol. 14, pp. 375–382, April 1977.

[3] Carrol, J.V. and Mehra, R.D., “Bifurcation analysis of nonlinear aircraft dynamics,” Journal of Guidance, Control and Dynamics, Vol. 5, No. 5, pp. 529–536, 1982.

[4] Day, R.E., Coupling Dynamics in Aircraft: A Historical Perspective, NASA Special Publication 532, 1997.

[5] Goto, N. and Kawakita, T., Advances in Dynamics and Control, CRC Press, Chap. 4, 2004.

Latex for Blogger: Pandoc

2011-08-13T18:32:00.000-04:00

The previous post showed some new math rendering tools for supporting Latex for Blogger. That still relied on htlatex (in the tex4ht package) for converting Latex into HTML. I just found another tool called Pandoc that converts between lots of different markup languages. Here’s the first paragraph from the manpage.

Pandoc is a Haskell library for converting from one markup format to another, and a command-line tool that uses this library. It can read markdown and (subsets of) Textile, reStructuredText, HTML, and LaTeX; and it can write plain text, markdown, reStructuredText, HTML, LaTeX, ConTeXt, RTF, DocBook XML, OpenDocument XML, ODT, GNU Texinfo, Medi‐ aWiki markup, EPUB, Textile, groff man pages, Emacs Org-Mode, and Slidy or S5 HTML slide shows.

Sounds promising. So, this post is written in Latex, and then I converted it to HTML with this command:

  [jstults@grafton pandoc]$ pandoc -f latex -t html -o pandoc.html pandoc.tex

Here’s some test equations:
f(x) = x²

$\frac{\partial U}{\partial t} = \nabla U $

Φ = ϕ - ∫ _t₀^tf(τ) dτ

Computational Explosive Astrophysics

2011-08-05T07:32:00.000-04:00

I got an update with an interesting set of slides on adaptive mesh refinement recently. I've always been more of a moving mesh kind of guy, but lots of people find the AMR approach useful. The slides are for a summer course on Computational Explosive Astrophysics. They have an index for the lecture notes where those slides came from. Last year's course was Galaxy Simulations.

Latex for Blogger

2011-07-30T21:57:00.009-04:00

A couple of years ago, I posted that I was using htlatex to do math-heavy posts.This worked in a pinch, the equations are all nicely rendered as *.png’s and theyhave all the normal Latex auto-numbering and cross-referencing. This only worksfor posts though, no equation support in comments. I learned about MathJax,which is a much better option than non-scaling pngs for math, and it supportsequations in comments. All it requires is adding a script to the blogger template file.

Then run htlatex with the mathml option.

[jstults@grafton]$ htlatex foo.tex ”xhtml,mathml”

This turns foo.tex into foo.html, which has the equations in MathMLmarkup. MathJax renders the MathML so everything looks pretty.

MathJax also renders any Latex it finds on the page (including in comments!).The normal single dollar sign for inline math does notwork, since this would cause trouble if people were trying to discussmoney. Here's what does work though:

  % inline math in comments use:
  $$ x + y = z $$
  % equations in comments use:
  \[ x + y = z \]
  % or (for numbered equations):
  \begin{equation} x + y = z \end{equation}

inline in comments:
\ ( x + y = z \)

A couple numbered equations in the post\begin{equation} \frac{\partial x}{\partial t} = A x \end{equation}foo\begin{equation}\frac{\partial y}{\partial t} = g(y) + f \end{equation}

Why ALPAL

2011-05-28T16:23:00.002-04:00

Really it's "why use symbolic math programs to develop number crunching software," but these are from the doc_282 in the ALPAL docs (my emphasis).

First, physics code could be generated and modified to incorporate new physics and features at a much greater speed than occurs with present codes. Such a language would directly foster improvements in the physics models employed, as well as teh numerical methods used to approximate these models. Two reasons point out the need to experiment with numerical methods for a given set of integro-differential equations. First, as a direct consequence of the paucity of mathematical theorems that constructively characterize partial differential equations (PDEs) in general, there is no means of determining what approximation technique will most faithfully capture the solution. Second, in almost all cases, analysis of an approximation technique is limited to idealized linear problems, so the stability and convergence properties of applying a given approximation technique to actual set of PDEs is unknown. Thus, the optimal algorithm for solving any given PDE or set of PDEs is not known, and any tool to aid the computational physicist must be capable of dealing with many different numerical algorithms and methods.
Second, much of the drudgery and concomitant errors in constructing simulation codes would be eliminated by such a language. Large amounts of uninteresting algebra are associated with both the development of appropriate physical models and the discrete versions of the these models on a computer. If the algebraic work involved could be largely automated, computational physicists could spend a great deal more time doing the physics they were trained to do.
Third, a more natural way of describing the physics model is possible with such a language. Errors in the modeling and numerical approximation process would become more obvious, thereby reducing the number of errors in the simulation code.
Fourth, it becomes possible to automate the computation of Jacobian matrices both for linear and nonlinear problems. Not only would this automatic calculation greatly reduce the number of algebraic errors commited for those cases that solve linear systems of equations (with or without a nonlinear solver0, but it would make possible a large number of implicit techniques for situations where it has just not been feasible to use them in the past.
Fifth, such a language could fulfill the need to optimize the very expensive computation that goes on in simulation codes. While a competent scientis can do a good job of optimizing a simple simulation code, the complexity of this task for large simulation codes is beyond the capabilities of even the most skilled scientist. Moreover, optimizing a code is a mundane task that once again does not reward the scientist in his primary pursuit. Optimization becomes an even more critical issue on non-scalar computer architectures such as the CRAY-XMP or an ultracomputer. A high-level view of how to vectorize and/or parallelize a given algorithm (or meta-algorithm) on one of these supercomputers is crucial so the scientist can substantially improve the cost-effectiveness of a simulation on that computer. In principle, a language like ALPAL can provide such a view.
Sixth, since the cost of developing simulation codes is great, especially for new machine architectures, this language could be used to dramatically cut development costs. This is true whether a simulation code is being created for a new machine or whether it is being ported from a previously used computer. Experience with vector computers over the past decade at LLNL has taught this lesson well. More recently, some simulation codes have been ported to the CRAY-XMP, with use being made of its distributed computing capability. This porting to a distributed computer has required an even larger investment of manpower. These large manpower development costs will be repeated many times because a great variety of parallel computers are now appearing, and because parallel computing is a far greater technical challenge than even vector computing.
Seventh, more complete and coherent documentation can be developed with such a language. In fact, a journal-style specification of a code can provide many details that are not generally provided in a journal article about the code. The journal-style specification is expressly designed for readability, wheras the text of a traditional simulation code is not. This is so because it is impossible to express high-level mathematical concepts such as derivatives and integrals together with their numerical approximations in Fortran or any other conventional high-level language.

International Journal for Uncertainty Quantification Latex Template

2011-05-14T15:27:00.004-04:00

I had a bit of trouble getting the Latex document class, ij4uq.cls, for the new International Journal for Uncertainty Quantification to work on my install of Fedora 14, so I figured I'd share my recipe for making it work. The editor and journal support staff were very helpful with quick responses and useful pointers.

First you need to download IJ4UQ Latex Template and unzip that in a convenient location.

[jstults@grafton ij4uq]$ unzip IJ4UQ-Latex-Template.zip

This creates a directory with the ij4uq.cls file you need to make documents for the journal. Instead of the standard article class you'll have something like

\documentclass[review,article]{ij4uq}

at the top of the Latex document. The zip file also contains author instructions, a latex template document, a readme and instructions on installing the Palatino font which is required by the journal style.

The reason I had so much trouble getting this style to work is that Fedora still uses TexLive 2007. There is an effort underway to update things to TexLive 2010 for Fedora 15. Until that becomes standard, you have to turn on a development repository and install texlive. Unless you want to really break your distro and install the styles by hand. However, the sane way is to use the package management system as much as possible.

Second you need to turn on the development repo and update (or if you don't already have texlive installed, then install texlive).

[root@grafton ij4uq] rpm -i http://jnovy.fedorapeople.org/texlive/2010/packages.f14/texlive-release.noarch.rpm [root@grafton ij4uq] yum clean all && yum update

Warning: this is a fairly significant download. Be prepared to go do something else while this crunches.

Third, try to compile the template document. It will likely break because you don't have all of the required *.sty files. Since you are running the sweet new development version of texlive, you can specify these to yum as they apear in the Latex error messages. This saves you from needing to track down which Fedora package provides the particular style file you are missing. This one-liner should get you most of the way there.

[root@grafton ij4uq] yum install 'tex(arial.sty)' 'tex(sectsty.sty)' 'tex(appendix.sty)' 'tex(changebar.sty)' 'tex(nicefrac.sty)' 'tex(lineno.sty)' 'tex(overpic.sty)' 'tex(stfloats.sty)' 'tex(textfit.sty)'

Fourth, you need to fix the unfree floatflt.sty.

[root@grafton ij4uq] mkdir -p /usr/share/texmf-texlive/tex/latex/floatflt[root@grafton floatflt] cd /usr/share/texmf-texlive/tex/latex/floatflt[root@grafton floatflt] rm -f floatflt.* float*.tex[root@grafton floatflt] wget http://mirror.ctan.org/macros/latex/contrib/floatflt/floatflt.ins[root@grafton floatflt] wget http://mirror.ctan.org/macros/latex/contrib/floatflt/floatflt.dtx[root@grafton floatflt] latex floatflt.ins[root@grafton floatflt] texhash

Fifth, you need to update the font maps for the new fonts.

[root@grafton ij4uq] updmap-sys --enable Map /usr/share/texlive/texmf-dist/fonts/map/dvips/palatino/upl.map

That's it. Now the ij4uq.cls template document that comes with the zip file should compile on your (only slightly broken) Fedora 14.

Storms of Our Grandfathers

2011-05-08T13:03:00.007-04:00

Are we "rolling 13s" and getting thousand year storms every year?

NOAA April 2011 Precipitation Anomaly

The contour plots below are taken from Theory of the hydraulic jump and backwater curves. These studies of historical storm records were used to inform design decisions for the Miami Valley Conservancy District's retarding basins and channel improvements following the 1913 floods. My previous post has pictures of the hydraulic jump below Huffman Dam in operation.

My question to Dr Curry about what value high-fidelity (read: relatively expensive to run and analyze) climate simulations have for decision makers was motivated by reading up on infrastructure projects like the retarding basins and channel improvements in the Miami Valley. I think it would be interesting to take a look at a historical project like this that included rudimentary analysis of climate (weather event frequency and magnitude) in its design, and say, "here's how it would be informed differently using modern tools."

The design philosophy taken by the engineers working for the Miami Valley Conservancy District was to design for the worst possible case (historical records from Europe were also considered since they went back further and more reliably) plus roughly twenty percent margin due to the inherent uncertainty in estimating the worst possible case.

If it were necessary to depend wholly on the records of storms which have occurred in the United States, it might be thought possible for moderately great storms to occur over a period of a few hundred years, and then to find, as an exception, a storm three or four times as great. Theoretically that is very improbable, simply because water vapor in sufficient quantities cannot be transported from the ocean or gulf fast and long enough to cause such exceptional storms. As stated in chapter XI, however, records were collected of the stages of rivers in Europe for long periods of time, and these furnish fairly conclusive proof that such great exceptional storms actually do not occur. On the Danube at Vienna, for instance, we have records since about the year 1000 A.D.; fairly accurate records are available for stages of floods in the Tiber at Rome for more than 2,000 years; and records have been made of floods on the Seine at Paris for a long period of years.
Relation of Great Storms to Maximum Possible

After making the extensive investigation of storms in the eastern United States, it is believed that the March, 1913, flood is one of the great floods of centuries in the Miami Valley. In the course of three or four hundred years, however, a flood 15 or 20 per cent greater may occur. We do not believe a flood will ever occur which is more than 20 or 25 per cent in excess of that of March 1913. There is a factor of ignorance, however, against which we must provide, and the only way to do this is arbitrarily to increase the size of the maximum flood to be provided for. If longer records were available a closer estimate could be made, but in planning works on which the protection of the Miami Valley depends, it is necessary to go beyond human judgment. This has been done on all the other phases of the design, and we believe it would not be good engineering practice to stop at our judgment on this phase. We must be able to say that the engineering works are absolutely safe in every respect. For this reason provision is made for a flood nearly 40 per cent greater than that of March 1913. This is 15 or 20 per cent in excess of what is believed to be the greatest possible flood that will ever occur.
Reasons for Choosing as a Basis for Design a Flood 40% Greater than that of March 1913

Would modern tools cut the design margin due to reduced uncertainty or would they indicate that the project is now under-designed due to projected climate change? The latter seems unlikely considering that the magnitude of the purported effects has been repeatably shown to be smaller than we can reliably detect given the length of our data record. Would there be any practically significant changes to the decisions and designs? If your system already has sufficient margin for projected changes in weather-event magnitude do projected changes in frequency matter?

Technocrats and Philosopher Kings can Save our Impotent Polity

2011-05-06T15:34:00.000-04:00

Wow, really awesome article on Climate Resistance, Trust Me, I Speak for Science. I liked these parts from the concluding paragraphs especially. I think you'll notice the parallels to my posts, The Social Ethic and Appeals for Technocracy and No Fluid Dynamicist Kings in Flight-Test.

This metaphysical confusion runs throughout Mooney’s argument. For Mooney, ‘ideology’ is some insidious, toxic force, the antithesis to ‘truth’ itself. The thrust of his argument is that we need particular scientific institutions to ameliorate this intrinsic weakness of human nature. And as such, these institutions deserve elevated status above the reach of those prone to ideology. Otherwise, we would tend towards creationism, to MMR-scares, to climate-change denial. In other words, our flawed minds would create a catastrophe, and it is this possibility of catastrophe that seemingly legitimises the elevated position of scientific institutions. Mooney reinvents Plato’s city state administrated by Philosopher Kings, the main differences being that Mooney conceives of a global polity, and the wisdom of the Guardians only produces the possibility of mere survival, not even a better way of life. To bring this back the matter of trust, Mooney doesn’t trust humans. Their minds are flawed. Their ambitions and ideas are mere fictions. The institutions they create are accordingly founded on false premises, which, instituted and acted upon, will cause disaster. Even when humans are exposed to ‘the truth’, it is, on Mooney’s view, absorbed into the poisonous, ideological programmes of partisans: liars and cheats who distort it. But without a disaster looming, this instance of a politics of fear would collapse.

He simply can’t make a popular argument for his political idea, and so turns to ‘science’ to identify the necessity of such a programme — i.e. the crisis — and to identify reasons why conventional democratic processes cannot realise it...

It's always a good day when you can throw a little Plato into the mix ; - )

Fooling Yourself is Easy

2011-05-04T20:03:00.001-04:00

Common problems from an interesting set of slides:

Confounding in experimental design
Mixing up the sample labels
Mixing up the group labels
Incomplete documentation

"Unfortunately, we suspect, The most simple mistakes are common."

AAAS, Feb 19: pursuing reproducibility audio / slides

Things we look for:

Data
Provenance
Code
Descriptions of nonscriptable steps
Descriptions of Planned Design, if Used

Dayton Flood Control Infrastructure at Work

2011-05-04T06:00:00.026-04:00

Photos of the flood control contrivances in and around Dayton. All of these were taken on 3 May 2011. But first, a little history and engineering detail so you'll have a better appreciation of the pictures.

Arthur P. Morgan came to Dayton after the 1913 flood to design a flood control system to protect the entire Miami Valley. One element of this system was a dry dam—a dam that held water only during a flood and released the water at a rate that the downstream riverbed could carry. The problem was that the speed of the water through the dam made it powerful and destructive. To solve that problem, Morgan went with Col. Edward Deeds to his farm in Moraine where they built models in his swimming pool. They developed the hydraulic jump, which sends water through a series of baffles and steps, and then finally into a low wall that forces the water back onto itself, dissipating its own energy. This process of turning water onto itself is the hydraulic jump. From there, the water flows downstream calmly. This technology is still used in hydrological engineering throughout the world.
From Dayton Inventors River Walk

Here's a graphical depiction of a hydraulic jump.

A basic 1-D analysis follows the figure (clicking the image should take you to the free Google e-book).

Something that's kind of neat is that this system of flood control was designed in the days when "computers" were people (often women) not machines:

These engineers were certainly sure of themselves:
The bottom line is the important part. The hydraulic jump works by increasing the rate of turbulent kinetic energy production, this leads rather quickly (immediately if you assume equilibrium turbulence) to an increased rate of turbulent kinetic energy dissipation at the bottom of the energy cascade. The destructive capability (momentum) of the water is greatly reduced in exchange for raising its temperature ever so slightly.

The following figure shows the cuts that had to be made for the outlet channels and hydraulic jump pools (note Huffman Dam in the center).

And this one shows an aerial shot of the Huffman Dam just after completion.

These views from the top of the Huffman Dam show the turbulence at theend of the outlet channels due to the hydraulic jump.

Huffman Dam Outlet Channels

Huffman Dam Hydraulic Jump Pool Turbulence

View From the Top of Huffman Dam

These types of momentum dissipation mechanisms are also used throughout the city. The submerged dams take momentum out of the four streams that come together in the Dayton city limits: Miami, Mad and Stillwater Rivers and Wolf Creek.

Low Dam North-West of Downtown Dayton

Low Dam downstream of Dayton Canoe Club

There's talk of replacing these with something more water-sport (canoe / kayak) friendly.

Spring 2011 UAV News

2011-05-03T11:59:00.002-04:00

Interesting stuff from today's AIAA news brief.

Light Manned Aircraft May Be Cheaper Option Than UAVs For Some.

Aerospace Daily and Defense Report (5/2, Fulgham) reported that because some countries cannot afford to maintain a force of UAVs for long periods, "a cheaper option is light, Predator-sized, manned aircraft equipped with sensors and weapons designed for the UAV market." Some of these include trainers that are "re-invented as light attack aircraft." Examples cited in the article include the Hawker Beechcraft/Lockheed-Martin AT-6B. According to the article, "given that the aircraft was designed for student-pilot abuse that's similar to the rigors of carrier landings, there appear to be a lot of operational options" such as "irregular warfare, homeland defense and civil support."

"Beast of Kandahar" Employed In Bin Laden Hunt.

Justin Hyde at Jalopnik (5/3) writes how the Lockheed Martin RQ-170 Sentinel UAV known as the "Beast of Kandahar" was used in the operation that ended in bin Laden's death. The drone "was likely the eyes and ears of the operation, streaming live feeds back to command centers."

Satellite Images Show Location.

Space (5/3) reports Digital Globe released archival satellite images taken in January of the area where Osama bin Laden was found and killed. The company "located the probable compound using coordinates and physical descriptions through open sources."

Acquisition Death Spirals

2011-04-17T11:35:00.007-04:00

This isn't about the normal death spiral of increasing unit costs driving production cuts, which increases unit costs, which drives production cuts, which.... It's about another sort of price spiral caused by the US government's infatuation with sole-sourcing critical capabilities. I think this is largely due to technocrats trusting simple, static industrial-age cost models which support decisions dominated by returns from economies of scale. The basic logic of the decisions these models support (an equilibrium solution) is: "things will be cheaper with one supplier because the overhead will be amortized over bigger quantities."

The basic mistake these models make is neglecting the dynamics. Price is a dynamic thing. If the capability is very critical, and there is only one supplier, then there is almost no ceiling on how high the price can rise. The price level reached under those dynamics is just below the point where you'd stop paying for the capability in favor of a more important one (I'll call this the buyer's "level of pain"). The alternative dynamics occurs when there are multiple competing suppliers. The price reached under these dynamics asymptotes towards the economic costs (this takes into account barriers to entry / opportunity costs). Here's a simple graph illustrating price behavior under these two situations.

Price Dynamics

This price increase doesn't happen because the contractors are evil. It happens because the contractors have a duty to their shareholders to maximize profit. This is, in fact, the ethical thing for them to do. If their customer is foolish and decides to create a little monopoly for them with every procurement, well, too bad for the customer's shareholders (taxpayers in this case)...

We see these dynamics play out in a variety of defense acquisitions. The F-35 engine program and the Evolved Expendable Launch Vehicle program are two exemplars that are currently making the news.

The F-35 engine procurement was initially structured to support two suppliers during development, much like the engine programs for F-15 and F-16. There are operational advantages to having two engines. If a problem is found in one model, only half of the Air Force's tactical aircraft would have to be grounded while the solution is found. The other advantage comes from the suppliers competing with each other on price for various lots of engines. A disadvantage is overhead and development cost for the two suppliers and possibly increased logistics footprint for supporting two different engine models.

The recent news is that the budget does not include funds for the second engine. Not a week after this budget is passed which makes the engine buy a sole-source deal, we have an Undersecretary of Defense for Acquisition complaining about the price from the remaining supplier.

"I'm not happy, as I am with so many parts of all our programs, with (the P&W engine's) cost performance so far," Carter told the House of Representatives Appropriations subcommittee on defense on Wednesday. "We need to drive the costs down."[...]"Our analysis does not show the payback," Carter told the subcommittee. He added that "people of good will come to different conclusions on this issue."U.S. "not happy" with F-35 engine cost overruns

Did the analysis include the second supplier offering to assume the risks and go fixed price on the development? Is complaining about the price, being really unhappy about paying it, but paying it anyway because there is no alternative anything but empty political theater? Makes for great content in the trade rags: Acquisition Official gives contractor a stern talking to! Contractor hangs head in a suitably chastened way, "yes, our prices are very high for these unique capabilities, we are working hard to contain the costs for our customer." Much harrumphing is heard from various congresscritters, meanwhile the price continues to spiral higher...

In the case of the EELV, despite the fact that the Air Force paid for two parallel rocket development programs we now have just a single supplier. The two launch service providers were so expensive, they could not compete in the commercial market. The business case for the two rockets hinged on them being able to make money in the commercial market and get their launch rates up. When no other customers but the US government could afford their high prices they had to combine into the single consortium: ULA. So it's sole-source with two vehicles. Recall the various advantages and disadvantages of developing two products discussed above in the case of the F-35 engine. Now EELV has the worst of both worlds: high overhead and logistics costs to support two vehicles, and no competition or customer diversification to get flight rates up and bring prices down.

Launch-service providers agree that their viability, as well as their ability to keep costs down, is based on launch rhythm. The more often a vehicle launches, the more reliable it becomes. Scale economies are introduced as well in a virtuous cycle.One U.S. government official agreed that if SpaceX is now allowed to break ULA’s monopoly on U.S. government satellite launches as indicated by the memorandum of agreement, it could force ULA’s already high prices even higher as it eats into ULA’s current market.“In the longer term we may be faced with questions about whether one of them [ULA or SpaceX] can remain viable without direct subsidies — the same questions we faced with ULA,” this official said. “Then what do we do? We have a policy of assured access to space, which means at least two vehicles. The demand for launches has not increased since ULA was formed, so we could be heading toward a nearly identical situation in a few years. But we are spending taxpayers’ money and if we can find reliable launches that are less expensive, we are not going to ignore that.” SpaceX Receives Boost in Bid To Loft National Security Satellites

It is interesting to note the thinking of the unnamed government official. He doesn't recognize the dynamics of the situation. He's living in a sole-source mindset, it just happens that he's going to change to this new, lower cost source.

NASA has a pricing model that shows savings from "outsourcing development", but not because of any interesting dynamics that they've included. The justification is the same as those underlying the broken decisions about aircraft engines: "returns from economies of scale". The dynamics of competition remains ignored.

NASA Deputy Administrator Lori Garver, in a separate presentation here April 12, said the agency’s policy of pushing rocket-development work onto the private sector will only reach maximum benefit if other customers also purchase the vehicles developed initially with NASA funding.Referring specifically to SpaceX, Garver said a conventional NASA procurement of a Falcon 9-class rocket would cost nearly $4.5 billion according to a NASA-U.S. Air Force cost model that includes the vehicle’s first flight. Outsourcing development to SpaceX, she said, would cut that figure by 60 percent, but only if other customers purchase the vehicle, thus permitting scale economies to reach maximum effect. After Servicing Space Station SpaceXs Priority is Taking on EELV

The way out of the death spiral is program dependent. In the EELV case it took a new entrant who has signed commercial contracts in addition to chasing the government launches (from a couple different agencies). SpaceX has built in significant customer diversification that ULA never developed (though this was hoped for in the early justifications of the program structure). Why did Lockheed-Martin and Boeing, and subsequently ULA never develop this customer diversification? Because they didn't have to. The government guaranteed their continued existence. SpaceX, on the other hand, has no such guarantee. The only option for their continued existence is to make a profit from more than one customer. In the F-35 engine case, DoD has decided that even the assumption of development risk by the second supplier under a fixed price contract is not enough to close the case. This puzzles me. Maybe the second engine has become a symbol of "duplication and waste" rather than "competition and efficiency". If so, then DoD has given the primary engine supplier a way to "frame" their competition out of existence with political argument (removing the need for them to earn market share honestly).

The outlook for lower cost space access looks good. However, as long as the DoD is stuck in its current frame, there is great opportunity for political theater that will serve mainly as a content generator for defense trade publications and a distraction from the root cause of steadily rising tactical aircraft engine costs into the future.

Red Hawks Host Black Knights

2011-02-20T12:10:00.004-05:00

I went to watch some collegiate and amateur bouts down in Oxford on Saturday. Miami University was hosting the cadets from West Point (last year's collegiate champs, nice write-up in NY Times). If you're used to the glammed-up barroom brawling of UFC or even the raw knock-out power of professional boxing, then the style and speed of amateur boxing might come as quite a surprise. I really like the amateur fights because they tend to pivot on conditioning, thinking and skillful execution rather than landing lucky or brutal head-shots.

The 14-bout evening started out with a couple of tough young ladies, one from Cincinati, one from Oxford, going three rounds. It's hard to do match-ups for women because there are just fewer boxers in an already small pool of athletes (since boxing is no longer an NCAA sport). The winner of this bout threw very disciplined, quick, straight punches which her clearly less experienced opponent was ill-equipped to catch or counter. This fight was followed by a few match-ups with local fighters out of Cincinnati, OSU and Miami University. The early fights consisted of lots of off-balanced brawling. The result of "first fight" jitters and inexperience for many of these young athletes.

The cadets from West Point fought out of the blue corner for the remainder of the evening against a line-up consisting of mainly Miami University fighters, with the occasional fighter from OSU or Xavier thrown in to the mix. From the first cadet to fight, on up to the "main event" it was clear why these gentlemen have won three championships in a row.A string of cadets won judges decisions handily over their opponents. In the process demonstrating solid fundamentals, and coolness under the frequent early, but generally dissipative, aggressiveness of their foes.

Then, in the first bout at 132lbs (the second being the "main event" of the night), Lang Clarke of Army landed a solid combination to the head followed-up with a deliberate two-two that sent the man from Xavier to the mat (the referee was in the midst of "stop" as the second right landed). The first and only knock-out of the evening. The Xavier athlete was back on his feet (to the relieved cheers of the crowd) after a quick nap and a check from the ring-side doc.

The only heavy-weight bout of the night was stopped by the referee near the end of the first round. The fighter from West Point landed repeated hooks-to-the-head which the young man from Oxford was not defending.

The fight at 195lbs was relatively surprising, not in outcome (the cadet won), but in tactics. Previously the cadets had employed a shorter and simpler version of the rope-a-dope tactic when their less disciplined and less well-conditioned opponents came out swinging. Rather than stand and brawl toe-to-toe, they defended and let the other fighter tire, then exploited that self-inflicted weakness with steady "work" for the rest of the round. This cadet stood up inside his opponent's early windmill, and landed straight punches and upper cuts to the head. One or two furious windmillings punctuated by deliberately thrown and well-landed opposing hits was all that it took for the windmill's blades to drop and the hub to wobble on it's axis. Referee stops contest.

The crowd was ready for their main event. A nice cheer went up for the wiry 132-pounder from Oxford as he stepped in the ring. Yours truly was the only one to cheer when the young man from West Point entered the ring (much to my wife's embarrassment). After she heard the raucous cheer when they actually introduced the Oxford fighter, she said, "OK, you can go ahead and yell for that West Point guy," so I did.

Both fighters were about equally conditioned, which made for a much more exciting fight. They were both able to work (with varying levels of effectiveness) for the majority of each round. The fighter from Oxford threw a great volume of widely-arcing punches, most of which seemed ineffective to me due to the cadet's competent defense. After the repeated, straight head-shots landed by the cadet in the third round, I thought the decision would go his way (if the fight was not stopped sooner, which had been the outcome of this sort of pounding previously). The judges had a different perspective on the bout, so the decision went to the man from Oxford, who, much to his credit, was able to recover repeatedly from the wobbliness induced by these direct blows and swing away until the bell relieved him.

The coaches, medical and officiating crew at Miami University should be congratulated for putting on such a professional event that took good care of these young athletes, and allowed them to further develop their skills.

Thought this was funny; why you don't want guys from the same gym on the card:

Sparring partners are endowed with habitual consideration and forbearance, and they find it hard to change character. A kind of guild fellowship holds them together, and they pepper each other's elbows with merry abandon, grunting with pleasure like hippopotamuses in a beer vat.
The Sweet Science

Historical Hydraulics

2011-02-19T13:51:00.001-05:00

Venturi's drawings of eddies look really modern too, kind of neat:

[h/t Dan Hughes]

Comments on Spatio-Temporal Chaos

2011-02-15T10:53:00.002-05:00

Some comments from a guest post on Dr Curry's site. I think she has a couple dueling chat bots who've taken up residence in her comments (see if you can guess who they are). This provides a bit more motivation for getting to the forced system results we started talking about earlier. The paper and discussion that Arthur Smith links is well worth a read (even though it isn't actually responsive ; - ).

Arthur Smith | February 13, 2011 at 8:22 pm | Reply

Tomas – you claimed to focus on my comment, but *completely ignored* the central element, which you even quoted:
“small random variations in solar input (not to mention butterflies)” [as what makes weather random over the long term]
Chaos as you have discussed it requires fixed control parameters (absolutely constant solar input) and no external sources of variation not accounted for in the equations (no butterflies). You gave zero attention in your supposed response to my comment to this central issue. Others here have been accused of being non-responsive, but I have to say that is pretty non-responsive on your part.
The fact is as soon as there is any external perturbation of a chaotic system not accounted for in the dynamical equations, you have bumped the system from one path in phase space to another. Earth’s climate is continually getting bumped by external perturbations small and large. The effect of these is to move the actual observed trajectory of the system randomly – yes randomly – among the different possible states available for given energy/control parameters etc.
The randomness comes not from the chaos, but from external perturbation. Chaos amplifies the randomness so that at a time sufficiently far in the future after even the smallest perturbation, the actual state of the system is randomly sampled from those available. That random sampling means it has real statistics. The “states available” are constrained by boundaries – solar input, surface topography, etc. which makes the climate problem – the problem of the statistics of weather – a boundary value problem (BVP). There are many techniques for studying BVP’s – one of which is simply to randomly sample the states using as physical a model as possible to get the right statistics. That’s what most climate models do. That doesn’t mean it’s not a BVP.

Arthur Smith | February 14, 2011 at 8:22 am | Reply

This isn’t anything new – almost every physical dynamical system, if it’s not trivially simple, displays chaos under most conditions. Statistical mechanics, one of the most successful of all physical theories, relies fundamentally on the reliability of a statistical description of what is actually deterministic (and chaotic – way-more-than-3-body) dynamics of immense numbers of atoms and molecules. This goes back to Gibbs over a century ago, and Poincare’s work was directly related.
Tomas’ comments about the 3-body system being not even “predictable statistically (e.g you can not put a probability on the event “Mars will be ejected from the solar system in N years”” is true in the strict sense of the exact mathematics assuming no external perturbations. That’s simply because for a deterministic system something will either happen or it won’t, there’s no issue of probability about it at all. But as soon as you add any sort of noise, your perfect chaotic system becomes a mere stochastic one over long time periods, and probabilities really do apply.
A nice review of the relationships between chaos, probability and statistics is this article from 1992:
“Statistics, Probability and Chaos” by L. Mark Berliner, Statist. Sci. Volume 7, Number 1 (1992), 69-90.
http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.ss/1177011444
and see some of the discussion that followed in that journal (comments linked on that Project Euclid page).

jstults | February 14, 2011 at 1:37 pm | Reply

Arthur Smith, while that is a very good paper that you linked (thank you for finding one that everyone can access), it only had a very short section on ergodic theory, and you’re back to the same hand-waving analogy about statistical mechanics and turbulent flows. The [lack of] success for simple models (based on analogy to kinetic theory btw) for turbulent flows of any significant complexity indicates to me that I can’t take your analogy very seriously.
Where’s the meat? Where’s the results for the problems we care about? I can calculate results for logistic maps and Lorenz ’63 on my laptop (and the attractor for that particular toy exists).

jstults | February 15, 2011 at 10:22 am | Reply

A more well-phrased attempt to explain why hand-waving about statistical mechanics is a diversion from the questions of significance for this problem (with apologies to Ruelle): what are the measures describing climate?

If one is optimistic, one may hope that the asymptotic measures will play for dissipative systems the sort of role which the Gibbs ensembles have played for statistical mechanics. Even if that is the case, the difficulties encountered in statistical mechanics in going from Gibbs ensembles to a theory of phase transitions may serve as a warning that we are, for dissipative systems, not yet close to a real theory of turbulence.

What Are the Measures Describing Turbulence?

Validation and Calibration: more flowcharts

2011-02-04T19:55:00.003-05:00

In a previous post we developed a flow-chart for model verification and validation (V&V) activities. One thing I noted in the update on that post was that calibration activities were absent. My google alerts just turned up a new paper (they reference the Thacker et al. paper the previous post was based on, I think you’ll notice the resemblance of flow-charts) which adds the calibration activity in much the way we discussed.

Figure 1:

Model Calibration Flow Chart of Youn et al. [1]

The distinction between calibration and validation is clearly highlighted, “In many engineering problems, especially if unknown model variables exist in a computational model, model improvement is a necessary step during the validation process to bring the model into better agreement with experimental data. We can improve the model using two strategies: Strategy 1 updates the model through calibration and Strategy 2 refines the model to change the model form.”

Figure 2:

Flow chart from previous post

The well-founded criticism of calibration-based arguments for simulation credibility is that calibration provides no indication of the predictive capability of a model so-tuned. The statistician might use the term generalization risk to talk about the same idea. There is no magic here. Applying techniques such as cross-validation merely add a (hyper)parameter to the model (this becomes readily apparent in a Bayesian framework). Such techniques, while certainly useful, are no silver bullet against over-confidence. This is a fundamental truth that will not change with improving technique or technology, and that is because all probability statements are conditional on (among other things) the choice of model space (particular choices of which must by necessity be finite, though the space of all possible models is countably infinite).

One of the other interesting things in that paper is their argument for a hierarchical framework for model calibration / validation. A long time ago, in a previous life, I made a similar argument[2]. Looking back on that article is a little embarrassing. I wrote that before I had read Jaynes (or much else of the Bayesian analysis and design of experiments literature), so it seems very technically naive to me now. The basic heuristics for product development discussed in it are sound though. They’re based mostly on GAO reports [3, 4, 5, 6], a report by NAS[7], lessons learned from Live Fire Test and Evaluation [8] and personal experience in flight test. Now I understand better why some of those heuristics have sound theoretical underpinnings.

There are really two hierarchies though. There is the physical hierarchy of system, sub-system and component that Youn et al. emphasize, but there is also a modeling hierarchy. This modeling hierarchy is delineated by the level of aggregation, or the amount of reductive-ness, in the model. All models are reductive (that’s the whole point of modeling: massage the inordinately complex and ill-posed into tractability), some are just more reductive than others.

Figure 3:

Modeling Hierarchy (from [2])

Figure 3 illustrates why I care about Bayesian inference. It’s really the only way to coherently combine information from the bottom of the pyramid (computational physics simulations), with information higher up the pyramid which rely on component and subsystem testing.

A few things I don’t like about the approach in [1]

The partitioning of parameters into “known” and “unknown” based on what level of the hierarchy (component, subsystem, system) you are at in the “bottom-up” calibration process. Our (properly formulated) models should tell us how much information different types of test data give us about the different parameters. Parameters should always be described by a distribution rather than discrete switches like known or unknown.
The approach is based entirely on the likelihood (but they do mention something that sounds like expert priors in passing).
They claim that the proposed calibration method enhances “predictive capability” (section 3), however this is misleading abuse of terminology. Certainly the in-sample performance is improved by calibration, but the whole point of making a distinction between calibration and validation is based on recognizing that this says little about the out-of-sample performance (in fairness, they do equivocate a bit on this point, “The authors acknowledge that it is difficult to assure the predictive capability of an improved model without the assumption that the randomness in the true response primarily comes from the the randomness in random model variables.”).

Otherwise, I find this a valuable paper that strikes a pragmatic chord, and that’s why I wanted to share my thoughts on it.

[Update: This thesis that I linked at Climate Etc. has a flow-chart too.

]

References

[1] Youn, B. D., Jung, B. C., Xi, Z., Kim, S. B., and Lee, W., “A hierarchical framework for statistical model calibration in engineering product development,” Computer Methods in Applied Mechanics and Engineering, Vol. 200, No. 13-16, 2011, pp. 1421 – 1431.

[2] Stults, J. A., “Best Practices for Developmental Testing of Modern, Complex Munitions,” ITEA Journal, Vol. 29, No. 1, March 2008, pp. 67–74.

[3] “Defense Acquisitions: Assesment of Major Weapon Programs,” Tech. Rep. GAO-03-476, U.S. General Accounting Office, May 2003.

[4] “Best Practices: Better Support of Weapon System Program Managers Needed to Improve Outcomes,” Tech. Rep. GAO-06-110, U.S. General Accounting Office, 2006.

[5] “Precision-Guided Munitions: Acquisition Plans for the Joint Air-to-Surface Standoff Missile,” Tech. Rep. GAO/NSIAD-96-144, U.S. General Accounting Office, 1996.

[6] “Best Practices: A More Constructive Test Approach is Key to Better Weapon System Outcomes,” Tech. Rep. GAO/NSIAD-00-199, U.S. General Accounting Office, July 2000.

[7] Michael L. Cohen, John E. Rolph, D. L. S., editor, Statistics, Testing and Defense Acquisition: New Approaches and Methodological Improvements, National Academy Press, Washington D.C., 1998.

[8] O’Bryon, J. F., editor, Lessons Learned from Live Fire Testing: Insights Into Designing, Testing, and Operating U.S. Air, Land, and Sea Combat Systems for Improved Survivability and Lethality, Office of the Director, Operational Test and Evaluation, Live Fire Test and Evaluation, Office of the Secretary of Defense, January 2007.

Have Red

2011-01-31T17:28:00.001-05:00

This Ain't New Mexico

Better Recurrence Plots

2011-01-23T12:43:00.004-05:00

In the previous post in the Lorenz63 series we used recurrence plots to get a qualitative feel for the type of behavior exhibited by a time series (stochastic, periodic, chaotic). Those were using the default colormap in matplotlib, and they seem to highlight the “holes” more than the “near returns” (at least to my eye). Here’s some improved ones that use the bone colormap and a threshold on the distance to better highlight the near returns.

(a) Single Trajectory

(b) Ensemble Mean

Figure 1:

Response of Lorenz 1963 model

(a) Periodic Series

(b) Stochastic Series

Figure 2:

Non-chaotic Series

(a) A little smoothing

(b) More smoothing

Figure 3:

Smoothing of a Lorenz 1963 trajectory

Recurrence, Averaging and Predictability

2011-01-22T13:15:00.008-05:00

Motivation and Background

Yet another installment in the Lorenz63 series. This time motivated by a commenter on Climate Etc. Tomas Milanovic claims that time averages are chaotic too in response to the oft repeated claim that the predictability limitations of nonlinear dynamical systems are not a problem in the case of climate prediction. Lorenz would seem to agree, “most climatic elements, and certainly climatic means, are not predictable in the first sense at infinite range, since a non-periodic series cannot be made periodic through averaging [1].” We’re not going to just take his word on it. We’ll see if we can demonstrate this with our toy model.

That’s the motivation, but before we get to toy model results a little background discussion is in order. In this previous entry I illustrated the different types of functionals that you might be interested in depending on whether you are doing weather prediction or climate prediction. I also made the remark, “A climate prediction is trying to provide a predictive distribution of a time-averaged atmospheric state which is (hopefully) independent of time far enough into the future.” It was pointed out to me that this is a testable hypothesis [2], and that the empirical evidence doesn’t seem to support the existence of time-averages (or other functionals) describing the Earth’s climate system that are independent of time [3]. In fact, the above assumption was critiqued by none other than Lorenz in 1968 [4]. In that paper he states,

Questions concerning the existence and uniqueness of long-term statistics fall into the realm of ergodic theory. [...] In the case of nonlinear equations, the uniqueness of long-term statistics is not assured. From the way in which the problem is formulated, the system of equations, expressed in deterministic form, together with a specified set of initial conditions, determines a time-dependent solution extending indefinitely into the future, and therefore determines a set of long-term statistics. The question remains as to whether such statistics are independent of the choice of initial conditions.

He goes on to define a system as transitive if the long-term statistics are independent of initial condition, and intransitive if there are “two or more sets of long-term statistics, each of which has a greater-than-zero probability of resulting from randomly chosen initial conditions.” Since the concept of climate change has no meaning for statistics over infinitely long intervals, he then defines a system as almost intransitive if the statistics at infinity are unique, but the statistics over finite intervals depend (perhaps even sensitively) on initial conditions. In the context of policy relevance we are generally interested in behavior over finite time-intervals.

In fact, from what I’ve been able to find, different large-scale spatial averages (or coherent structures, which you could track by suitable projections or filtering) of state for the climate system face similar limits to predictability as un-averaged states. The predictability just decays at a slower rate. So instead of predictive limitations for weather-like functionals on the order of a few weeks, the more climate-like functionals become unpredictable on slower time-scales. There’s no magic here, things don’t suddenly become predictable a couple decades or a century hence because you take an average. It’s just that averaging or filtering may change the rate that errors for that functional grow (because in spatio-temporal chaos different structures, or state vectors, will have different error growth rates and reach saturation at different times). Again Lorenz puts it well, “the theory which assures us of ultimate decay of atmospheric predictability says nothing about the rate of decay” [1]. Recent work shows that initialization matters for decadal prediction, and that the predictability of various functionals decay at different rates [5]. For instance, sea surface temperature anomalies are predictable at longer forecast horizons than surface temperatures over land. Hind-casts of large spatial averages on decadal time-scales have shown skill in the last two decades of the past century (though they had trouble beating a persistence forecast for much of the rest of the century) [6].

I’ve noticed in on-line discussions about climate science that some people think that the problem of establishing long term statistics for nonlinear systems is a solved one. That is not the case for the complex, nonlinear systems we are generally most interested in (there are results for our toy though [7, 8]). I think this snippet sums things up well,

Atmospheric and oceanic forcings are strongest at global equilibrium scales of 10⁷ m and seasons to millennia. Fluid mixing and dissipation occur at micorscales of 10^-3 m and 10^-3s, and cloud particulate transformations happen at 10^-6 m or smaller. Observed intrinsic variability is spectrally broad band across all intermediate scales. A full representation for all dynamical degrees of freedom in different quantities and scales is uncomputable even with optimistically foreseeable computer technology. No fundamentally reliable reduction of the size of the AOS [atmospheric oceanic simulation] dynamical system (i.e., a statistical mechanics analogous to the transition between molecular kinetics and fluid dynamics) is yet envisioned. [9]

Here McWilliams is making a point similar to that made by Lorenz in [4] about establishing a statistical mechanics for climate. This would be great if it happened, because that would mean that the problem of turbulence would be solved for us engineers too. Right now the best we have (engineers interested in turbulent flows and climate scientists too) is empirically adequate models that are calibrated to work well in specific corners of reality.

Lorenz was responsible for another useful concept concerning predictability, that is predictability of the first and second kind [1]. If you care about the time-accurate evolution of the order of states then you are interested in predictability of the first kind. If, however, you do not care about the order, but only the statistics, then you are concerned with predictability of the second kind. Unfortunately, Lorenz’s concepts of first and second kind predictability have been morphed in to a claim that first kind predictability is about solving initial value problem (IVP)s and second kind predictability is about solving boundary value problem (BVP)s. For example, “Predictability of the second kind focuses on the boundary value problem: how predictable changes in the boundary conditions that affect climate can provide predictive power [5].” This is unsound. If you read Lorenz closely, you’ll see that the important open question he was exploring about whether the climate is transitive, intransitive or almost intransitive has been assumed away by the spurious association of kinds of predictability with kinds of problems [1]. Lorenz never made this mistake, he was always clear that the difference in kinds of predictability depends on the functionals you are interested in, not whether it is appropriate to solve an IVP or a BVP (what reason could you have for expecting meaningful frequency statistics from a solution to a BVP?). Those considerations depend on the sort of system you have. In an intransitive or almost intransitive system even climate-like functionals depend on the initial conditions.

A good early paper on applying information theory concepts to climate predictability is by Leung and North [10], and there is a more recent review article that covers the basic concepts by DelSole and Tippett [11].

Recurrence Plots

Recurrence plots are useful for getting a quick qualitative feel for the type of response exhibited by a time-series [12, 13]. First we run a little initial condition (IC) ensemble with our toy model. The computer experiment we’ll run to explore this question will consist of perturbations to the initial conditions (I chose the size of the perturbation so the ensemble would blow-up around t = 12). Rather than sampling from a distribution for the members of the ensemble, I chose them according a stochastic collocation (this helps in getting the same results every time too).

(a)EnsembleTrajectories

(b)EnsembleMean

Figure 1:

Initial Condition Ensemble

One thing that these two plots makes clear is that it doesn’t make much sense to compare individual trajectories with the ensemble mean. The mean is a parameter of a distribution describing a population of which the trajectories are members. While the trajectories are all orbits on the attractor, the mean is not.

(a)SingleTrajectory

(b)EnsembleMean

Figure 2:

Chaotic Recurrence Plots

Comparing the chaotic recurrence plots with the plots below of a periodic series and a stochastic series illustrates the qualitative differences in appearance.

(a)PeriodicSeries

(b)StochasticSeries

Figure 3:

Non-chaotic Recurrence Plots

Clearly, both the ensemble mean and the individual trajectory are chaotic series, sort of “between” periodic and stochastic in their appearance. Ensemble averaging doesn’t make our chaotic series non-chaotic, what about time averaging?

Predictability Decay

How does averaging affect the decay of predictability for the state of the Lorenz63 system, and can we measure this effect? We can track how the predictability of the future state decays given knowledge of the initial state by using the relative entropy. There are other choices for measures such as mutual information [10]. Since we’ve already got our ensemble though, we can just use entropy like we did before. Rather than just a simple moving average, I’ll be calculating an exponentially weighted one using an FFT-based approach, of course (there’s some edge effects we’d need to worry about if this were a serious analysis, but we’ll ignore that for now). The entropy for the ensemble is shown for three different smoothing levels in Figure 4 (the high entropy prior to t = 5 for the smoothed series is spurious because I didn’t pad the series and it’s calculated with the FFT).

Figure 4:

Entropy of Exponentially Weighted Smoothed Series

While smoothing does lower the entropy of the ensemble (lower entropy for more smoothing / smaller λ), it still experiences the same sort of “blow-up” as the unsmoothed trajectory. This indicates problems for predictability even for our time-averaged functionals. Guess what? The recurrence plot indicates that our smoothed trajectory is still chaotic!

Figure 5:

Smoothed Trajectory Recurrence Plot

This result shouldn't be too surprising, moving averages or smoothing (of whatever type you fancy) are linear operations. It would probably take a pretty clever nonlinear transformation to turn a chaotic series into a non-chaotic one (think about how the series in this case is generated in the first place). I wouldn't expect any combination of linear transformations to accomplish that.

Conclusions

I’ll begin the end with another great point from McWilliams (though I’ve not heard of sub-grid fluctuations referred to as “computational noise,” that term makes me think of round-off error) that should serve to temper our demands of predictive capability from climate models[9]:

Among their other roles, parametrizations regularize the solutions on the grid scale by limiting fine-scale variance (also known as computational noise). This practice makes the choices of discrete algorithms quite influential on the results, and it removes the simulation from the mathematically preferable realm of asymptotic convergence with resolution, in which the results are independent of resolution and all well conceived algorithms yield the same answer.

If I had read this earlier, I wouldn’t have spent so much time searching for something that doesn’t exist.

Regardless of my tortured learning process, what do the toy models tell us? Our ability to predict the future is fundamentally limited. Not really an earth-shattering discovery; it seems a whole lot like common sense. Does this have any implication for how we make decisions? I think it does. Our choices should be robust with respect to these inescapable limitations. In engineering we look for broad optimums that are insensitive to design or requirements uncertainties. The same sort of design thinking applies to strategic decision making or policy design. The fundamental truism for us to remember in trying to make good decisions under the uncertainty caused by practical and theoretical constraints is that limits on predictability do not imply impotence.

References

[1] Lorenz, E. N., The Physical Basis of Climate and Climate Modeling, Vol. 16 of GARP publication series, chap. Climatic Predictability, World Meteorological Organization, 1975, pp. 132–136.

[2] Pielke Sr, R. A., “your query,” September 2010, electronic mail to the author.

[3] Rial, J. A., Pielke Sr, R. A., Beniston, M., Claussen, M., Canadell, J., Cox, P., Held, H., Noblet-Ducoudr, N. D., Prinn, R., Reynolds, J. F., and Salas, J. D., “Nonlinearities, Feedbacks And Critical Thresholds Within The EarthS Climate System,” Climatic Change, Vol. 65, No. 1-2, 2004, pp. 11–38.

[4] Lorenz, E. N., “Climatic Determinism,” Meteorological Monographs, Vol. 8, No. 30, 1968.

[5] Collins, M. and Allen, M. R., “Assessing The Relative Roles Of Initial And Boundary Conditions In Interannual To Decadal Climate Predictability,” Journal ofClimate, Vol. 15, No. 21, 2002, pp. 3104–3109.

[6] Lee, T. C., Zwiers, F. W., Zhang, X., and Tsao, M., “Evidence of Decadal Climate Prediction Skill Resulting from Changes in Anthropogenic Forcing,” Journal of Climate, Vol. 19, 2006.

[7] Tucker, W., The Lorenz Attractor Exists, Ph.D. thesis, Uppsala University, 1998.

[8] Kehlet, B. and Logg, A., “Long-Time Computability of the Lorenz System,”http://lorenzsystem.net/.

[9] McWilliams, J. C., “Irreducible Imprecision In Atmospheric And Oceanic Simulations,” Vol. 104 of National Academy of Sciences, National Academy of Sciences, pp. 8709 – 8713.

[10] Leung, L.-Y. and North, G. R., “Information Theory and Climate Prediction,”Journal of Climate, Vol. 3, 1990, pp. 5–14.

[11] DelSole, T. and Tippett, M. K., “Predictability: Recent insights from information theory,” Reviews of Geophysics, Vol. 45, 2007.

[12] Eckmann, J.-P., Kamphorst, S. O., and Ruelle, D., “Recurrence Plots of Dynamical Systems,” EPL (Europhysics Letters), Vol. 4, No. 9, 1987, pp. 973.

[13] Marwan, N., “A historical review of recurrence plots,” The European PhysicalJournal - Special Topics, Vol. 164, 2008, pp. 3–12, 10.1140/epjst/e2008-00829-1.

Empiricism and Simulation

2011-01-19T16:22:00.013-05:00

There are two orthogonal ideas that seem to get conflated in discussions about climate modeling. One is the idea that you’re not doing science if you can’t do a controlled experiment, but of course we have observational sciences like astronomy. The other is that all this new-fangled computer-based simulation is untrustworthy, usually because “it ain’t the way my grandaddy did science.” Both are rather silly ideas. We can still weigh the evidence for competing models based on observation, and we can still find protection from fooling ourselves even when those models are complex.

What does it mean to be an experimental as opposed to an observational science? Do sensitivity studies, and observational diagnostics using sophisticated simulations count as experiments? Easterbrook claims that because climate scientists do these two things with their models that climate science is an experimental science [1]. It seems like there is a motivation to claim the mantle of experimental, because it may carry more rhetorical credibility than the merely observational (the critic Easterbrook is addressing certainly thinks so). This is probably because the statements we can make about causality and the strength of the inferences we can draw are usually greater when we can run controlled experiments than when we are stuck with whatever natural experiments fortune provisions for us (and there are sound mathematical reasons for this, having to do with optimality in experimental design rather than any label we may place on the source of the data). This seeming motivation demonstrated by Easterbrook to embrace the label of empirical is in sharp contrast to the denigration of the empirical by Tobis in his three part series [2, 3, 4]. As I noted on his site, the narrative Tobis is trying to create with those posts has already been pre-messed with by Easterbrook, his readers just pointed out the obvious weaknesses too. One good thing about blogging is the critical and timely feedback.

The confusions of these two climate warriors are an interesting point of departure. I think they are both saying more than blah blah blah, so it’s worth trying to clarify this issue. The figure below is based on a technical report from Sandia [5], which is a good overview and description of the concepts and definitions for model verification and validation as it has developed in the computational physics community over the past decade or so. I think this emerging body of work on model V&V places the relative parts, experiment and simulation, in a sound framework for decision making and reasoning about what models mean.

Figure 1:

Verification and Validation Process (based largely on [5])

The process starts at the top of the flowchart with a “Reality of Interest”, from which a conceptual model is developed. At this point the path splits into two main branches. One based on “Physical Modeling” and the other based on “Mathematical Modeling”. Something I don’t think many people realize is that there is a significant tradition of modeling in science that isn’t based on equations. It is no coincidence that an aeronautical engineer might talk of testing ideas with a wind-tunnel model or a CFD model. Both models are simplifications of the reality of interest, which, for that engineer, is usually a full-scale vehicle in free flight.

Figure 2 is just a look at the V&V process through my Design of Experiments (DoE) colored glasses.

Figure 2:

Distorted Verification and Validation Process

My distorted view of the V&V process is shown to emphasize that there’s plenty of room for experimentalists to have fun (maybe even a job [3]) in this, admittedly model-centric, sandbox. However, the transferability of the basic experimental design skills between “Validation Experiments” and “Computational Experiments” says nothing about what category of science one is practicing. The method of developing models may very well be empirical (and I think Professor Easterbrook and I would agree it is, and maybe even should be), but that changes nothing about the source of the data which is used for “Model Validation.”

The computational experiments highlighted in Figure 2 are for correctness checking, but those aren’t the sorts of computational experiments Easterbrook claimed made climate science an experimental science. Where do sensitivity studies and model-based diagnostics fit on the flowchart? I think sensitivity studies fit well in the activity called “Pre-test Calculations”, which, one would hope, inform the design of experimental campaigns. Diagnostics are more complicated.

Heald and Wharton have a good explanation for the use of the term “diagnostic” in their book on microwave-based plasma diagnostics: “The term ‘diagnostics,’ of course, comes from the medical profession. The word was first borrowed by scientists engaged in testing nuclear explosions about 15 years ago [c. 1950] to describe measurements in which they deduced the progress of various physical processes from the observable external symptoms” [6]. With a diagnostic we are using the model to help us generate our “Experimental Data”, so that would happen within the activity of “Experimentation” on this flowchart. This use of models as diagnostic tools is applied to data obtained from either experiment (e.g. laboratory plasma diagnostics) or observations (e.g. astronomy, climate science), so it says nothing about whether a particular science is observational or experimental. Classifying scientific activities as experimental or observational is of passing interest, but I think far too much emphasis is placed on this question for the purpose of winning rhetorical “points.”

The more interesting issue from a V&V perspective is introducing a new connection in the flowchart that shows how a dependency between model and experimental data could exist (Figure 3). Most of the time the diagnostic model, and the model being validated are different. However, this case where they are the same is an interesting and practically relevant one that is not addressed in the current V&V literature that I know of (please share links if you “know of”).

Figure 3:

V&V process including model-based diagnostic

It should be noted that even though the same model may be used to make predictions and perform diagnostics, it will usually be run in a different way for those two uses. The significant changes between Figure 1 and Figure 3 are the addition of a “Experimental Diagnostic” box and the change to the mathematical cartoon in the “Validation Experiment” box. The change to the cartoon is to indicate that we can’t measure what we want directly (u), so we have to use a diagnostic model to estimate it based on the things we can measure (b). An example of when the model-based diagnostic is relatively independent of the model being validated might be using laser-based diagnostic for fluid flow. The equations describing propagation of the laser through the fluid are not the same as those describing the flow. An example of when the two codes might be connected would be if you were trying to use ultrasound to diagnose a flow. The diagnostic model and the predictive model could both be Navier-Stokes with turbulence closures. Establishing the validity of which is the aim of the investigation. I’d be interested in criticisms of how I explained this / charted this out.

Afterward

Attempt at Answering Model Questions

I’m not in the target population that professor Easterbrook is studying, but here’s my attempt at answering his questions about model validation[7].

“If I understand correctly–a model is ’valid’ (is that a formal term?) if the code is written to correctly represent the best theoretical science at the time...”
I think you are using an STS flavored definition for “valid.” The IEEE/AIAA/ASME/US-DoE/US-DoD definition differs. “Valid” means observables you get out of your simulations are “close enough” to observables in the wild (experimental results). The folks from DoE tend to argue for a broader definition of valid than the DoD folks. They’d like to include as “validation” activities of a scientist comparing simulation results and experimental results without reference to an intended use.
“– so then what do the results tell you? What are you modeling for–or what are the possible results or output of the model?”
Doing a simulation (running the implementation of a model) makes explicit the knowledge implicit in your modeling choices. The model is just the governing equations, you have to run a simulation to find solutions to those governing equations.
“If the model tells you something you weren’t expecting, does that mean it’s invalid? When would you get a result or output that conflicts with theory and then assess whether the theory needs to be reconsidered?”
This question doesn’t make sense to me. How could you get a model output that conflicted with theory? The model is based on theory. Maybe this question is about how simplifying assumptions could lead to spurious results? For example, if a simulation result shows failure to conserve mass/momentum/energy in a specific calculation possibly due to a modeling assumption (more likely due to a more mundane error), I don’t think anyone but a perpetual-motion machine nutter would seriously reconsider the conservation laws.
“Then is it the theory and not the model that is the best tool for understanding what will happen in the future? Is the best we can say about what will happen that we have a theory that adheres to what we know about the field and that makes sense based on that knowledge?”
This one doesn’t make sense to me either. You have a “theory,” but you can’t formulate a “model” of it and run a simulation, or just a pencil and paper calculation? I don’t think I’m understanding how you are using those words.
“What then is the protection or assurance that the theory is accurate? How can one ‘check’ predictions without simply waiting to see if they come true or not come true?”
There’s no magic; the protection from fooling ourselves is the same as it has always been, only the names of the problems change.

Attempt at Understanding Blah Blah Blah

“The trouble comes when empiricism is combined with a hypothesis that the climate is stationary, which is implicit in how many of their analyses work.” [8]
The irony of this statement is extraordinary in light of all the criticisms by the auditors and others of statistical methods in climate science. It would be a valid criticism, if it were supported.
“The empiricist view has never entirely faded from climatology, as, I think, we see from Curry. But it’s essentially useless in examining climate change. Under its precepts, the only thing that is predictable is stasis. Once things start changing, empirical science closes the books and goes home. At that point you need to bring some physics into your reasoning.” [2]
So we’ve gone from what could be reasonable criticism of unfounded assumptions of stationarity to empiricism being unable to explain or understand dynamics. I guess the guys working on embedding dimension stuff, or analogy based predictions would be interested to know that.
“See, empiricism lacks consilience. When the science moves in a particular direction, they have nothing to offer. They can only read their tea leaves. Empiricists live in a world which is all correlation, and no causation.” [3]
Lets try some definitions.
empiricism
knowledge through observation
consilience
unity of knowledge, non-contradiction

How can the observations contradict each other? Maybe a particular explanation for a set of observations is not consilient with another explanation for a different set of observations. This seems to be something that would get straightened out in short order though: it’s on this frontier that scientific work proceeds. I’m not sure how empiricism is “all correlation.” This is just a bald assertion with no support.
“While empiricism is an insufficient model for science, while not everything reduces to statistics, empiricism offers cover for a certain kind of pseudo-scientific denialism. [...] This is Watts Up technique asea; the measurements are uncertain; therefore they might as well not exist; therefore there is no cause for concern!” [4]
Tobis: Empiricism is an insufficient model for science. Feynman: The test of all knowledge is experiment. Tobis: Not everything reduces to statistics. Jaynes: Probability theory is the logic of science. To be fair, Feynman does go on to say that you need imagination to think up things to test in your experiments, but I’m not sure that isn’t included in empiricism. Maybe it isn’t included in the empiricism Tobis is talking about.
So that’s what all this is about? You’re upset at Watts making a fallacious argument about uncertainty? What does empiricism have to do with this? It would be simple enough to just point out that uncertainty doesn’t mean ignorance.

Not quite blah blah blah, but the argument is still hardly thought out and poorly supported.

References

[1] Easterbrook, S., “Climate Science is an Experimental Science,”http://www.easterbrook.ca/steve/?p=1322, February 2010.

[2] Tobis, M., “The Empiricist Fallacy,” http://initforthegold.blogspot.com/2010/11/empiricist-fallacy.html, November 2010.

[3] Tobis, M., “Empiricism as a Job,”http://initforthegold.blogspot.com/2010/11/empiricism-as-job.html, November 2010.

[4] Tobis, M., “Pseudo-Empiricism and Denialism,”http://initforthegold.blogspot.com/2010/11/pseudo-empiricism-and-denialism.html, November 2010.

[5] Thacker, B. H., Doebling, S. W., Hemez, F. M., Anderson, M. C., Pepin, J. E., and Rodriguez, E. A., “Concepts of Model Verification and Validation,” Tech. Rep. LA-14167-MS, Los Alamos National Laboratory, Oct 2004.

[6] Heald, M. and Wharton, C., Plasma Diagnostics with Microwaves, Wiley series in plasma physics, Wiley, New York, 1965.

[7] Easterbrook, S., “Validating Climate Models,”http://www.easterbrook.ca/steve/?p=2032, November 2010.

[8] Tobis, M., “Empiricism,”http://initforthegold.blogspot.com/2010/11/empiricism.html, November 2010.

Thanks to George Crews and Dan Hughes for their critical feedback on portions of this.

[Update: George left a comment with suggestions on changing the flowchart. Here's my take on his suggested changes.

A slightly modified version of George's chart. I think it makes more sense to have the "No" branch of the validation decision point back at "Abstraction", which parallels the "No" branch of the verification decision pointing at "Implementation". Also switched around "Experimental Data" and "Experimental Diagnostic." Notably absent is any loop for "Calibration"; this would properly be a separate loop with output feeding in to "Computer Model."

]

Mathematical Science Foundations of Validation, Verification, and Uncertainty Quantification

2011-01-08T17:08:00.003-05:00

There is an interesting study being conducted by the National Academies. Here's their "Statement of Task":

A committee of the NRC will examine practices for verification and validation (V&V) and uncertainty quantification (UQ) of large-scale computational simulations in several research communities.
Identify common concepts, terms, approaches, tools, and best practices of V&V and UQ.
Identify mathematical sciences research needed to establish a foundation for building a science of V&V and for improving practice of V&V and UQ.
Recommend educational changes needed in the mathematical sciences community and mathematical sciences education needed by other scientific communities to most effectively use V&V and UQ.

Here's a list of the folks on the committee. It should be interesting to see the study results (it's an 18 month long effort).

Dayton Diode

2010-12-28T23:34:00.000-05:00

Looks like Dayton has a hackerspace now: Dayton Diode Space. Tell your favorite south-western Ohio hacker!

TSA Screening, Terminal 2, SAN, Nov. 13, 2010

2010-11-16T08:53:00.005-05:00

"If you touch my junk, I'll have you arrested."

TSA's security theater is an emerging threat to our Constitutional Republic.

Grafton Hill Pumpkin Display

2010-11-02T16:54:00.001-04:00

Thanks to Judy Chaffin for making a great pumpkin display happen.

My favorite was Casper the Friendly Ghost.It was great to see all the folks out and about in Grafton Hill enjoying the brisk evening air and beautiful display.

NALPAL: Not A Livermore Physics Applications Language

2010-10-20T11:44:00.007-04:00

Background and Motivation

The problem I am interested in: leverage high-level symbol manipulation capabilities in a computer algebra system (CAS) to generate compile-able code for number crunching. More specifically, I am interested in generating routines for numerical partial differential equation (PDE) solvers. This is not a new idea, significant work was accomplished starting in the early 1980s [1, 2, 3], continued into the late ’80s [4, 5, 6], and early ’90s [7]. However, those capabilities have not been consolidated and formally incorporated into my open-source CAS of choice, Maxima (though there is fairly recent work implementing similar ideas for Mathematica [8, 9]). The early work [1] quotes Hamming’s Turing Award speech [10] for motivation:

We seem not to be able to use the machine, which we all believe is a very powerful tool for manipulating and transforming information, to do our own tasks in this very field. We have compilers, assemblers, monitors, etc. for others, and yet when I examine what the typical software person does, I am often appalled at how little he uses the machine in his own work.

I had been thinking about this problem for a while because of my interest in method of manufactured solutions (MMS) for code verification. I thought things were really going to take off when I came across Cook’s 1990 technical report describing code generation for PDE solvers using A Livermore Physics Applications Language (ALPAL), which was built on top of DOE-MACSYSMA. ALPAL aimed to be a general purpose domain specific language for turning symbolic descriptions of PDEs into code. Here’s a solution to the problem already developed! Of course, it couldn’t be that easy. I asked the Maxima mailing list if anyone knew what became of this effort: ALPAL question to Maxima list, and ended up answering my own question by getting a hold of some of the old ALPAL devs: ALPAL answer to the list. Unfortunately, there isa long history behind the divergence between different versions of MACSYMA (and different groups of developers) that mitigates against this old code ever working with the fork that became Maxima (should an archive of the source ever actually turn up, update: it did, see comment)[11, 12, 13].

As you may have guessed from this post’s title, and the difficulties described in the previous paragraph, I’ve decided to pursue a more toolbox approach rather than (re)implementing a domain specific language and associated decision logic. The working title for my toolbox of utilities will be NALPAL, both as a nod to the valuable historical work in this area, and an indication of the path not taken.

Wirth categorizes three main approaches to systems for numerical PDE solution: black-box systems, subroutine packages, and code generation aids [1]. Black-box systems are intended for the novice user, and tend to be constrained to very specific problem types or incorporate a great deal of decision logic. Subroutine packages provide reusable building blocks, but require more knowledge and coding on the part of the user, and tend to remain problem specific. Code generation aids require the user to be an expert in numerical analysis, and tend not to automate any of the analytical work that precedes the code generation step.

Wirth’s suggested approach is a Unix-like toolbox approach where standard subroutine libraries are used when it makes sense to do so, and utilities for doing custom code generation are written in the CAS environment. The alternative is to create a full language, and incorporate the semantics and decision logic to automate the entire process for the novice user. The work of Wirth, Steinberg and Roache is a good example of the former approach, and Cook’s work on ALPAL is an example of the latter (though his early work [3] is more along the lines of the toolbox approach, going so far as to say “the viewpoint taken is that one should be able to start with integro-differential equations and end up with optimal FORTRAN code segments. The key in achieving this goal was to tackle only the specific algebraic problem-i at hand, and not to attempt to provide tools to cover every conceivable numerical scheme.”). The toolbox approach keeps more of the work for the problem solver in set-up/tear-down/customization, while the language approach loads more of the burden on to the domain specific language developer. Wirth’s second objective sums up well the approach that I think makes the most sense (emphasis original):

Build productivity enhancing tools of broad applicability for the expert user so that efficient, special purpose PDE codes can be built reliably and quickly, rather than attempt to second guess the expert and build general purpose PDE codes (black box systems) of doubtful efficiency and reliability.

There are still useful things to learn from ALPAL even if we have chosen the less ambitious road, since the problem being solved is the same. A basic description of the ALPAL use-case[7]:

Take as input a PDE description, along with boundary and initial condition definitions
Discretize the PDE
Analyze the result (e.g. for stability)
Calculate the Jacobian (needed for Newton methods in implicit time-integration or non-linear boundary value problem (BVP)s)
Generate code

Even if we don’t have a full-featured language, the user of our set of utilities will still be interested in accomplishing largely these same steps. In fact, Wirth’s early work gives these steps (reproduced here verbatim) [1]:

Manipulate the set of partial differential equations to cast them into a form that is amenable to numerical solution. For vector PDEs, this might include vector differential calculus operations and reexpression in scalar (component) form, and the application of a linearization approximation for non-linear PDEs.
Discretize the time and space domain, and transform the partial differential operators in the PDEs into finite difference operators. This transforms the partial differential equations into a set of algebraic equations. A multitude of possible transformations for the differential operators are possible and the boundary conditions for the PDEs also must be appropriately handled. The resulting difference equations must be analyzed to see if they form an accurate and numerically stable approximation of the original equation set. For real world problems, this analysis is usually difficult and often intractable.
After choosing a solution algorithm from numerical linear algebra, the finite difference equations and boundary conditions are coded in a programing language such as FORTRAN.
The numerical algorithm is then integrated with code for file manipulations, operating system interactions, graphics output, etc. forming a complete computer program.
The production program is then executed, and its output is analyzed, either in the form of numerical listings or computer-generated graphics.

Wirth goes on to say (emphasis added),

With continuing advances in computer technology, the last step in this process has become easier. For a given class of problems, answers can be calculated more quickly and economically. More importantly, harder problems which require more computational resources can be solved. But the first four steps have not yet benefited from advances in computer performance; in fact, they are aggravated by it.

Also, this additional bit of motivation from Chapter 5,

Taken together with the software described in other chapters, these tools allow the user to quickly generate a FORTRAN code, run numerical experiments, and discard the code without remorse if the numerical results are unsatisfactory.

This is similar in sentiment to the idea of numerical throw away code.

PDE Code Gen Recipes

With the problem background and motivation set, the rest of this post will focus on pulling out useful Maxima recipes demonstrated by folks who have generated FORTRAN from MACSYMA with intent to inflict grievous arithmurgical damage on hapless PDEs.

Much work is done in most of these articles from the 1980s to avoid array references and break up expressions by hand to reduce the computational cost in MACSYMA. Also, MACSYMA’s knowledge of the chain rule is not used for calculating the transformations because of an explosion in the number of terms [5], rather an identity for the derivative of a matrix inverse is used. A lot of this effort seems unnecessary today because speed and memory have improved so much. However, the basic approach and variables used to define the problem is still relevant (compare with this example of Burgers’ equation on a curvilinear grid):

  dep : [f, sigma]; /* the dependent variables */
  curvi : [xi, eta, zeta]; /* the curvilinear coordinates */
  indep : [x, y, z]; /* the independent variables */
  depends(curvi, indep);
  depends(dep, curvi);
  nn : length(indep);
  eqn : sum(diff(sigma * diff(f, indep[i]), indep[i]), i, 1, nn);

The result is rather large, but it illustrates Maxima’s knowledge of the chain rule. Of course, generally it is easiest to compute rather than for an arbitrary grid, so you need to make substitutions based on the inverse of the Jacobian of transformation. In Maxima we might do something like

  J : zeromatrix(3,3);
  for j : 1 thru 3 do (
    for i : 1 thru 3 do (
      J[i,j] : ’diff(indep[j],curvi[i])
    )
  );
  K : zeromatrix(3,3);
  for j : 1 thru 3 do (
    for i : 1 thru 3 do (
      K[i,j] : diff(curvi[j], indep[i])
    )
  );
  grid_trans_subs : matrixmap(”=”, K, invert(J));
  /* making substitutions from a list is easier than from a matrix */
  grid_trans_sublis : flatten(makelist(grid_trans_subs[i],i,1,3));

which gives us a list of nine equations we can use to make substitutions so that all our derivatives are with respect to the computational coordinates.

  trans_eqn : subst(grid_trans_sublis, eqn) $
  /* Evaluation took 0.0510 seconds (0.0553 elapsed) using 265.164 KB. */
  trans_eqn_factor : factor(trans_eqn) $
  /* Evaluation took 2.4486 seconds (2.5040 elapsed) using 48.777 MB. */

Factoring the result starts getting on up there in compute time and memory, but still not untractable, or even that uncomfortable for an interactive session. Of course we still haven’t made any difference expression substitutions, and that will expand the number of terms even further. The assumption that the grid is arbitrary is a decision point that would have to be dealt with in a black-box style implementation. Best to leave it to the (hopefully) knowledgeable user.

As an aside, linearization is another example of an assumption that is probably best left to the user. Wirth assumes that a linearization is required to solve nonlinear PDEs, whereas Cook provides for calculating the system Jacobians needed for Newton methods (of course Wirth does note that this is just one approach that could be used, and the tools are flexible enough to be extended to other methods). Using the Jacobian is a more modern approach made practical by the successful development of Newton-Krylov methods. It is impossible to predict the future development of numerical methods that will change the choices that need to be made in discretizing PDEs (though that doesn’t prevent speculation [14]), so a flexible toolbox approach is again indicated.

Once a PDE is defined substitutions of finite differences for partial derivatives must be made to create the discrete approximation. Wirth uses a function called DISCRETIZE which uses the dependencies list (created by a call to depends) to substitute indexed variables for the independent variables. Then the partial derivatives of the indexed variables are replaced by finite difference operators. The substitutions of difference expressions is controlled by using Maxima’s pattern matching rules. The basic recipe is

Define dependencies between independent and dependent (and possibly computational coordinates)
Associate a list if indices with the coordinates
Define rules that transform differential terms into difference terms with the appropriate index shifts and constant multipliers corresponding to the coordinate which the derivative is with respect to and the selected finite difference expression
Apply the rules to the PDE to give a finite difference equation (FDE)
Use Maxima’s simplification and factoring capabilities to simplify the FDE
Output the FDE in FORTRAN format and wrap with subroutine boilerplate using text processing macros

The boilerplate is a set of standard templates for a particular solution procedure. The example Wirth uses is an alternating direction implicit (ADI) scheme. A more modern example might be a Newton-Krylov scheme. Wirth describes an environment, Fast FORTRAN Programing (FFP), whose goal is to move computational physics out of a “cottage industry” state. He describes it thusly, “The system consists of two major components: a subroutine library and a command language for building, compiling and running codes.” Based on my experience, that sounds a whole lot like Scipy, which is built on top of Python, and packages together many of the classic scientific computing libraries.

Pattern matching in Maxima is relatively straight-forward. For instance, say I have a function that I’d like to use to calculate my derivatives (such as one based on the discrete cosine transform (DCT)), I could declare patterns that replaced derivatives with function calls.

  matchdeclare([fmatch,xmatch,nmatch],all),
  defrule(deriv_subst_1, ’diff(fmatch,xmatch,1), diff_1(fmatch,xmatch)),
  defrule(deriv_subst_2, ’diff(fmatch,xmatch,2), diff_2(fmatch,xmatch)),
  mat_expr : apply1(mat_expr, deriv_subst_1),
  mat_expr : apply1(mat_expr, deriv_subst_2),

This would result in all of the first and second derivatives in mat_expr being replaced by function calls diff_1 and diff_2 respectively.

The initial set-up steps Cook uses in [3] are roughly the same as those used by Wirth, with the addition of a constant list. The focus is more on what to do with the discrete expressions once they are generated. The basic recipe is

reduce_cnst()
gcfac()
optimize()
fortran()

The constant list allows reduce_cnst to pull constants out of loops, and optimize pulls out common sub-expressions to speed things up. optimize returns a Maxima block, which is similar to a Fortran subroutine. In fact, turning blocks into subroutines in other languages is a common problem which has been addressed on the Maxima mailing list (e.g.block2c, see also cform), and as Dan Stanger points out, this is the purpose of the GENTRAN package. GENTRAN is not yet successfully ported to work with Maxima [13] (though there is a gentran directory that ships in the Maxima source tarball if you want to take a look).

The only reason Cook went to the Lisp level in [3] was to reduce the cost of the factor routines (which is less of a concern now), and to make optimize work with lists of expressions (which it does now). One of the examples cites a need for an outrageous 500 million words of computer memory for one calculation, but that’s roughly half what’s in my old desktop computer I got used about a year ago (for about $100). The computing power available to the average amateur has come a long way since 1982.

Both Wirth and Cook assume that an arbitrary orthogonal coordinate system (like spherical, polar, cylindrical, Cartesian, etc.) will be defined. A slightly more modern approach is presented by Roache et al. [4, 5, 6]. They assume an arbitrary curvilinear coordinate system, which may not have analytical scale factors and metrics (i.e. they include the numerical grid generation task as part of the problem statement / solution procedure).

The approach demonstrated in [5] focuses on transforming a set of PDEs to arbitrary curvilinear coordinates described by Laplace’s equation. This approach couples the PDE solution and grid generation methods together. Modern approaches in the engineering world generally assume that the grid will be generated by a separate tool (the fully coupled approach can still be useful for adaptive or moving grids), though solving problems on a single, canonical grid seems to still be common in the sciences. The MACSYMA routines presented there are

change(): change variables, convert arbitrary second order differential equation in nn variables to an arbitrary coordinate frame in the variables xi[i]
notate: atomic notation for derivatives
notation(exp,vari): primitive atomic notation
scheme(): introduce differences of unknowns
difference(u,f,exp): primitive differences, scheme and difference collect the coefficients of the differences and calculate the stencil of the solver and coordinate transformations
myFORTRAN(): write the FORTRAN code

A lot of this extra effort is avoidable now, because it is tractable to use Maxima’s knowledge of the chain rule, and the built-in indexed variables and pattern matching facilities.

Conclusion

After digging in to the old literature on generating code from MACSYMA, and trying out a few things with current (v5.20.1) Maxima, it seems like all the pieces you need to generate PDE solving code already ship with Maxima (not touched on in this post were the vector calculus and tensor capabilities that Maxima has as well). I kind of already knew this since I'd been generating solver code in an ad hoc sort of way already. Perhaps there is a place for building up a pattern matching rules database, and maybe boilerplate templates. That seems to be the area that the modern efforts are focused on (e.g. Scinapse claims that it’s rules encoding PDE solution knowledge make up half it’s 120kloc and is the fastest growing part of the code-base [9]). The recipes presented here seem more like a documentation, or tutorial item rather than a possible new Maxima package.

References

[1] Wirth, M. C., On the Automation of Computational Physics, Ph.D. thesis, University of California, Davis, 1980.

[2] Wirth, M. C., “Automatic generation of finite difference equations and fourier stability analyses,” SYMSAC ’81: Proceedings of the fourth ACM symposium on Symbolic and algebraic computation, ACM, New York, NY, USA, 1981, pp. 73–78.

[3] Cook, G. O., Development of a Magnetohydrodynamic Code for Axisymmetric, High-β Plasmas with Complex Magnetic Fields, Ph.D. thesis, Brigham Young University, December 1982.

[4] Florence, M., Steinberg, S., and Roache, P., “Generating subroutine codes with MACSYMA,” Mathematical and Computer Modelling, Vol. 11, 1988, pp. 1107 – 1111.

[5] Steinberg, S. and Roache, P. J., “Symbolic manipulation and computational fluid dynamics,” Journal of Computational Physics, Vol. 57, No. 2, 1985, pp. 251 – 284.

[6] Steinber, S. and Roache, P., “Using VAXIMA to Write FORTRAN Code,” Applications of Computer Algebra, edited by R. Pavelle, Kulwer Academic Publishers, August 1984, pp. 74–94.

[7] Cook, G. O., “Construction of large-scale simulation codes using ALPAL (A Livermore Physics Applications Language),” Technical Report UCRL-102469, Lawrence Livermore National Labs, October 1990.

[8] Husa, S., Hinder, I., and Lechner, C., “Kranc: a Mathematica package to generate numerical codes for tensorial evolution equations,” Computer Physics Communications, Vol. 174, No. 12, 2006, pp. 983 – 1004.

[9] Akers, R. L., Kant, E., Randall, C. J., Steinberg, S., and Young, R. L., Enabling Technologies for Computational Science, Vol. 548 of The Springer International Series in Engineering and Computer Science, chap. SCINAPSE: A problem solving environment for partial differential equations, Springer, 2000, pp. 109–122.

[10] Hamming, R. W., “One Man’s View of Computer Science,” Journal of the Association for Computing Machinery, Vol. 16, No. 1, January 1969, pp. 3–12.

[11] Cook, G. O., electronic mail communication, August 2010.

[12] Fateman, R. J., electronic mail communication, August 2010.

[13] Stanger, D., electronic mail communication, August 2010.

[14] Houstis, E. N. and Rice, J. R., “Future problem solving environments for computational science,” Mathematics and Computers in Simulation, Vol. 54, No. 4-5, 2000, pp. 243 – 257.

A little bit of Dayton in Paris

2010-10-14T09:07:00.003-04:00

Found this in the Musée de l'Orangerie (whose main attraction is the Water Lilies).

Henri-Julien Félix Rousseau - Les Pêcheurs à la ligne
[1908 - 1909]

Looks like an early version of the Wright's glider with the fixed double rudder. The French Aero club at the time was trying to replicate their early gliding results. The brothers made a trip to France in 1908 to show off the Flyer (powered, single steerable rudder), which really cemented their fame (previously the Europeans had been broadly and aggressively dismissive of their claims).
This is another case where the hardware preceded full understanding of the physics. The Wrights found in their bicycle, and subsequent wind tunnel, tests that the published empirical constants in the theoretical lift equations were pretty far off.

Just because the tower is cool (and GIMP is cool as well):

Only Opportunity

2010-09-27T16:33:00.006-04:00

Happened upon this

The constraints imposed by the planetary ecosystem require continuous adjustment and permanent adaptation. Predictive skills are of secondary importance.
Hendrik Tennekes

and thought of this

There is no security on this earth; there is only opportunity.
General Douglas MacArthur

Tennekes concludes with

From my background in turbulence I look forward with grim anticipation to the day that climate models will run with a horizontal resolution of less than a kilometer. The horrible predictability problems of turbulent flows then will descend on climate science with a vengeance.

I have his book; it is rather good.

Goodbye Blue Sky

2010-09-24T11:34:00.002-04:00

Look mummy, there's an airplane up in the sky

over Rotary Park in Beavercreek.

Click this one to watch the prop go 'round:

Fountains from Riverscape

2010-08-13T13:30:00.000-04:00

No Fluid Dynamicist Kings in Flight-Test

2010-08-03T13:33:00.000-04:00

This was a guest post over on Pielke's site.

Dr Pielke's Honest Broker concepts resonate with me because of practical decision support experiences I've had, and this post is an attempt to share some of those from a realm pretty far removed from the geosciences. All the views and opinions expressed are my own and in no way represent the position or policy of the US Air Force, Department of Defense or US Government. I am writing as a simple student of good decision making. My background is not climate science. I am an Aeronautical Engineer with a background in computational fluid dynamics, flight test and weapons development. I got interested in the discussions of climate policy because the intersection of computational physics and decision making under uncertainty is an interesting one no matter what the subject area. The discussion in this area is much more public than the ones I'm accustomed to, so it makes a great target of opportunity. The decision support concepts Dr Pielke discusses make so much sense to me now, but I can see how hard they are for technical folks to grasp because I used to be a very linear thinker when I was a young engineer right out of school.

My journeyman's education in decision support came when I got the chance to lead a small team doing Live Fire Test and Evaluation for the Air Force (you may not be familiar with LFT&E, it is a requirement that grew out of the Army gaming testing of the Bradley fighting vehicle in the 1980s, a situation that was fairly accurately lampooned in the movie "Pentagon Wars"). The competing values of the different stakeholders (folks appointed by congress to ensure sufficient realistic testing compared to folks at the service level doing product development) was really an eye-opening education for a technical nerd like me. I initially thought, "if only everyone can agree on the facts, the proper course of action will be clear". How naive I was! Thankfully, the very experienced fellows working for me didn't mind training up a rash, newly-minted, young Captain.

It's tough for some technical specialists (engineers/scientists) to recognize worthy objectives their field of study doesn't encompass. The reaction I see from the more technically oriented folks like Tobis (see how he struggles) reminds me a lot of the reaction that engineers in product development offices would have to the role of my little Live Fire office. A difficulty we often encountered was the LFT&E oversight folks wanted to accomplish testing that didn't have direct payoff to narrower product development goals that concerned the engineers. "What those people want to do is wasteful and stupid!" This parallels the recent sand berm example. The preferred explanation from the technician's perspective is that the other guy is bat-shit crazy, and his views should be ridiculed and de-legitimized. The truth is usually closer to the other guy having different objectives that aren't contained within the realm of the technician's expertise. In fact, the other person is probably being quite rational, given their priors, utility function and state of knowledge.

In my little Live Fire Office we had lots of discussion about what to call the role we did, and how to best explain it to the program managers. I wish I had heard of Dr Pielke's book back then, because "Honest Broker" would have been an apt description for much of the role. We acted as a broker between the folks in the Pentagon with the mandate from congress for sufficient, realistic testing, and the Air Force level program office with the mandate for product development. The value we brought (as we saw it), was that we were separate from the direct program office chain of command (so we weren't advocates for their position), but we understood the technical details of the particular system, and we also understood the differing values of the folks in the Pentagon (which the folks in the program office loved to refuse to acknowledge as legitimate, sound familiar?). That position turns out to be a tough sell (program managers get offended if you seem to imply they are dishonest), so I can empathize with the virulent reaction Dr Pielke gets on applying the Honest Broker concepts to climate policy decision support. People love to take offense over their honor. That's a difficult snare to avoid while you try to make clear that, while there's nothing dishonest about advocacy, there remains significant value in honest brokering. Maybe Honest Broker wouldn't be the best title to assume though. The first reaction out of a tight-fisted program manager would likely be "I'm honest, why do I need you?"

One of the reason my little office existed was because of some "lessons learned" from the Tri-Service Standoff Missile debacle (all good things in defense acquisition must grow out of historical buffoonery). The broader Air Force leadership realized that it was counterproductive to have product development engineers and program managers constantly trying to de-legitimize the different values that the oversight stake-holders brought (the differences springing largely from different appetites for risk and priors for deception) by wrangling over largely inconsequential, technical nits (like tree rings in the Climate Wars). The wiser approach was to maintain an expertise whose sole job was to recognize and understand the legitimate concerns of the oversight folks and incorporate those into a decision that meets the service's constraints as quickly and efficiently as possible. Rather than wasting time arguing, product development folks could focus on product development.

The other area where I've seen this dynamic play out is in making flight test decisions. In that case though, the values of all the stake-holders tend to align more closely, so the separation between technical expertise and decision making is less contentious (Dr Pielke's Tornado analogy). In contrast to the climate realm where it's argued that science compels because we're in the Tornado mode, the flight-test engineers understand that the boss is taking personal responsibility for putting lives at risk based on their analysis. They tend to be respectful of their crucial, but limited, role in the broader risk management process. Computational fluid dynamics can't tell us if it's worth risking the life of an air crew to collect that flight test data. In that case there is no confusion about who is king, and over what questions the technical expert must "pass over in silence."

Ohio Aerospace Hub Road Project

2010-06-14T18:43:00.000-04:00

Here's a map of the recently announced ~~pork~~ road development project to support our innovation hub:
View Ohio Aerospace Hub Roadwork Development Grant in a larger map

I'm sure this will be welcome by the retailers on Brown St (now that they've successfully shuffled off their vagrant problem).

Falcon 9: Lift Off!

2010-06-04T15:01:00.005-04:00

Falcon 9 launch successful on first test flight. Wow!

First Stage In Flight

Successful Stage Separation

Second Stage Burn

Yeah, yeah, it's just a test. But the cost of the entire development program (~$335M) for Falcon 9 is the same as a single test flight for Ares I-X (~$445M)...

From my perspective (as someone who's sat in the hot seat conducting flight tests), the really impressive thing with the SpaceX operation was their ability to light the engines, auto-abort, and turn a new countdown/launch at the end of their range time. There was clearly a whole lot of work in the design phase leading up to the impressive execution today that made saving the mission possible. Nice.

I Still Like Ike

2010-05-17T13:19:00.002-04:00

Free Concert at Masonic Center

2010-05-10T10:25:00.000-04:00

The Miami Valley Symphony Orchestra is putting on a free show at the Masonic Center in Grafton Hill on Sunday, May 16th at 3 p.m.

Updatyon Summit: Struggles of an Organization Town

2010-04-18T18:07:00.032-04:00

I recently attended the updayton summit. I'm a newcomer to Dayton and was curious about this unique-sounding gathering. My first impression on hearing the words Young Creatives (YCs) Summit was "what a pretentious sounding group". I went with a list of preconceived questions (culled from this set of notes) that I wanted to try and answer through observation of how the summit was conducted and how the participants evolved their involvement. My initial idea was to look for evidence supporting either asynergistic group process or emergent individual creativity at thesummit. I note my impressions and preconceptions upfront so thatyou'll understand my observations are not disinterested, though I triedto be 'minimally involved' and objective (with only limited success,the participants and facilitators were really nice and their optimismwas infectious). If you think I missed something significant pleasepoint it out in the comments.

See Me? I'm behind the bald guy.

The updayton summit's main goal is to come up with annual projects that serve to excite YCsand will in-turn help Dayton retain recent college graduates. Themeans used to achieve this goal were facilitated consensus generationand voting. Attendees were split up by self-selected interestcategories, and then further into several sub-groups. My interestcategory was entrepreneur. These sub-groups met independently at the beginning of the summit in breakout sessionsto generate ideas and vote on their 'top two' options for projects. After that, the results of all the sub-groups were collected by summitstaff. The attendees then went to panel discussions in their interest category, and then all met together in a Town Hall to vote on the final projects for the coming year.

Therewere seven resulting top ideas in the entrepreneur interest category. Six of these consisted primarily of websites. Five of those websiteswere about mentoring for young entrepreneurs by established ones,entrepreneur support groups, information clearinghouses or somecombination thereof. Consensus building is certainly brutal in seekingout the lowest common denominator. How to argue againstsomething as innocuous and pervasive as a website in our networkedage? And what were the big success stories out of last year's summit? Weeding and painting and, you guessed it, a web resource. Remember, these groups met independently at the beginning of thesummit. No significant prior communication between sub-group membersother than mingling at vendor booths while surfing swag.

Howdoes what I observed in the summit breakout sessions look in light ofestablished thoughts on creativity (or lack thereof) in groups? Thereare two superficially competing views of the group creative process. There is the whole-is-more-than-the-sum-of-parts school popularized byStephen Covey. In opposition, is the creative-acts-are-individual-actsschool. William Whyte's The Organization Manprovides an extensive denial of useful creative genius in groups, and acall for renewed focus on the dignity and efficacy of the individualcontribution. The former I'll call the Synergy School, the latter theSolitary School.

Here's part of Whyte's criticism of the consensus building group,

Thinkfor a moment of the way you behave in a committee meeting. In yourcapacity as group member you feel a strong impulse to seek commonground with the others. Not just out of timidity but out of respectfor the sense of the meeting you tend to soft-pedal that which would goagainst the grain. And that, unfortunately, can include unorthodoxideas. A really new idea affronts current agreement -- it wouldn't bea new idea if it didn't -- and the group, impelled as it is toagreement, is instinctively hostile to that which is divisive. Withwise leadership it can offset this bias, but the essential urge willstill be to unity, to consensus. After an idea matures -- after peoplelearn to live with it -- the group may approve it, but that is afterthe fact and it is an act of acquiescence rather than creation.

Ihave been citing the decision-making group, and it can be argued thatthese defects of order do not apply to information-exchanging groups. It is true that meeting with those of common interests can betremendously stimulating and suggest to the individuals fresh ways ofgoing about their own work. But stimulus is not discovery; it is notthe act of creation. Those who recognize this limitation do notconfuse the functions and, not expecting too much, profit from themeeting of minds.

Others, however, are not so wise, and fastbecoming a fixture of organization life is the meeting self-consciouslydedicated to creating ideas. It is a fraud. Much of suchhigh-pressure creation -- cooking with gas, creating out loud, spitballing, and so forth -- is all very provocative, but if it is stimulating, itis stimulating much like alcohol. After the glow of such a session hasworn off, the residue of ideas usually turns out to be a refreshedcommon denominator that everybody is relieved to agree upon -- and ifthere is a new idea, you usually find that it came from a capital ofideas already thought out -- by individuals -- and perhaps held inescrow until someone sensed an opportune moment for its introduction.
Togetherness

The scientific conference exemplifies Whyte's informational exchangemeeting. No one attends to make decisions (or vote with dots), theattendees are looking to share their work, and learn about theircolleagues' work. These meetings are an important part of modernscientific progress. This years' updayton meeting did have information exchange components, which I'll get to later.

The Synergy School might argue that the updaytonbreakout sessions can provide an opportunity for synergisticcollaboration, where alternative solutions emerge that are better thanany of the individual solutions brought by group members. The SynergySchool's three levels of communication are

Thelowest level of communication coming out of low trust situations ischaracterized by defensiveness, protectiveness, and legalistic languagewhich covers all the bases and spells out qualifiers and escape clausesin the event things go sour.
The middle level of communication is respectful communication -- where fairly mature people communicate.
The highest level of communication is synergistic (win/win) communication.

However, the acknowledged goal of the updaytonsummit is 'stimulating like alcohol' and 'engagement' rather thansynergy. So, while I went looking for evidence of high-levelcooperative action I should have paid closer attention to the marketingmaterials and lowered my expectations accordingly. There was no effortat establishing group trust (we didn't even introduce ourselves at thestart of the breakout). We jumped right in to the scripted consensusprocess. Low-trust communication among mature professionals leading tocompromise (consensus) is the best we can hope for from events likethese, and, unsurprisingly, that's exactly what we got. This naturallyraises the question: why bother? If all we can reasonably hope for issecond tier communication then why invest the effort? The gist I getfrom a closer look at the promotional material is 'to get buy-in forthe projects which will excite YCs to stay'. Which, in a moment ofcynicism, might strike one as rather manipulative. Instead of manufacturingradiators, now we're manufacturing community-spiritedness. We might not be able to offer you gainful employment, but you can volunteer to weed our sidewalks!

Insupport of the Solitary School's idea about the capital of individualideas, the winning project from the entrepreneur interest category was the one option that wasn't a website. The young man whose idea formed the core of this project said, "this issomething I've been writing about for years". Something he was clearlypassionate about, something that he expended his individual creativeeffort to flesh out beforehand on his own, and subsequently pitch tothe group. The other options presented by the members of the groupwere relentlessly mashed into web-sameness by the gentle actions of thefacilitators and the listless shrugs of individual acquiescence fromwell-meaning group members searching for common ground. Whena thoughtful member of the breakout session asked the only reallyimportant question, "how do you create an innovator?" His question wasmet with more shrugs around the room followed quickly by redirectionfrom the facilitators. Clearly that question cannot be packaged into apublic relations project.

What about the skillssessions? Surely these have redeeming aspects, the Solitary Schoolwould appreciate these as information exchange, and the Synergy Schoolmight appreciate them as 'sharpening the saw'. The most interesting aspect of the panel discussions was the incipientfrustration I observed in some of David Gasper's comments. Roughly,"there are so many great resources for entrepreneurs in the Daytonregion. Why don't we have more entrepreneurs!? Dayton needs moreentrepreneurs." Some of the resources mentioned by the panelists were Dayton SCORE, EntrepenuerOhio and Dayton Business Resource Center. As Theresa Gasper observed,"People seem to want the information PUSHED to them, but then feeloverwhelmed with all the information coming at them. No one seems towant to PULL the information – meaning, many don't want to search forthe info." This is consistent with the majority of "needs" identifiedin the entrepreneur breakout sessions. These folks are looking forchecklists, guarantees of stability and someone to tell them what todo. In fact, one participant in my session thought that the biggestbarrier to entry for entrepreneurs was the lack of the safety netoffered by nationalized health-care! If you were to ask me what is theopposite of the entrepreneurial spirit, I could not have come upwith a better answer. Probably the opposite of the definitions thepanel members gave of entrepreneur too:

some one who has put something of value to them at risk
some one with significant "skin in the game"

Dayton already has a tough time with entrepreneurial thinking because of its recent history as a factory town (far removed from the celebrated, early-industrial "great men"). In his article on a New Era of Joblessness,Don Peck identifies psychological work that points to an additionalgenerational component contributing to this dearth of entrepreneurs,

Many of today’s young adults seem temperamentally unprepared for the circumstances in which they now find themselves. Jean Twenge,an associate professor of psychology at San Diego State University, hascarefully compared the attitudes of today’s young adults to those ofprevious generations when they were the same age. Using national surveydata, she’s found that to an unprecedented degree, people who graduatedfrom high school in the 2000s dislike the idea of work for work’s sake,and expect jobs and career to be tailored to their interests andlifestyle. Yet they also have much higher material expectations thanprevious generations, and believe financial success is extremelyimportant. “There’s this idea that, ‘Yeah, I don’t want to work, butI’m still going to get all the stuff I want,’”Twenge told me. “It’s a generation in which every kid has been told, ‘You can be anything you want. You’re special.’”

In her 2006 book, Generation Me, Twenge notes that self-esteem in children began rising sharply around 1980, and hasn’tstopped since. By 1999, according to one survey, 91 percent of teensdescribed themselves as responsible, 74 percent as physicallyattractive, and 79 percent as very intelligent. (More than 40 percentof teens also expected that they would be earning $75,000 a year ormore by age 30; the median salary made by a 30-year-old was $27,000that year.)Twenge attributes the shift to broad changes in parentingstyles and teaching methods, in response to the growing belief thatchildren should always feel good about themselves, no matter what. Asthe years have passed, efforts to boost self-esteem—and to decouple itfrom performance—have become widespread.

Theseefforts have succeeded in making today’s youth more confident andindividualistic. But that may not benefit them in adulthood,particularly in this economic environment.Twenge writes that“self-esteem without basis encourages laziness rather than hard work,”and that “the ability to persevere and keep going” is “a much betterpredictor of life outcomes than self-esteem.” She worries that manyyoung people might be inclined to simply give up in this job market.“You’d think if people are more individualistic, they’d be moreindependent,” she told me. “But it’s not really true. There’s anelement of entitlement—they expect people to figure things out forthem.”

Seeking'solutions' which enable this emerging neurosis, rather than healingit, is probably not the answer to a more dynamic Dayton.

Please don't misunderstand my criticisms of this updayton process (or cooperation in general). I am in agreement with both Covey and Whyte that our biggest challenges require innovative cooperation to solve.

Ourmost important work, the problems we hope to solve or the opportunitieswe hope to realize require working and collaborating with other peoplein a high-trust, synergistic way...
Interdependence

Letme admit that I have been talking principally about the adverse aspectsof the group. I would not wish to argue for a destructiverecalcitrance, nor do I wish to undervalue the real progress we havemade in co-operative effort. But to emphasize, in these times, thevirtues of the group is to be supererogatory. Universal organizationtraining, as I will take up in the following chapters, is now availablefor everybody, and it so effectively emphasizes the group spirit thatthere is little danger that inductees will be subverted intorebelliousness.

Overand above the overt praise for the pressures of the group, the veryease, the democratic atmosphere in which organization life is nowconducted makes it all the harder for the individual to justify tohimself a departure from its norm. It would be a mistake to confuseindividualism with antagonism, but the burdens of free thought aresteep enough that we should not saddle ourselves with a guiltyconscience as well.

Togetherness

However,what Dayton lacks towards its success is not more resources fromgovernment, more focus on community, more committee meetings or trendy bohemian culture to attract jobless hipsters. In fact, if the attendance of the updayton summit is any indication, Dayton has no lack of optimistic joiners. However, coddling these agreeable, cooperative, and risk-averse Organization Volk is not the answer if what you are seeking is a flowering of 1000 new entrepreneurs in Dayton. As Whyte argues, we lack a recognition that "[t]he central ideal -- that the individual, rather than society, must be the paramount end [...] is as vital and as applicable today as ever". Lowerthe barriers to entry (taxes / zoning / regulation / governmentsubsidized competitors), and the passionate individuals uninterested inpaternalism will exploit the opportunities that emerge to deliver forDayton's future.

[
The winner of the 'best swag contest' was MetroParks with their D-ring key fob:

Yes. My keys are now a' swingan'...
]

Explosively Formed Projectiles: An Impact of Climate Change

2010-04-14T22:53:00.007-04:00

This new ad campaign is quite terrible. Fear-mongering with future catastrophes is not enough. If we don't pass climate legislation, then it's like we're Killing American Troops.

The part about more powerful improvised explosive devices (IEDs) being used in Iraq, and explosively formed projectiles (EFPs) likely being imported from Iran is accurate. The problem is that you don't have to have precision manufacturing to make pretty darn good devices. In fact, simply formed copper plates and modest amounts of explosive does just fine. So will passing that climate legislation to 'cure our addiction to foreign oil' save anyone from a terrorist's road-side device? Nope. Soldiers will still be in harms way. They'll continue to drive down those same roads, but now they have the additional distinction of appearing as props in climate-politics theater.

2010 Young Creatives Summit: Get the Shirt

2010-04-13T00:41:00.005-04:00

The 2010 Young Creatives Summit is imminent here in the shining Gem City!

You may forget all the folks you meet during the networking and breakout sessions since they'll be plenty of beverages at the after-party, but you'll always have the shirt!

Get yours while there's still time.

Energy and GDP

2010-04-08T17:05:00.000-04:00

More Google Public Data fun.

Ohio Personal Income

2010-03-25T17:45:00.011-04:00

The Dayton Business Journal has a recent article about incomes in Ohio falling less than the national averages. Here's some interactive graphs from Google public data on the topic:

Interesting how DC is such an outlier in these graphs; it's good to be king.

Here's one that's just interesting, not necessarily Dayton, Ohio-centric (you can drag the labels around if it starts out too cluttered):
Rumors of the death of US manufacturing seem greatly exaggerated.

Zen Uncertainty

2010-03-17T14:53:00.001-04:00

Zen Uncertainty: Attempts to understand uncertainty are mere illusions; there is only suffering.
-- WARNING: Physics Envy May Be Hazardous To Your Wealth!

Should we give up? No, there's plenty we can do to make the suffering more bearable. Lo and Mueller give an uncertainty taxonomy of five levels in their 'Physics Envy' paper:

Complete Certainty: the idealized deterministic world
Risk without Uncertainty: an honest casino
Fully Reducible Uncertainty: the odds in the honest casino are not posted, we have to learn them from limited experience
Partially Reducible Uncertainty: we're not quite sure which game at the casino we're playing so we have to learn that as well as the odds based on limited experience
Irreducible Uncertainty: we're not even sure if we're in the casino, we might be outside splashing around in the fountain...

At the bottom of the decent we find level infinity, Zen Uncertainty.

Section 2 of the paper provides a nice historical overview of the early work of Paul A. Samuelson, who single-handedly brought statistical mechanics to the economists, and they have never been the same since. Samuelson acknowledged the deep connection between his work and physics:

Perhaps most relevant of all for the genesis of Foundations, Edwin Bidwell Wil- son (1879–1964) was at Harvard. Wilson was the great Willard Gibbs’s last (and, essentially only) protege at Yale. He was a mathematician, a mathematical physicist, a mathematical statistician, a mathematical economist, a polymath who had done first-class work in many fields of the natural and social sciences. I was perhaps his only disciple . . . I was vaccinated early to understand that economics and physics could share the same formal mathematical theorems (Euler’s theorem on homogeneous functions, Weierstrass’s theorems on constrained maxima, Jacobi determinant identities underlying Le Chatelier reactions, etc.), while still not resting on the same empirical foundations and certainties.

Related to this theme, there's an interesting recent article over on Mobjectivist site about using ideas from physics to model income distributions.

Lo and Mueller propose to operationalize their uncertainty taxonomy with a 2-D checklist (table). The levels provide the columns across the top, and there is a row for each business component of the activity being evaluated, here's their description:

The idea of an uncertainty checklist is straightforward: it is organized as a table whose columns correspond to the five levels of uncertainty of Section 3, and whose rows correspond to all the business components of the activity under consideration. Each entry consists of all aspects of that business component falling into the particular level of uncertainty, and ideally, the individuals and policies responsible for addressing their proper execution and potential failings.

This seems like an idea that could be adapted and combined with best practices for model validation (and checklist sorts of approaches) in helping to define what sorts of uncertainties we are operating under when we make decisions using science-based decision support products.

Their final paragraph echos Lindzen's sentiments about climate science:

While physicists have historically been inspired by mathematical elegance and driven by pure logic, they also rely on the ongoing dialogue between theoretical ideals and experimental evidence. This rational, incremental, and sometimes painstaking debate between idealized quantitative models and harsh empirical realities has led to many breakthroughs in physics, and provides a clear guide for the role and limitations of quantitative methods in financial markets, and the future of finance.
-- WARNING: Physics Envy May Be Hazardous To Your Wealth!

The Social Ethic and Appeals for Technocracy

2010-03-13T22:20:00.004-05:00

Climate activists, in discussing the implementation (or lack thereof) of policy solutions for our 'modern problems', hold forth an interesting combination of ideas about the great need for more of the Social Ethic on the one hand,

If we don't revisit the notion of collective responsibility and sobriety soon, our descendants will pay a heavy price.
-- Michael Tobis

What we lack is an ethical framework for inter-generational responsibilities (such as “pass on a habitable planet to our children”). Cost-benefit analysis avoids these ethical questions, at a time when we desperately need to address them.
-- Steve Easterbrook

and claims of the failure of democracy or public discourse on the other,

They have to see the need for pain, to sense the danger of doing nothing. They have to lead their leaders as well as follow – once they switch off, nothing good happens easily, if at all.
Wanted: an eco prophet

Perhaps we have to accept that there is no simple solution to public disbelief in science. The battle over climate change suggests that the more clearly you spell the problem out, the more you turn people away. If they don’t want to know, nothing and no one will reach them.
-- George Monbiot

Setting aside for now the unsound conflation of scientific insight with political consensus, the sentiment at the base of this meme is just as troubling. It is basically an argument that our old ethical theories and extant systems of governance are incapable of solving the problems, real or perceived, facing modern civilization. W.H. Whyte already wrote the response to this line of thought more ably than I ever could (though his target was mainly the rise of bureaucracy in business, and associated societal changes, his critique seems topical in this case as well).

My charge against the Social Ethic, then, is on precisely the grounds of contemporary usefulness it so venerates. It is not, I submit, suited to the needs of "modern man," but is instead reinforcing precisely that which least needs to be emphasized, and at the expense of that which does. Here is my bill of particulars
It is redundant. In some societies individualism has been carried to such extremes as to endanger the society itself, and there exist today examples of individualism corrupted into a narrow egoism which prevents effective co-operation. This is a danger, there is no question of that. But is it today as pressing a danger as the oberse -- a climate which inhibits individual initiative and imagination, and the courage to exercise it against group opinion? Society is itself an education in the extrovert values, and I think it can be rightfully argued that rarely has there been a society which has preached them so hard. No man is an island unto himself, but how John Donne would writhe to hear how often and for what reasons, the thought is so tiresomely repeated.
It is premature. To preah technique before content, the skills of getting along isolated from why and to what end the getting along is for, does not produce maturity. It produces a sort of permanent prematurity, and this is true not only of the child being taught life adjustment but of the organization man being taught well-roundedness. This is a sterile concept, and those who believe that they have mastered human relations can blind themselves to the true bases of co-operation. People don't co-operate just to co-operate; they co-operate for substantive reasons, to achieve certain goals, and unless these are comprehended the little manipulations for morale, team spirit, and such are fruitless.
And they can be worse than fruitless. Held up as the end-all of organization leadership, the skills of human relations easily tempt the new administrator into the practice of a tyranny more subtle and more pervasive than that which he means to supplant. No one wants to see the old authoritarian return, but at least it could be said of him that what he wanted primarily from you was your sweat. The new man wants your soul.
It is delusory. It is easy to fight obvious tyranny; it is not easy to fight benevolence, and few things are more calculated to rob the individual of his defenses than the idea that his interests and those of society can be wholly compatible. The good society is the one in which they are most compatible, but they can never be completely so, and one who lets The Organization be the judge ultimately sacrifices himself. Like the good society, the good organization encourages individual expression, and many have done so. But there always remains some conflict between individual and The Organization. Is The Organization to be the arbiter? The Organization will look to its own interests, but it will look to the individual's only as The Organization interprets them.
It is static. Organization of itself has no dynamic. The dynamic is in the individual and thus he must not only question how The Organization interprets his interests, he must question how it interprets its own. The bold new plan he feels is necessary, for example. He cannot trust that The Organization will recognize this. Most probably, it will not. It is the nature of a new idea to confound current consensus -- even the mildly new idea. It might be patently in order, but, unfortunately, the group has a vested interest in it miseries as well as its pleasures, and irrational as this may be, many a member of organization life can recall instances where the group clung to known disadvantages rather than risk the anarchies of change.
It is self-destructive. The quest for normalcy, as we have seen in suburbia, is one of the great breeders of neuroses, and the Social Ethic only serves to exacerbate them. What is normalcy? We practice a great mutual deception. Everyone knows that they themselves are different -- that they are shy in company, perhaps, or dislike many things most people seem to like -- but they are not sure that other people are different too. Like the norms of personality testing, they see about them the sum of efforts of people like themselves to seem as normal as others and possibly a little more so. It is hard enough to learn to live with our inadequacies, and we need not make ourselves more miserable by a spurious ideal of middle-class adjustment. Adjustment to what? Nobody really knows -- and the tragedy is that they don't realize that the so-confident-seeming other people don't know either.
[...]
Science and technology do not have to be antithetical to individualism. To hold that they must be antithetical, as many European intellectuals do, is a sort of utopianism in reverse. For a century Europeans projected their dreams into America; now they are projecting their fears, and in so doing they are falling into the very trap they accuse us of. Attributing a power to the machine that we have never felt, they speak of it almost as if it were animistic and had a will of its own over and above the control of man. Thus they see our failures as inevitable, and those few who are consistent enough to pursue the logic of their charge imply that there is no hope to be found except through a retreat to the past.
This is a hopelessly pessimistic view.
The Organization Man

Whyte goes on to dismiss the nostalgic and naive caricature of individualism bandied about by the right, but rather calls for a pragmatic recognition of the natural tension between the individual and society, and that "[t]he central ideal -- that the individual, rather than society, must be the paramount end [...] is as vital and as applicable today as ever", impending climate catastrophes notwithstanding.