tag:blogger.com,1999:blog-5822805028291837738.post3027888245973165053..comments2023-12-09T03:51:33.158-05:00Comments on Various Consequences: Discussion of V and VJoshua Stultshttp://www.blogger.com/profile/03506970399027046387noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-5822805028291837738.post-63573153780639254712011-10-16T15:07:50.497-04:002011-10-16T15:07:50.497-04:00George, I agree with you for the most part. VV&am...George, I agree with you for the most part. VV&UQ, properly understood, is about risk management (or credibility building).<br /><br />For certain codes (numerical PDE solvers for instance), we <i>can</i> say that we've established the correctness of the implementation by demonstrating the design order of convergence. Just because it's intractable to prove the correctness of software in general doesn't mean we can't prove some useful limited or special cases.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-61057599549288312432011-10-15T20:13:55.344-04:002011-10-15T20:13:55.344-04:00Haha, maybe it could be crow-sourced.
Interesting...Haha, maybe it could be <a href="http://en.wikipedia.org/wiki/Project_Pigeon" rel="nofollow">crow-sourced</a>.<br /><br />Interesting: <a href="http://www.slideserve.com/arleen/verification-and-validation-as-applied-epistemology" rel="nofollow">How I Learned to Stop Worrying and Love [the DOE's approach to verifying and validating models of] the Bomb</a>.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-76876165276901832202011-10-11T21:00:09.206-04:002011-10-11T21:00:09.206-04:00Minor nit: crow-sourced is probably crowd-sourced ...Minor nit: crow-sourced is probably crowd-sourced unless there is some kind of corn crop involved. (I'm just catching up on some blogs after spending two weeks in conferences myself.)shallow monkeyhttps://www.blogger.com/profile/13034168320000048613noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-75057683332830379402011-10-10T12:47:58.996-04:002011-10-10T12:47:58.996-04:00Hi Josh,
I've used this SCOTUS-style techniq...Hi Josh, <br /><br />I've used <a href="http://rogerpielkejr.blogspot.com/2011/10/sarewitz-on-consensus.html" rel="nofollow">this SCOTUS-style technique</a> when presenting the "consensus" IV&V process/requirements tailoring to stakeholders for their acceptance. Obfuscation is more difficult using this approach.<br /><br />Quality hinges on its corrective action processes. By putting forward a diversity of opinion, later necessary corrective actions become easier. Yet the documented majority tailoring opinion provides the justification for moving forward in face of risk.George Crewshttp://www.gmcrews.comnoreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-4709993434735269192011-10-02T15:08:04.299-04:002011-10-02T15:08:04.299-04:00George, I think I'm familiar with the sort of ...George, I think I'm familiar with the sort of consensus process you are talking about. I think consensus on <i>verification</i> is achievable. One of the significant barriers to this working for <i>validation</i> is the lack of shared purpose, trust, and diverse risk appetites of the various stake-holders. As <a href="http://www.variousconsequences.com/2009/11/converging-and-diverging-views.html" rel="nofollow">Jaynes points out</a>, all of us good Bayesians shouldn't necessarily expect convergence of views when political motivations become significant. <br /><br />I am interested in hearing more about your experience in consensus building though.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-92195994784013728602011-10-02T01:07:33.342-04:002011-10-02T01:07:33.342-04:00The cost of IV&V must be weighed against its b...The cost of IV&V must be weighed against its benefits (largely risk reduction/confidence building). Therefore, it is always necessary, based on the nature and usage of the software, to tailor the SQA requirements and processes (of which IV&V is an important component).<br /><br />For the GCMs, you clearly point out that we run the full gamut for suggested tailoring -- the extremes being "any IV&V is impossible" to "formally prove the software correctly executes".<br /><br />What are the reasons for this? Well, tailoring is subjective. So we can encounter issues if 1) there is incomplete understanding of the nature of IV&V and its role in SQA, or 2) there is ethically/politically/financially motivated obfuscation.<br /><br />So a consensus tailoring will be difficult. But, IMHO, a big benefit for tackling the problem directly would be to draw the subjectivity away from the climate science itself, where it is having a corrosive effect. And I am confident this is the thing to do. I have had some experience with IV&V tailoring consensus building in a contentious area (nuclear) and know it can be done.George Crewshttp://www.gmcrews.comnoreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-77232338232417452912011-10-01T16:24:08.984-04:002011-10-01T16:24:08.984-04:00I meant to link the Models as Ink Blots post too.I meant to link the <a href="http://judithcurry.com/2011/09/18/climate-models-as-ink-blots-2/" rel="nofollow">Models as Ink Blots</a> post too.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.com