tag:blogger.com,1999:blog-5822805028291837738.post3315977831626353751..comments2022-11-30T11:50:19.185-05:00Comments on Various Consequences: Chaotic Method of Manufactured Solutions: Lorenz 63Joshua Stultshttp://www.blogger.com/profile/03506970399027046387noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-5822805028291837738.post-26528139438061660622010-10-15T07:59:38.648-04:002010-10-15T07:59:38.648-04:00The point Tomas raises here about averages being c...The point Tomas raises <a href="http://judithcurry.com/2010/09/22/the-uncertainty-monster/#comment-2592" rel="nofollow">here</a> about averages being chaotic deserves a post in the Lorenz63 series; that's something we can show with the toy model.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-25744388314541264832010-09-27T14:53:43.544-04:002010-09-27T14:53:43.544-04:00Interesting comment on control of chaotic systems,...<a href="http://judithcurry.com/2010/09/25/climate-book-shelf/#comment-2027" rel="nofollow">Interesting comment</a> on control of chaotic systems, which is very closely related to verification with MMS (since control terms show up as forcings in the governing equations).Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-55112904675425470052010-02-06T17:58:32.298-05:002010-02-06T17:58:32.298-05:00You gents might appreciate the discussion over her...You gents might appreciate <a href="http://scienceblogs.com/stoat/2007/11/a_bit_more_stealth.php" rel="nofollow">the discussion over here</a> on <a href="http://www.climateprediction.net/science/pubs/PNAS_knightetal_2007.pdf" rel="nofollow">this paper</a> about the effects of different hardware / software on ensemble results.<br /><br />From the abstract:<br /><i>[...] We demonstrate that effects of parameter, hardware and software variation are detectable, complex and interacting. However we find most of the effects of parameter variation are due to a small subset of parameters. [...] We demonstrate the effect of hardware and software is small relative to the effect of parameter variation and, over the wide range of systems tested, may be treated as equivalent to that caused by changes in initial conditions. </i>Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-60882366629595827862010-01-21T05:48:26.505-05:002010-01-21T05:48:26.505-05:00But if the system is NOT ergodic , it means that t...But if the system is NOT ergodic , it means that there are "holes" and "traps" . For such system you can be never sure what it will do for a given initial condition . Even 2 initial conditions separated by the smallest possible interval (the Heisenberg limit) might lead to 2 very different trajectories .<br />You are right that fractal sets (strange attractors) are typically an example .<br />I will take an example of convergence difficulty - the Newton-Rhapson iteration .<br />If you apply it to some complex polynomial (f.ex z^4 - 1) and vary the initial conditions you will obtain a fractal boundary between the points that lead to a convergence to one of the 4 roots . <br />That means that there will be an infinity of points such as that you can make an interval I as small as you want around these points and for points interior to this interval you will converge to any of the 4 roots .<br />So actually you are unable to say to what root will converge the Newton Raphson iteration for a given initial point regardless of the smallness of the interval between 2 initial conditions .<br /><br />The Lorenz attractor is a "well behaved" chaos .<br />While the attractor itself is strange , its strangeness only translates the already known "banal" chaos feature - unpredictability of individual trajectories .<br />But its boundary is not fractal so you will always finish on the attractor if you are in the chaotic regime .<br />Once you are on it , the ergodicity will make sure that you will run through it by visiting every part but the trajectories will NEVER intersect (demonstrated very important result for all chaotic systems) .<br />This property allows me to mention an amusing paradox showing the problems with numerical integrations .<br />You choose the smallest possible integration step that the best computer allows .<br />You take some initial condition and begin to compute . Given the computer accuracy there is a FINITE number of different points it can represent .<br />Even if this number is very large , if you compute long enough your trajectory will eventually find a point which is EXACTLY the same that it has already visited and from that point on your computer will only periodically cycle this same trajectory .<br />The computer just violated in the major way the fundamental rule that trajectories never intersect !<br />And I have just proven that it will always happen with any computer with finite accuracy :)<br /><br />As for the books , there are of course many of varying difficulty .<br />Personnaly I like and usually recommend the R.Hilborn's Introduction to chaos and non linear dynamics . Comprehensive , rigorous and gives all the references and ways to go deeper in the matters if one wishes to do so .Tom Vonknoreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-56446545796255776682010-01-21T05:47:49.895-05:002010-01-21T05:47:49.895-05:00Well you are rediscovering the chaos theory ;)
I a...Well you are rediscovering the chaos theory ;)<br />I already mentioned it but would like to stress again the importance of ergodicity .<br />The Lorenz system is ergodic that's why eventual results obtained with this system can't be generalized to all chaotic systems that might be non ergodic .<br />This point joins your remark about different basins of attraction .<br />Ergodicity means broadly that the trajectories go everywhere in the attractor . In other words with any initial conditions if you wait long enough the trajectory will visit all points .<br />Hence the notion of pdf and the possibility to treat the system stochastically (of course the individual trajectories can't still be predicted beyond some finite time) .Tom Vonknoreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-7053695476735124512010-01-20T13:05:38.835-05:002010-01-20T13:05:38.835-05:00Since Tom got me thinking, I found this article on...Since Tom got me thinking, I found this <a href="http://www.scholarpedia.org/article/Basin_of_attraction" rel="nofollow">article on attractors</a> which discusses the effect of uncertainty in initial conditions has on determining which basin an IC is in. Basins can have fractal boundaries, so it could have 'interesting' features even at the scale of the machine precision.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-17701028893700982822010-01-20T12:38:47.361-05:002010-01-20T12:38:47.361-05:00Ok, I thought about it some more; I guess you coul...Ok, I thought about it some more; I guess you could argue that even if you put your ensemble member's ICs as close as possible given the limits of your machine, there could still be ICs 'between' those that are in different basins of attraction and you'd never know it.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-85364226405788789992010-01-20T11:55:53.432-05:002010-01-20T11:55:53.432-05:00Interesting discussion. My thanks to both of you.Interesting discussion. My thanks to both of you.George M. Crewshttps://www.blogger.com/profile/05795380097849494251noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-77268731252973719052010-01-20T11:32:11.024-05:002010-01-20T11:32:11.024-05:00I buy the upper-bound part, I kind of experienced ...I buy the upper-bound part, I kind of experienced that with this little toy problem, trying to converge the solutions further and further out required really cranking down the time-step size to the point that it became impractical (this is a blog after all, it's not worth waiting a week for solutions). And the ensemble where the ICs were perturbed on the order of the round-off error all eventually diverged (at about the same time) no matter what the time-step.<br /><br /><i>That's why some people call all these curves numerical artefacts because nobody knows if those curves are parts of solutions restricted to some finite [0,T] interval .</i><br /><br />This is the part I have trouble with, if I can show that my implementation converges to a manufactured solution, and I can show time-step convergence for an ensemble (with IC's perturbed on the order of the machine precision), then haven't I shown that the trajectory is <i>not</i> a numerical artifact? <br /><br />I haven't done much reading on 'chaos' per se, do you have any references you could point me at that would clear this up (no hurry, this is just my idle curiosity)?<br /><br />Thanks again for your comments.Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-27334553983257530802010-01-20T11:10:20.564-05:002010-01-20T11:10:20.564-05:00Hello
Only fast today .
It is not for ANY finite ...Hello<br /><br />Only fast today .<br />It is not for ANY finite time . There is an upper bound given by the Heisenberg relation . But even the Heisenberg accuracy/step is completely out of reach of any computer .<br />So what is practically feasible leads to very short times and if you wanted to go beyond , you'd need to decrease the steps exponentially .<br />That gives an upper time limit for numerical solutions very fast .<br />In short there is no converged numerical solution . <br />Actually you can not even be sure that a limit when the step goes to 0 is near to what you get with non 0 steps .<br />That's why some people call all these curves numerical artefacts because nobody knows if those curves are parts of solutions restricted to some finite [0,T] interval .<br />Not really surprising for a chaotic system where even the beginning may change when the values reach , pass some thresholds :)<br /><br />I don't do numerical models so have little experience with climate models too . But they use LES in the dynamical core .<br />However one must not think that the climate models "solve" N-S in any sense of the word !<br />What they basically do is to compute 2 different equilibrium solutions and compare them .<br />It's 2 snapshots , it is not a movie .TomVonknoreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-81690402326047785942010-01-18T09:22:54.169-05:002010-01-18T09:22:54.169-05:00Hi Tom,
Thank you for leaving such thoughtful comm...Hi Tom,<br />Thank you for leaving such thoughtful comments. Dan emailed me too and gave me a 'nudge' about actually doing some demonstration calculations based on the ideas in this post: <a href="http://j-stults.blogspot.com/2010/01/chaotic-time-convergence-and-mms.html" rel="nofollow">they are at the next post</a>, which I think you'll find interesting.<br /><br />You said: <i>The size of the step used in the numerical solution can be considered as the difference between 2 initial conditions . But as any chaotic system has at least one positive Ltapounov coefficient , we know that 2 trajectories diverge exponentially .</i><br /><br />I understand that, but my background is engineering, so I'm interested in the behaviour at finite time. So I'm happy as long as the time-step size or the numerical method still gives me a 'knob' I can tweak to get arbitrarily close to the solution for arbitrary (but finite) times. The convergence results in <a href="http://j-stults.blogspot.com/2010/01/lorenz-63-ensemble.html" rel="nofollow">this post</a> and <a href="http://j-stults.blogspot.com/2010/01/chaotic-time-convergence-and-mms.html" rel="nofollow">this post</a> seem to show that this is quite possible. <br /><br />You said:<i>Kolmogorov turbulence theory is a kind of attempt at statistical fluid dynamics but it doesn't work well and it fails completely for lower Reynolds numbers.</i><br /><br />While that's true, it's also quite understandable considering the assumptions that go in to deriving the scaling relations of Kolmogorov's 'energy cascade', we shouldn't expect it to work at low Re. <br /><br />You said: <i>So when one sees that in the so called "climatology" people get rid of the space by simply ... AVERAGING (!) and then "analyse" the time series of spatially averaged parameters (like GMST) , one wonders what kind of physics they do .</i><br /><br />I agree, boiling the climate down to a single globally averaged number is foolish, and most likely politically motivated. Averaging on smaller scales in space and time and introducing models for the extra terms you introduce is practically useful though (RANS / LES). I'm not really familiar with the internals of the climate models, do they do LES (wasn't Smagorinsky a climate modeler)?Joshua Stultshttps://www.blogger.com/profile/03506970399027046387noreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-43228344581157002082010-01-18T08:51:21.243-05:002010-01-18T08:51:21.243-05:00- stochasticity . Obviously a chaotic system canno...- stochasticity . Obviously a chaotic system cannot be predicted deterministically be it numerically or otherwise (see above) . Therefore a legitimate question is whether there can be a statistical description of the trajectories . Or of the frequency of places that the trajectory will visit what is a similar question .<br />Well here the answer is : sometimes yes .<br />The key word is ergodicity and that's why it is (almost) impossible to study the chaos theory without studying the ergodic theory too .<br />Indeed an ergodic system fulfills the conditions of the ergodic theorem which says broadly that "the time average of a dynamical parameter taken along a trajectory converges to the phase space average weighted by a suitable probability density function when time goes against infinity" .<br />The ergodicity is the condition of existence because it guarantees that a pdf exists such as it permits to connect the phase space states to the temporal evolutions .<br />Not all chaotic systems are ergodic . F.ex colliding hard spheres are chaotic and ergodic while planetary orbits are chaotic and non ergodic . The use of ergodicity in the former leads to statistical thermodynamics which works very well . On the other hand the unpredictability of planetary orbits is both in the deterministic and the stochastical sense .<br /><br />How does the climate fit in ?<br />Well it doesn't fit in . The chaos theory is mathematically a problem of non linear ODE while the climate (and weather) is a problem of non linear PDE . Space matters . We deal here with SPATIO-temporal chaos . The best way to intuitively grasp teh spatio temporal chaos is to look at the cigarette smoke . You clearly see that chaotic structures exist both in time and in space . The ergodic theorem applies only on dynamical systems depending on time . Finding an equivalent of ergodicity for spatio-temporal systems is cutting edge science - it is just beginning .<br />What is known and what the cigarette smoke also demonstrates is that while temporal chaos is often ergodic in physical systems of interest , it is not the case for spatio-temporal systems .<br />One consequence of that is that there is no equivalent of statistical thermodynamics for the Navier Stokes equations - you can't find "statistical" properties of solutions .<br />We don't even know if there is a unique smooth solution for given boundary/initial conditions . Kolmogorov turbulence theory is a kind of attempt at statistical fluid dynamics but it doesn't work well and it fails completely for lower Reynolds numbers .<br />So when one sees that in the so called "climatology" people get rid of the space by simply ... AVERAGING (!) and then "analyse" the time series of spatially averaged parameters (like GMST) , one wonders what kind of physics they do . Etc .<br />Well I think I have been too long already so I hope it shed some light .TomVonknoreply@blogger.comtag:blogger.com,1999:blog-5822805028291837738.post-46536908087194030542010-01-18T08:50:19.205-05:002010-01-18T08:50:19.205-05:00Hello
I am visiting here for the first time becau...Hello<br /><br />I am visiting here for the first time because Dan Hughes sent me a mail pointing to your blog .<br />Perhaps you will find the following few comments helpful .<br />First . My chaos interest comes from the mathematical physics point of view so I have difficulty to read and interpret the pages of code that you post .<br />If I understood well , you began to look at 2 aspects of deterministic chaos - convergence and stochasticity .<br />As it happens I have a rather good knowledge of both so if you continue being interested by the chaos theory , I could perhaps show you some avenues .<br /><br />- The convergence question is easy . <br />There is none . <br />A fast qualitative argument bases on the Lyapounov coefficient . The size of the step used in the numerical solution can be considered as the difference between 2 initial conditions . But as any chaotic system has at least one positive Ltapounov coefficient , we know that 2 trajectories diverge exponentially .<br />Now the Heisenberg uncertainty relation gives us the lower bound for the accuracy with which we can distinguish 2 different initial conditions .<br />Follows that the greatest accuracy achievable corresponding to this lower bound leads to trajectories that still diverge (exponentially).<br />In other words a quantum fluctuation is enough to give an upper bound on the predictability time . Beyond that time no converging solution is computable even in principle .TomVonknoreply@blogger.com