Fashions, Feedbacks, And Propagation Of Error

Visitor put up by Kevin Kilty


I had contemplated a contribution involving suggestions diagrams, techniques of equations, differential equations, and propagation of error ever since Nick Stoke’s unique contribution a couple of system of differential equations involving suggestions method again in June of this yr. A few days in the past the person Bartemis posted a pair of differential equations in a put up of Roy Spencer which offered some inspiration on tying all these ideas collectively in a single posting. Lastly, an added inspiration got here from the controversy about Pat Frank’s latest contribution. With out taking a stand on the numerical values Pat calculated, or his strategy, I hope to show why he and his critics, Nick for instance, are actually discussing various things; and why Pat’s tackle this matter deserves very cautious consideration. Right here goes.

Let’s take into account the next system of equations:



We are able to view this as a mannequin of water vapor suggestions the place; T is a floor temperature, C is a focus of water vapor, and a,b,c,d are constants. The set of equations is a system of first order differential equations, however non-linear. One also can view particular person phrases as second order if we differentiate the primary equation, and substitute into it the second. As an illustration, within the instance of temperature (T):


This, too, is non-linear however now a second order differential equation.

Thus, we will have a look at the temperature downside as a part of a primary order

system, or as a second order differential equation. It doesn’t matter besides that the primary order system is simpler to take care of mathematically.

1. Making a linear approximation

The system of equations has two regular options, which an individual can confirm by easy substitution. These are (T0 = zero, C0 = zero) or (T0 = (bc/da)1/three , C0 = c/d · T0). The primary is a trivial answer of no curiosity. The second turns into some extent round which we are going to make a linear approximation to the unique equation set. The algebra is a bit tedious and has little bearing on the problems at hand. But when we name ζ a small deviation from the regular answer in water vapor, and θ a small deviation in temperature the ultimate type of the set of equations is.



These are legitimate close to the stationary answer (T0 = (bc/da)1/three , C0 = c/d·T0). Doing as we did to supply the second order Equation three) we arrive on the following linear second order approximation.


2. Suggestions Block Diagrams

Earlier than persevering with towards my major function, I’d like to indicate the connection of the differential equations, above, to suggestions diagrams over which individuals have spilled electrons galore, though solely inexperienced ones, on these pages. Figures 1a and 1b present two doable block fashions of the second order differential equation in θ. The block fashions, and the differential equations are simply totally different descriptions of the identical factor. One has no mojo that the opposite doesn’t have. They’ve totally different utility. For instance, engineers should flip a system into digital or mechanical hardware that realizes the system, and the block diagram is beneficial for making this transformation.

Determine 1. Figures a, and b present various block fashions of the second order differential equation in θ. In a) the mannequin consists of two integrators, which flip θ¨ into θ, and the suggestions loops include achieve blocks. In b) there’s a filter that features as a leaky integrator. Block representations are usually not distinctive.

three. Stability of the Linear System

We’re ready now to use Nick’s stability evaluation. Discovering the eigenvalues of a two by two matrix is comparatively simple. It simply makes use of the quadratic components. The reader may seek the advice of Wolfram Math World on-line ( which exhibits eigenvalues on this occasion explicitly.


There are two adverse eigenvalues for some combos of (h,b,c,d), which means; the linearized system is secure, and the unique non-linear system is then secure towards infinitesimal disturbances. As Nick Stokes indicated, preliminary errors on this system will damp out. This, nevertheless, is just not the total story. For my part it’s the appropriate reply to a unsuitable query. The query of stability is not only a matter of the conduct of matrix A, the place:

but in addition the query of what happens with specific enter to the system. In a secure system like we will state bounded enter produces a bounded output. That’s good to know, however bounded doesn’t reply the extra advanced query of whether or not the output is beneficial for a selected function. This can be a area that propagation of error addresses. It’s, I sense, the place Pat Frank was taking in his essay, and whereas I don’t converse for him, one can’t dismiss the significance of what he was attempting to do.

four. The Actual Concern of Error Propagation

Let’s return to the linearized system (Eqs. four and 5 ). The system actually doesn’t do a lot of something attention-grabbing as a result of there’s nothing to drive it. We will need to have a vector of driving phrases, one involving the motive force of temperature, and probably one to drive water vapor. With out this all the answer ever does is decay again to the regular answer–i.e. its errors vanish. However this misses a number of necessary issues. In design of management system, the engineer has to take care of inputs and disturbances to the system. Thus, the photo voltaic fixed varies barely and pushes the answer away from the equilibrium one.[1] We’d put this within the vector U in Equation eight). There are undoubtedly disturbances driving the quantity of water vapor as effectively. The following degree of issue is having a mannequin that’s not absolutely specified. As an illustration, El Nino is just not a part of the state vector, but it surely does provide a disturbance to temperature and humidity. Thus it belongs in vector e in Equation eight). Maybe these lacking parameters present random influences which then seem to have come from the photo voltaic fixed or from water vapor. Lastly, by being solely a mannequin, we can’t probably know true values of the matrix components of ; we estimate them greatest we will, however they’re unsure.

A extra lifelike mannequin of what we’re coping with seems like this state house mannequin involving the vector of two state variables, temperature and humidity (X), and the drivers and random errors of enter (U + e). Simply to be full I’ve famous that what we observe is just not essentially the state variables, however moderately some operate of them which will have handed via devices first. What we observe is then another vector, Y, which could have its personal added errors, w, utterly unbiased of errors added to the drivers.

(eight) X˙ = · X + · (U + e)

(9) Y = C · X + · (U + w)

Despite the fact that it’s a extra lifelike mannequin, it nonetheless doesn’t describe propagation of error, however its answer is required equipment to get at propagated error. What we want to estimate, utilizing an answer to those equations is an expectation of the distinction between what we might observe with a greatest mannequin, which we will’t probably know precisely, and what the mannequin equations above produce with uncertainties thought of.

Here’s a regular answer to our state variables: X = · U + e. The matrix comes from a mix of the matrices and , the small print of which don’t matter right here.[2] As a matrix we will say merely that seems like this:

5. An Estimate of Uncertainty

There isn’t any function to turning into slowed down in extreme arithmetic. So, let‘s focus consideration on solely the equation for temperature, θ. It’s obvious that we would not have an entire or correct mannequin of temperature. Thus, let regardless of the equations eight) and 9) produce as an answer be referred to as an estimate of temperature. We use the image θˆ for this. The caret image is what statisticians typically use for an estimator. Then let the true temperature we might have discovered from an ideal mannequin be θ. Despite the fact that we don’t really know the true θ we determine our mannequin is just not too dangerous and so θˆis close by regardless of uncertainties.

Typically folks use a Taylor sequence to construct a linear approximation to θˆ, and calculate error propagation from it. This Taylor sequence is written when it comes to partial derivatives of all unsure unbiased variables and parameters, the pis, within the following.


However in our particular case the phrases of G, the gij, are coefficients of the linear approximation, with the drivers, the weather of the vector (U + e), as inputs. Thus our first order Taylor sequence seems like:

(11) (θˆ− θ) = g11 · (u1 + e1) + g12 · (u2 + e2)

As an estimate of propagated error what we require is the variance of (θˆ−θ) as a result of, now having random enter, this distinction has grow to be a random variable itself. Variance is outlined normally as E((Z−E(Z))2); the place E(…) means expectation worth of the random variable inside the parentheses.[3] Subsequently we must always sq. equation 11) and take the expectation worth, in order that what resides on the left hand aspect is the expectation worth of variance we search. Name this Sθ2. The best hand aspect of equation 11) produces many phrases as there are unsure parameters, plus cross merchandise.


Sθ2 = g112 ·E[(u1+e1)2]+g122 ·E[(u2+e2)2]+2·g11g12·E[(u1+e1)·( u2+e2)]

The phrases inside sq. brackets are variances or covariances. Even when the expectation values of the random inputs are zero, their variances are usually not. Thus, regardless of having a secure system of differential equations, the variance of the state variables most likely won’t are inclined to zero as time progresses.

There’s a additional level to debate. The matrix is just not identified precisely. Every aspect has some uncertainty which equations eight) and 9) don’t point out explicitly. One option to embrace that is to position a uncertainty matrix in sequence with ; which then turns into + . is sort of a matrix of random variable which we assume have expectation values of zero for all components, however as soon as once more would not have zero variance. This matrix will produce uncertainty in , via its relationship to . A whole propagation of error takes some thought and care.

6. Conclusion

The worth of Sθ2 which ends from an entire evaluation of the contributors to uncertainty when in comparison with the precision wanted, is what actually determines whether or not or not mannequin outcomes are match for function. As I wrote in a remark at one level in Pat’s unique posting, the subject of propagation of error is advanced; and I used to be advised that it’s certainly not advanced. I feel this dialogue exhibits that it’s extra advanced than many individuals suppose, and I hope it helps reinforce Pat Frank’s factors.

7. Notes:

(1) = ( −λ·I)−1 See, for example, Ogata, System Dynamics, 2004, Pearson-Prentice Corridor.

(2) Mototaka Nakamura, in his latest monograph, which is on the market for obtain at, alludes to small variations within the photo voltaic fixed. This might go into the vector U for example.

(three) That is defined effectively in Bevington, Information Discount and Error Evaluation for the Bodily Sciences, McGraw-Hill, 1969. Bevington illustrates quite a lot of easy instances. Neither his, nor another reference I do know, tackles propagated error via a system of linear equations.

Like this:

Like Loading…

Leave a Reply

Your email address will not be published. Required fields are marked *