Linear Systems

BIBLIOGRAPHY

Linear systems are systems of equations in which the variables are never multiplied with each other but only with constants and then summed up. Linear systems are used to describe both static and dynamic relations between variables.

In the case of the description of static relations, systems of linear algebraic equations describe invariants between variables such as:

a₁₁x + a₁₂x₁ = c₁

a₂₁x + a₂₂x ₂ = c₂

Here, one would be interested in the values of x₁ and x₂ for which both equations hold. This system of equations can easily be written in matrix form:

or, more concisely:

Ax = c

The solution can be written as x = A^-1 c if the matrix A is invertible. Another frequent application of systems of linear algebraic equations is the following:

y_i = b₀ + b₁x_{i 1} + b₂x_{i 2} + b₃x_{i 3} + … + b_mx_{i m} + e_i

The above is a regression equation stating that for every object i, its attribute Y has a value that can be expressed as the weighted sum of its attributes X₁ through X_m, with a measurement error of E (whose variance should be a minimum in the classical regression analysis). The term linear is derived from the fact that the graphical representation of the above equation for m = 1 is a straight line (with intercept b₀ and slope b ₁). The above equation holds for all i, thus the system of equations:

y ₁ = b ₀ + b ₁x ₁₁ + b ₂x ₁₂ + b ₃x ₁₃ + … + b _m x _1m + e ₁

y ₂ = b ₀ + b ₁x ₂₁ + b ₂x ₂₂ + b ₃x ₂₃ + … + b _m x _2m + e ₂

y _n = b ₀ + b ₁x _n1 + b ₂x _n2 + b ₃x _n3 + … + b _m x _nm + e _n

for all n objects is often written in the abbreviated form, using matrices and vectors,

y = Xb + e

where y and e are column vectors containing all y_i and e_i, respectively (i = 1 … n ), b is a row vector containing all b_j (j = 0 … m ), and X is a n × (m + 1) matrix (with n rows and m + 1 columns) containing x_ij in the cells in row i and column j (where x_{i 0} = 1 for all i ). Here one is interested in the values of the regression coefficients b_j (j = 0 … m ) which minimize the variance of the regression residual E. This is solved by calculating e^T e which is the sum of the squares of the still unknown e_i and calculating the derivatives of e^T e with respect to all (also unknown) b_j. These derivatives will be 0 for e^T e = min, and the solution of this minimization problem is expressed as follows:

Linear systems are also used to describe dynamic relationships between variables. An early standard example from political science is English physicist Lewis Fry Richardson’s (1881-1953) model of arms races, which consists of the following simplifying hypotheses:

• The higher the armament expenses of one military block, the faster the increase of the other block’s armament expenses (as the latter wants to adapt to the threat as quickly as it can).
• The higher the armament expenses of a military block, the slower those expenses will increase (as it becomes more difficult to increase the proportion of military expenses with respect to the gross national product).

Calling the armament expenses of the two blocks x₁ and x₂, respectively, and their increase rates xẋ ₁ and xẋ ₂, respectively, one can model the increase rates as proportional both to the military expenses of the other block and the nonmilitary expenses of the own block (the total max max expenses and being constant), with some proportionality constants a, b, m, and n :

or, in shorter form:

Such linear systems of differential equations usually have a closed solution, that is, there is a vector-valued function q (t ) that fulfills this vector-valued differential equation. Usually, the precise form of q (t ) is not very interesting, but generally speaking it has the form:

q (t ) = θ ₁ q ₁ e ^λ1t + θ ₂ q ₂ e ^λ2t + q_s

where q ₁, q ₂ and q_s are constant vectors and θ ₁, θ ₂ λ ₁ and λ ₂ are constant numbers of which mainly q_s, λ₁ and λ₂ are of special interest. q_s is the stationary state of the system of differential equations, that is, once the system has acquired this state, it will never leave that state, as the derivatives with respect to time vanish. λ ₁ and λ ₂ are the so-called eigenvalues of the matrix A, which, as the exponents of the two exponential functions in the above equation, determine whether the function q (t ) will grow beyond all limits over time or whether the two first terms in the right-hand side of the above equation will vanish as time approaches infinity. For negative values of both λ ₁ and λ ₂ the latter will happen, and the overall function will approach its stationary state (in which case the stationary state is called stable or an attractor or sink ). If both eigenvalues are positive, then the function will grow beyond all limits (in which case the stationary state is called unstable or a repellor or source ). If one of the λ _s is positive while the other is negative, the stationary is also unstable, but is called a saddle point because the function will first approach the stationary state and then move away. There is a special case when λ ₁ and λ ₂ are pure imaginary numbers—which happens if 4ab < 0 and m = n = 0, as . In the arms race model this is not a reasonable assumption, as one block would increase its arms expenses faster, the smaller the arms expenses of the other block are (exactly one of a and b would be negative and the respective block would behave strangely, the other constant would be positive, so the other block would behave normally) and both would increase or decrease its arms expenses regardless of what their current values are (m = n = 0 means that there is no influence of the current value of arms expenses upon its change for both blocks). In this case both variables x ₁ and x ₂ oscillate around the stationary state.

The example demonstrates that systems of linear differential equations always have a closed solution, which can be expressed in several different forms. There is always exactly one stationary state (except in the case that the matrix A is not invertible) that can either be a sink or a source or a saddle or a center. Nonlinear systems often have more than one stationary state, but their behavior can be analyzed in a similar way, taking into account that a linear approximation of a nonlinear system behaves approximately the same in a small neighborhood of each of its stationary states.

SEE ALSO Input-Output Matrix; Matrix Algebra; Nonlinear Systems; Simultaneous Equation Bias; Vectors

BIBLIOGRAPHY

Arnold, Vladimir I. 1973. Ordinary Differential Equations. Cambridge, MA: MIT Press.

Richardson, Lewis Fry. 2003. Mathematics of War and Foreign Politics. In The World of Mathematics, vol. 2, ed. James R. Newman. Mineola, NY: Dover Publications.

Robinson, Derek J. S. 1991. A Course in Linear Algebra with Applications. River Edge, NJ: World Scientific.

Klaus G. Troitzsch