1 / 27

CSC5160 Topics in Algorithms Tutorial 1

CSC5160 Topics in Algorithms Tutorial 1. Jan 25 2007 Jerry Le jlle@cse.cuhk.edu.hk. Outline. Basic Linear Algebra – a quick review Vector space, Spanning set Linear independency Matrix: operations, rank, inverse, eigenvalue, determinant Introduction to Linear Programming.

terris
Download Presentation

CSC5160 Topics in Algorithms Tutorial 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSC5160 Topics in AlgorithmsTutorial 1 Jan 25 2007 Jerry Le jlle@cse.cuhk.edu.hk

  2. Outline • Basic Linear Algebra – a quick review • Vector space, Spanning set • Linear independency • Matrix: operations, rank, inverse, eigenvalue, determinant • Introduction to Linear Programming

  3. Linear AlgebraVector space, spanning set • Vector space - a set V that is closed under the ‘+’ and ‘•’ operations - e.g. Note: A vector space does not necessarily contain “vectors”!

  4. Linear AlgebraVector space, spanning set • Spanning set - Definition: the spanning set of a nonempty set S is the set of all linear combinations of vectors in S - Definition: a subspace of a vector space is itself a vector space - Lemma: The span of a subspace is itself a vector space - e.g.

  5. Linear AlgebraLinear independency • Linear independency - Definition: A subset of a vector space is linearly independent if none of its elements can be represented as a linear combination of the others. - e.g. linearly independent linearly dependent

  6. Linear AlgebraLinear independency • Linear independency - in other words: - Basis Definition: A basis of vector space V is a linearly independent subset whose span is V

  7. Linear AlgebraMatrix • Operations - sums and scalar products - multiplication • Rank of a matrix Definition: the rank of a matrix A is the maximal number of linearly independent rows (or columns) of A

  8. Linear AlgebraMatrix • Rank of a matrix (cnt.) - Properties (consider an m-by-n matrix A) 1. the rank of A is at most min(m,n) 2. if the rank is n (m), A has full column (row) rank 3. if A is a square matrix (m=n), A is invertible iff it has rank n (Invertible: there exists B such that AB=In) 4. if B is an n-by-k matrix with rank n, then AB has the same rank as A 5. if B is an k-by-m matrix with rank m, then BA has the same rank as A …

  9. Linear AlgebraMatrix • Rank of a matrix (cnt.) - Computation by Gauss elimination (or Gauss-Jordan reduction) rank: 3

  10. Linear AlgebraMatrix • Invertible matrix - an n-by-n matrix A is invertible iff 1. there exists an n-by-n matrix B such that AB=BA=In OR 2. A has rank n OR 3. the equation Ax=0 has only trivial solution x=0 OR 4. the equation Ax=b has exactly one solution in Rn OR 5. the rows (columns) of A are linearly independent OR 6. the rows (columns) of A form a basis of Rn OR 5. the transpose AT is invertible OR …

  11. Linear AlgebraMatrix • Eigenvalue & Eigenvector - consider an m-by-n matrix A as linear transformation: - Definition: for an n-by-n matrix A, if there exist an non-zero vector x and a scalar λ, such that Ax=λx then λis the eigenvalue of A, and x is the eigenvector of A. - The eigenvalue (λ)“visualizes” the effect that a linear transformation (A) produces on a vector (x).

  12. Linear AlgebraMatrix • Eigenvalue & Eigenvector (cnt.) - how to compute? Ax=λx Ax=λInx (A-λIn)x=0 (A-λIn)x=0 must have non-trivial solutions i.e. B=(A-λIn)x is NOT invertible • Determinant - how to determine if an (n-by-n) matrix (A) is invertible? by Gauss elimination

  13. Linear AlgebraMatrix • Determinant (cnt.) - Gauss elimination: by linear row (column) operations If AFJMX=0, then A is not invertible so, can we define some function det(A), which is equal to AFJMX, or at least equal to λAFJMX?

  14. Linear AlgebraMatrix • Determinant (cnt.) - Tracing every step of a specific process of Gauss elimination, we get

  15. Linear AlgebraMatrix • Determinant (cnt.) - determinant as size functions More generally, the absolute value of det(A) is the volume of the box formed by A’s column vectors V1,V2,…Vn in n-dimensional space

  16. Outline • Basic Linear Algebra – a quick review • Introduction to Linear Programming • Definition • Example • Basic feasible solution • Geometry of LP

  17. Linear programmingDefinition • General optimization • Linear programming - when all f(x),gi(x),hj(x) are linear (general form) objective function inequality constraints equality constraints

  18. Linear programmingExample • Example: the Diet problem m kinds of nutrient

  19. Linear programmingExample • Example: the Diet problem - question: how to buy the food such that the cost is minimized, while the minimal nutrient requirement is satisfied? Canonical form of LP (without equality constraints in the general form)

  20. Linear programmingExample • Example: the Diet problem - what if the amount of nutrient must be exactly the same with the requirement? Standard form of LP (without inequality constraints in the general form)

  21. Linear programmingExample • Equivalent of the three forms of LP - the general form, canonical form (without equality constraints) and standard form (without inequality constraints) are equivalent - One form can be transformed into another Therefore, we can simply consider standard form without loss of generality for all LPs

  22. Linear programmingBasic feasible solution • Basic feasible solutions - consider the standard form of LP: How to find the solution? 1. solve the equation system Ax=b (find basic solutions) 2. pick the basic feasible solutions satisfying x≥0, out of the basic solutions 3. find the solution that minimize cx

  23. Linear programmingBasic feasible solution • how to find the basic solution of Ax=b? • If A is invertible, then x has unique solution • If A is not invertible, assuming A has rank m • 1. select a collection of m linearly independent column vectors of A • 2. set the components of x that are not corresponding to the selected • column vectors to zero • 3. solve the m resulting equations to get the remaining component of x • Basic feasible solutions

  24. Linear programmingBasic feasible solution • Basic feasible solutions - how to find the basic solution of Ax=b? (cnt.) (0,0,0,4,2,3,6) feasible (0,4,0,0,2,3,-6) not feasible (because x7<0)

  25. Linear programmingGeometry of LP • Geometry of LP - consider the LP

  26. Linear programmingGeometry of LP • Geometry of LP in Rn space - the linear constraints form a closed convex polytope - optimal solution appears on the boundaries - local optimal is global optimal - Simplex algorithm: starting from a vertex, search along the edges until reaching the vertex of the optimal solution

  27. Thanks for your attention

More Related