HKOI 2005 Training

1. HKOI 2005 Training Introduction to Algorithms Alan, Tam Siu Lung

2. What is Algorithm? • Algorithm • A set of data manipulations instructions to solve a problem (informatics problem) • Problem • A mapping from each input (in a specific format) to an output (in a specific format) • Pseudo-Code • Realization of Algorithm in the current domain • Mapping from problem input to algorithm input • Mapping from algorithm output to problem output • Program • Realization of Pseudo-Code given • A compiler and the language supported • Data in plain text format and the IO routines supported

3. A B C Algorithm1 Algorithm2 Algorithm3 B C D Coding Source-Code int main() { A a = Read(); B b = Algorithm1(a); C c = Algorithm2(b); D d = Algorithm3(c); Write(d); } Visualization of Program Pseudo-Code

4. Algorithm Example • Sorting Problem • Given an array of N comparable elements with comparablekeys • Output a permutation of the elements • such that no element is has a keygreater than that of the one there before • Note • Array = a sequence of elements • Element = a key-value pair • Permutation = ... • Greater than = …

5. Realization Example • To print in sorted order a set of points in a cetacean plane by the distance from origin: • Realization contains • Key = distance from origin • Value = coordinates = (x, y) • Comparison of 2 elements = comparison of the numerical values of distances • Pseudo-Code contains • Store the set in an array • Sort them • Output the array in order

6. Why Algorithms? • Computational Biology • Database • Cryptography • Quantitative Analysis • Artificial Intelligence • Scheduling

7. Example – Multiple Sequence Alignment • Input • GCTAGCTAAC • AACTAGC • AGCGCTAGCTA • TAACGACTAT • Find a string with minimum which contains all these sub-strings

8. Example – Multiple Sequence Alignment • Find a string with minimum which contains all these sub-strings • Output AACTAGCGCTAGCTAACGACTAT GCTAGCTAAC AACTAGC AGCGCTAGCTA TAACGACTAT

9. Example – Sorting By Reversal • Given a sequence of integers 1..N • Every step you can reverse a continuous sub-subsequence • Find the minimum number of steps to sort it

10. Example – Sorting By Reversal • Input • 4 3 1 5 6 2 • Output • 4 3 1 5 6 2 • 1 3 4 5 6 2 • 1 6 5 4 3 2 • 1 2 3 4 5 6

11. To know what algorithms are • Mathematics • Symbols and notations • Methods • Data Structures • Specification of input and output • Internal storage during processing • You can recite them. • It is science. You can also derive them.

12. To know how to use algorithms • Techniques • Common ways that algorithms are built upon • Formulations • How to map the problem domain into entities in algorithms • It is art. We have no way to teach you! • That’s why you need exercises.

13. Techniques • Recursion • Reduction into sub-problems • Divide and Conquer • Reduction into >1 sub-problems • Collect the results • Exhaustion • Expand all reductions into sub-problems • Greedy • Use only 1 reduction into sub-problem(s) • Dynamic Programming • Try all and remember the one

14. To know why algorithms work • Complexity Analysis • Proof of correctness • Mathematics-oriented • Out of our scope, but … • To “invent” new algorithms …

15. HKOI concerns • All basics and techniques specified above • Mathematics, Data Structures • Recursion, Exhaustion, Greedy, • Algorithms which implement the techniques • Sorting, Searching, DFS, BFS, Prim, Kruskal • Dijkstra, Bellman-Ford, Warshall-Floyd • Famous IO models used by algorithms • Tree, Graph • Search Space (Graph Searching) • State Space (Constraint Satisfaction)

16. Introduction to Algorithms Questions?

17. HKOI also concerns • Modifications to algorithms • To accommodate the situation • Multi-state BFS • For speed • Branch & Bound • For memory • Bit-wise States • Other paradigms • Open test data (known input) • Interactive (hidden input)

18. Complexity • An approximation to the runtime and memory requirement of a program. • Note that there are • Best-case complexity • Average-case complexity • Worst-case complexity • In most cases, we concern runtime only.

19. Measuring Runtime • Definition • An operation • Problem size • Consider bubble sort: for i := N - 1 downto 1 do for j := 1 to i do if A[j] > A[j+1] then swap(A[j], A[j+1]); • In worst case how many “if” operations?“swap” operations? “for j” operations?“for j” operations? Integer comparisons?

20. Complexity • Give a name to a group of functions with similar growth property, e.g. use O(n2) to mean n2 + n, 2n2 – 10n, 10n2, … • Since the constant won’t be too big practically, it is a good indicator of expected runtime. • We uses “worst case runtime complexity” (a.k.a. order) extensively to describe runtime of a program

21. Graph 1 2 8 4 6 3 7 5

22. NT Q WA SA NSW V T Graph

23. Graph • A directed graph G=(V,E), where • V = the set of Node/Vertex • E = the set of Edge/Arc, where • E=(v1,v2) where v1 and v2 are in V • Extreme: |E| ≤ |V|2 • So we have real algorithms with order: • O(|V|3) • O(|V| + |E|) • O(|V| log |V| + |E|) • O(|E| log(log* |E| – log* |E|/|V|))

24. Difficulty of Problem • Definitions (INCORRECT) • A problem with order being a polynomial is called polynomial-time solvable (P) • A problem whose solution is verified in polynomial time is said to be polynomial-time verifiable (NP) • A problem with no known polynomial-time solution to date is called NP-hard • Difficulty of problems are roughly classified as: • Easy: in P (of course all P problems are also in NP) • Hard: in NP but not in P (NP-complete) • Very Hard: not even in NP

25. Complexity • When you have a problem, given input (output) constraints and runtime constraints, you can guess whether an algorithm works or not. • But sometimes the stated order of an algorithm depends on some tricky data structures, which we won’t have skills or time to implement!

26. IOI/NOI Problems Requires • Understanding of basic algorithms • Innovation on ways to realize the algorithms • Technical capability of the realization • Remember: Different people care different things in different situations

27. Understanding, Why Induction Deduction Knowledge, How Information, What Finally