MER9020

1. Prof emer. Rein Küttner & juhtivteadur Jüri Majak TTÜ, masinaehituse instituutTelef: 588 78 889E-mail: juri.majak@ttu.ee MER9020

2. Aineprogramm Sissejuhatus • Matemaatilised mudelid ja vahendid nende loomiseks/kirjeldamiseks. Eksperimentaalsete mudelite kasutamine optimeerimisel. Inseneri statistika (Engineering statistics) • Asendusmudelid ja nende kasutamine • Statistika ja projekteerimine. Eksperimentidest määratud mudelite kasutamine optimeerimisel. Katsete planeerimine. Engineering statistics and design. Statistical Design of Experiments. Use of statistical models in design. • Kaasaegsed meetodid asendusmudelite koostamiseks. Närvivõrkude metoodika kasutamine. • Mudeli täpsuse hindamine. Riskide arvestamine, töökindlus, ohutus. Statistical tests of Hypotheses. Analysis of Variance. Risk, reliability, and Safety • Insenerilahenduste optimeerimine, üldalused (Basics of Optimisation methods and search of solutions) • Optimeerimisülesannete tüübid (Types of Optimization Problem) • Mitmekriteriaalsed optimaalse valiku ülesanded. Pareto-optimaalsed lahendid. (Multicriterial optimal decision theory. Pareto optimality, Use of malticriterial decisiontheory in engineering design). • Klassikalised optimeerimisülesanded. (Optimization by Differential Calculus. Lagrange multipliers. Examples of use the classical optimization method in engineering). • Matemaatilise planeerimise meetodid. Otsene ja duaalne planeerimise ülesanne. Duaalse ülesande kasutamine. Optimeerimisülesennete lahendite tundlikkuse analüüs. (Mathematical programming methods and their use for engineering design, process planning and manufacturing resource planning.Direct and Dual Optimization tasks. Sensitivity analysis). • Geneetilised optimeerimise algoritmid.. • Tehniliste süsteemide ja protsesside simuleerimine. Simuleerimine kasutades juhuslike arvude generaatoreid ja SIMULINK’i • Toodete ja tootmisprotsesside optimaalne projekteerimine arendused. Optimeerimise kasutamise näited.

3. Katsed, arvutikatsed, Protsesside jälgimine, katsetulemuste statistiline hindamine, mudelite hindamine jms Katsete plaanimine (DOE) Simuleerimine, analüüs Asendusmudel; Surrogaatmudel (response surface), asendusmudelite kasutamine Optimeerimine:optimaalse valiku ülesanded, lineaarne, mittelineaarne planeerimine Tehniliste süsteemide optimeerimine (skeem) Kursuse üldine skeem

4. Iseseisev-praktiline töökoosneb järgnevatest töödest: • Katsetulemuste statistiline hindamine: peamiste statistiliste hinnangute leidmine, kahe erineva katseseeria vahelise seose hindamine(korrelatsioon) , katseeeria varieeruvuse erinevuse hindamine. • Regressioonanalüüsi mudeli koostamine: Valitud mudeli sobivuse analüüs. • Närvivõrkude mudeli leidmine • Lineaarse (mittelineaarse) optimeerimisülesande lahendamine, tulemuste analüüs • Mudeli statistiline simuleerimine arvutil.

5. Overall Goal in Selecting Methods The overall goal in selecting basic research method(s) in engineering design is to get the most useful information to key decision makers in the most cost-effective and realistic fashion. Consider the following questions:1. What information is needed to make current decisions about a product or technology?2. How much information can be collected and analyzed., using experiments ,questionnaires, surveys and checklists?3. How accurate will the information be?4. Will the methods get all of the needed information?5. What additional methods should and could be used if additional information is needed?6. Will the information appear as credible to decision makers, e.g., to engineers or top management? 7. How can the information be analyzed?

6. Basic Guidelines to Problem Solving and Decision Making 1. Define the problemIf the problem still seems overwhelming, break it down by repeating steps Prioritize the problems 2. Look at potential causes for the problem 3. Identify alternatives for approaches to resolve the problemAt this point, it's useful to keep others involved. 4. Select method, tool ,technique etc to solve the problem 5. Plan the implementation of the best alternative solution ( action plan) 6. Monitor implementation of the plan 7. Verify if the problem has been resolved or not

7. Complexity Complexity of engineering problem Three types of complexity • Numerical complexity • Description complexity • Understanding (regognation) complexity

8. Complexity clsses(reference-functions Big O notation) : • Logaritmiccomplexity, O(log n) • Linear complexity, O(n) • Polynomial complexity, O( nq ) • Eksponentialcomplexity • Factorialcomplexity • Double-eksponentialcomplexity. Algorithmic complexity is concerned about how fast or slow par ticular algorithm performs. We define complexity as a numerical function T(n) - time versus the input size n. We want to define time taken by an algorithm without depending on the implementation details. But you agree that T(n) does depend on the implementation!

9. Complexity clsses http://www.cs.cmu.edu/~adamchik/15-121/lectures/Algorithmic%20Complexity/complexity.html Asymptotic Notations The goal of computational complexity is to classify algorithms according to their performances. We will represent the time function T(n) using the "big-O" notation to express an algorithm runtime complexity. For example, the following statement T(n) = O(n2) says that an algorithm has a quadratic time complexity. Definition of "big Oh" For any monotonic functions f(n) and g(n) from the positive integers to the positive integers, we say that f(n) = O(g(n)) when there exist constants c > 0 and n0 > 0 such that f(n) ≤ c * g(n), for all n ≥ n0 Intuitively, this means that function f(n) does not grow faster than g(n), or that function g(n) is an upper bound for f(n), for all sufficiently large n→∞

10. Complexity clsses Exercise. Let us prove n2+ 2 n + 1 = O(n2). We must find such c and n0that n2+ 2 n + 1 ≤ c*n2. Let n0=1, then for n ≥ 1 1 + 2 n + n2≤ n + 2 n + n2≤ n2+ 2 n2+ n2= 4 n2 Therefore, c = 4.

11. Complexity clsses Constant Time: O(1) An algorithm is said to run in constant time if it requires the same amount of time regardless of the input size. Example:array: accessing any element Linear Time: O(n)An algorithm is said to run in linear time if its time execution is directly proportional to the input size, i.e. time grows linearly as input size increases. Examples:array: linear search, traversing, find minimumi:=1p:=1for i:=1 to n Ex: Find complexity of the algorithm p:=p*i i:=i+1endfor

12. Complexity clsses Logarithmic Time: O(log n) An algorithm is said to run in logarithmic time if its time execution is proportional to the logarithm of the input size. Example:binary search Quadratic Time: O(n2)An algorithm is said to run in quadratic time if its time execution is proportional to the square of the input size. Examples:bubble sort, selection sort, insertion sort for i:=1 to n for j:=1 to n A(i,j):=x Ex: Find complexity of the algorithm endfor endfor

13. Complexity clsses Ex: Find complexity of the algorithm s:=0 for i:=1 to n for j:=1 to i s:=s+j*(i-j+1) endfor endfor

14. Complexity clsses

15. Complexity for Recursive algorithms Initial problem with data capacity n will be divided into b subproblems with equal capacity. Only a (a<b) subproblems is solved (others not needed). (1) Theorem: Assuming a>=1 ja b>1 are constants, f(n) function and T(n) defined for non-negative n by formula (1). Then: a) T(n) is if f(n) is (e – positive constant) , b) T(n) is if f(n) is c) T(n) is if f(n) is (e – positive constant , and af(n/b)<=cf(n) Ex1: apply theorem to binary search Ex2: apply theorem for problem where a=2, b=4, f(n)=n2+2n+9 Ex3: apply theorem for problem where a=2, b=4, f(n)=3 Ex4: Find number of opertions for a=2, b=4, f(n)=2, n=2000.

16. Protsesside objektide mudelite koostamine kasutades matemaatilise statistika meetode, jaotuse hindamineExcel-Descriptive Statistics

17. Korrelation (Correlation). Seosed juhuslike arvude vahel

18. Example of correlation using Excel Data analysis

19. Katseeeria varieeruvuste erinevuse hindamine, kasutades F kriteeriumit

20. Küsimused seoses kodutööga 1 • Seletada ja võrrelda erinevaid juhuslike suuruste jaotust iseloomustavaid näitajaid • Anda hinnang ja erinevate katsetulemuste omavahelisele seosele • Hinnata erinevate katsetulemuste (sh varieeruvuse) erinevust.

21. MATLAB Statistics Descriptive StatisticsData summariesStatistical VisualizationData patterns and trendsProbability DistributionsModeling data frequencyHypothesis TestsInferences from dataAnalysis of VarianceModeling data varianceRegression AnalysisContinuous data modelsMultivariate MethodsVisualization and reductionCluster AnalysisIdentifying data categoriesModel AssessmentIdentifying data categoriesClassificationCategorical data modelsHidden Markov ModelsStochastic data modelsDesign of ExperimentsSystematic data collectionStatistical Process ControlProduction monitoring

22. Linear regression analysisThe earliest form of regression was the method of least squares, which was published by Legendre in 1805, and by Gauss in 1809.

23. Regression models Regression models involve the following variables: • The unknown parameters, denoted as β, which may represent a scalar or a vector. • The independent variables, X. • The dependent variable, Y. A regression model relates Y to a function of X and β. The approximation is usually formalized as E(Y | X) = f(X, β). To carry out regression analysis, the form of the function f must be specified. Sometimes the form of this function is based on knowledge about the relationship between Y and X that does not rely on the data. If no such knowledge is available, a flexible or convenient form for f is chosen.

24. Linear regression ModelsRegressioonanalüüs vaatluste (katse) plaan

25. Regressioonanalüüs(katseplaan)

26. Regressioonanalüüs(teisendatud katseplaan)

27. Regressiooni analüüsi tulemuste näide

28. Mittelinearse funktsiooni leidmine

29. MATLAB regression analysis Regression Plots, Linear Regression, Nonlinear Regression, Regression Trees, Ensemble Methods

30. Küsimused seoses kodutööga 2 • Valida katsetulemuste kirjeldamiseks sobiv mudel. Esitada regresioonanalüüsi tulemusena saadud mudel • Koostada sobiv „katseplaan“ • Hinnata valitud mudeli sobivust katsetulemuste kirjeldamiseks. Millisel viisil oleks otstarbekas mudelit arendada? • Hinnata leitud mudeli (tema parameetrite) täpsust

31. Full-Factorial Experiments

32. Parameter study

33. One at a Time

34. Latin Hypercubes

35. Design of Experiments (DOF). Katsete planeerimineFactorial Designs

36. 1. Keskpunktiga komposiitne plaan (Central composite design)2. Kolmetasandeline katseplaan ( A Box-Behnken design )

37. Murdfaktoriaalne plaan. Fractional Factorial Designs

38. Parameetri mõju leidmine

39. Katsete planeerimine

40. Närvivõrkude mudelid, mudeli element (neuron) ja ülekandefunktsioonid (McCulloch and Pitts (1943)) f(x) = 1/(1 + e-x)

41. Närvivõrkude mudelid (2)

42. ANN applications Real-life applications. The tasks artificial neural networks are applied to tend to fall within the following broad categories: • Function approximation, or regression analysis, including time series prediction, modeling. • Classification, including pattern and sequence recognition, novelty detection and sequential decision making. • Data processing, including filtering, clusteringand compression. • Robotics, including directing manipulators • Control, including Computer numerical control Application areas include system identification and control (vehicle control, process control, natural resources management), game-playing and decision making, pattern recognition (radar systems, face identification, object recognition and more), sequence recognition (speech, handwritten text recognition), medical diagnosis, financial applications (automated trading systems), data mining (or knowledge discovery in databases ), visualization and e-mail spam filtering.

43. Neural Network Toolbox NN Toolbox provides functions for modeling complex nonlinear systems that are not easily modeled with traditional methods. Neural Network Toolbox supports supervised learning with feedforward, and dynamic networks. It also supports unsupervised learning with self-organizing maps and competitive layers. With the toolbox you can design, train, visualize, and simulate neural networks.

44. Närvivõrkude mudeli loomine • Programmi näide (vaja uus näide): Reinforcement: yes, no • % Lähteandmed • p=[ 3 3 1 2 3 2 1 3 3 2 2 1 2 1 1 2 3 2 1 2; … • 2 1 1 0 2 2 2 0 2 0 1 1 1 0 2 2 1 2 2 0;... • 2 3 2 2 1 2 3 3 1 1 1 0 0 2 3 2 3 1 1 2; … • 2 0 1 2 2 0 1 0 1 1 2 2 2 0 0 2 0 2 1 0;... • 2 3 2 1 1 2 3 3 1 2 2 3 2 1 2 3 2 1 2 3; … • 2 2 2 1 1 1 1 2 2 1 1 2 2 2 1 2 1 2 1 1 ] • t=[1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 0 ] • % Närvivõrgu mudeli kirjeldamine • net =newff(minmax(p),[4,1],{'logsig','logsig'},'trainlm'); • % Muudame treeningu parameetreid • net.trainParam.show=100; • net.trainParam.lr=0.05; • net.trainParam.epochs=3000; • net.trainParam.goal=0.00001; • [net,tr]=train(net,p,t); • % Kontrollime lahendit • a=round(sim(net,p)) • % Väljastame parameetrid • bias1=net.b{1} • bias2=net.b{2} • weights1=net.IW{1,1} • weights2=net.LW{2,1}

45. Tehniliste süsteemide optimaalne projekteerimine Projektlahendite otsimise ja optimeerimise (DSO – Design search and optimization) metoodika areng on kestnud ligi nelikümmend aastat ja käesoleval ajal on projekteerijate käsutuses mitmed võimsad DSO-d toetavad süsteemid. Üha olulisemaks saab vajadus olemasolevaid meetodeid otstarbekalt kasutada. Üldisel juhul võib projektlahendite optimeerimist defineerida kui lahendite otsimist, millised annavad minimaalse (või maksimaalse) väärtuse erinevatele etteantud sihifunktsioonidele ja rahuldavad etteantud piiranguid (tõkkeid). Sihifunktsioonideks on tavaliselt toote maksumus, tarneaeg, toote tasuvus ning mitmed toodete tehnilis-majanduslikke omadusi kirjeldavad suurused. Lahendeid piiravad piirangud toote parameetritele, tootmissüsteemist tulenevad piirangud, kasutada olevad ressursid jm. Modelleerimiseks kasutatavad seosed võivad sealjuures olla esitatud matemaatiliste avaldistena, töömahukate simuleerimistena, või eksperimentide/jälgimiste tulemustena. Projekteerimise optimeerimise tulemus on seda effektiivsem, mida keerukam on lähteprobleem, kuna inimene ei suuda taolises keerulises olukorras otstarbekalt probleemi hinnata ja lahendada. Optimeerimistehnikate kasutamine muutub eriti vajalikuks, kui üksiku komponendi/operatsiooni projekteerimiselt minnakse üle süsteemi (toote või tootmisprotsessi) integreeritud optimeerimisele.

46. Optimeerimisülesannete tüübid (Types of Optimization Tasks) • Eksisteerib teadusharu, operatsiooni analüüs (operation reseacrh), Otsuste teooria. “Operatsiooni analüüs on meetodite kogum , mis on ettenähtud kõige efektiivsemate ja ökonoomsemate otsuste leidmiseks”. Sõltuvalt ülesande iseloomust võib olla tegemist • Optimaalse valiku ülesandega • Planeerimise ülesandega, (süsteemi parameetrite optimeerimisega) Sõltuvalt kasutatavate mudelite iseloomust võib olla tegemist: • Determineeritud ülesandega • Stohhastilise ülesandega või • Ühesammulise otsustusega • Mitmesammulise otsustusega Lahenduse liigist sõltuvalt: • Analüütilise lahendusega • Numbrilise lahendusega jms. Planeerimisel eristatakse: • lineaarset planeerimist • täisarvulist planeerimist’ • mittelineaarset planeerimist • dünaamilist planeerimist • mängute teooriat

47. Optimaalne valik (alternatiivide hulgast) In sciences and engineering , preference refers to the set of assumptions related to ordering some alternatives, based on the degree of satisfaction, or utility they provide, a process which results in an optimal "choice" (whether real or theoretical). 1. Determineeritud valiku ülesanded • Lihtsad , ühe kriteeriumiga valikud • Multikriteriaalsed valikud 2. Stohhastilised (puuduliku informatsiooniga) valiku ülesanded

48. Lihtsad , ühe kriteeriumiga detemineeritud valik,kaks sammu: lubatud lahendite leidmine, parimate lahendite valik

49. Valik puuduliku info tingimustes Kriteeriumid variantidele on juhuslikud suurused • Variante hinnatakse keskmiste järgi • Arvestatakse usaldusvahemikke ja varieeruvust • Arvestatakse korrelatsiooni jm

50. Mitme kriteeriumiga valik Multicriterion choice (decision making) , optimal portfolio selection