Scientific Programming Introduction

1. Dr. Thomas M. Eidson teidson@spsu.edu 678-333-8513 Scientific Programming Introduction

2. In the beginning there was scientific programming. Digital computer hardware was designed to compute numbers and to make logical decisions with relative efficiency. Since computations and data transfers were slow and internal memory was small by todays standards, target problems needed to map efficiently to the computer hardware. As hardware performance and compiler technology improved, the number of computer tasks that were reasonable to compute grew enormously. The importance of efficiency in computing changed from the program execution arena to the programmer productivity arena. The modern focus of programming is to develop applications that support the needs of users in a wide variety of disciplines---research, engineering, business, entertainment, and others. Modern programs can vary from “quick and dirty” solutions to detailed modeling of complex and elaborate systems. Emphasis may be on flexibility for research needs or for modeling a rapidly changing environment. Or, emphasis may be on reliability to support either business requirements or critical missions. Software engineering has developed as a field to develop techniques and tools for programmers who develop applications with these more sophisticated requirements. Overview

3. While general programming requirements also apply to scientific programming to some degree, scientific programs have differences that result in a need for different programming approaches. Scientific programs are often developed to support learning and so they more frequently must be modified. Flexibility is very important. Scientific studies often need high precision. This precision is needed in the physical models, the mathematical models, the computer language implementation, and the computer architecture. Determining which of these sources is the cause of a precision problem can be very difficult. Scientific studies can need very high execution performance. Presentation of computed results can require a great deal of work and study. Data may require sophisticated transformations to find the best view to analyze the output. Sophistication visualization techniques are often needed. Scientific programs can require hours or weeks to complete. They may need checkpoint and restart capability since they may not complete in 1 run. Overview

4. Programming is a technique performed by humans that must deal with precise computer hardware processes, the abstract-level human thinking process, as well as many hybrid precision/abstraction approaches. While computer hardware processes (using von Neumann and Turing models) are well understood, the human thinking process is essentially a black box. While it seems desirable for human-computer interfaces to be similar to the human thinking process, this approach will not be practical until the human thinking process is understand in more detail. Thus, the current approach is for humans to adapt to the way that computers work (imperative programming). The alternative, where computer systems interpret commands given in styles natural to the human thought process (declarative programming) is not currently practical for general usage. What is Programming?

5. Fortunately, many mathematical algorithms map naturally to the imperative programming style. Early programming languages were design to give the programmer amble freedom to implement these algorithms in any style. As the size of the target problems and algorithms grew, the result was that some programmers used (or abused) this freedom to generate unwieldy applications (sometimes called spaghetti codes). While these applications may have been correct, they were so complex that humans had difficultly managing them and compilers had difficulty producing the best optimized executable code. Eventually, to encourage and to support better application development, a formal programming style evolved, structured programming. Structured programming provides the required expressiveness for accurate implementations, yet the resulting code is easier to understand and modify. Also, compilers can be developed that produce code with very good performance. As computational needs grew, many new programming characteristics became more important: very high performance, programming efficiency, correctness, flexibility, organization, understandability, reuse, and others. Sometimes these characteristics conflict with each other, so compromises are often used in practice. Programming Evolution

6. Fortran was the first highly used and standardized language. Fortran has always been focused on natural implementation of mathematical models and execution performance. Eventually, structure programming (which first appeared in ALGOL) was added. Other modern programming characteristics have been added---some enthusiastically (array notation, automatic variables) and some reluctantly (some object-oriented concepts). The C programming language was developed to support system programming and other low-level requirements. C complements Fortran to cover most programming needs but neither provided formal support for modern programming concepts. Parallel (using MPI) and concurrent (using threads or OpenMP) programming styles eventually were developed to enhance the performance of these languages to keep pace with user requirements. Scripting and interpreted languages (Perl, Tk/Tcl, Python, MatLab) also were developed to provide languages to support rapid development and prototyping. Object-oriented programming eventually evolved to include formal support for many of these modern concepts. C++ and Java have emerged as the most commonly used. While many other languages have been developed, the above languages provide sufficient capability for most programming needs. Commonly-Used Languages

7. One other evolutionary development in programming is distributed programming. One type of complex application, a composite application, is composed of multiple element applications, which can be stand-alone or a set of library routines. Some of the element applications may need to run on different computers for a variety of reasons: to get better performance on a computer with a different architecture; to be closer (in terms of network performance) to a data source; to be directly accessible by some person; to be on a computer with a required license; or to be in a secure facility. If these element applications are coupled in the sense that they run simultaneously and share data, the composite application is considered to be distributed. A number of communication systems have been developed to support these applications. They include Globus (Grid computing, focus on scientific applications) and CORBA (business and some military applications). Distributed Programming

8. In summary, scientific programmers may need to know a great deal about the overall computer programming process---from the details of computer architecture to the techniques of using high-level programming languages to skills in data analysis and presentation. Clearly, only a few experts will master all these skills. Modern computer science has evolved to support a layered approach. Each layer of the programming process from the low-level details to the high-level languages is mastered and supported by different people. Most programmers, who are tasked with supporting business productivity, can focus only on the high level and will produce good applications. Scientific programmers often need an understanding of more levels than they can typically master, they need to be judicious about what they learn. A typical approach is to master a middle-level language (C or Fortran). The scientific programmer then learns enough about the details about languages below and above this level to accomplish the goals at hand. Summary