Media Newsletters - April 2006

Distributed Computing Speeds Application Development for Technical Computing

by Silvina Grad-Freilich, The MathWorks

As requirements for technical computing applications become more complex and development time shrinks, engineers and scientists must solve problems of increasing computational intensity, many involving huge data sets. Domain experts report that the computational intensity of their problems exceeds the capabilities of their computers, so they increasingly rely on high-performance computing environments to speed application development.

In the past, high-performance computing was available mainly to government agencies and big research labs, who could afford to purchase supercomputers. Today, most supercomputers have been replaced by commercial-off-the-shelf (COTS) computer clusters that provide affordable, high-performance distributed environments. Fifteen years ago a Cray supercomputer cost about $40 million and provided a performance of about 10 gigaflops (10 billion instructions per second). In 1998, the same performance from a Sun HPC server cost about $1 million. Today, a four-processor Dell configuration provides the same level of performance for only $4,000.

Problems with Traditional Workflows

Despite this technology’s availability and low cost, the complexity of developing software applications for distributed environments still creates a substantial practical barrier. Most domain experts work in high-level languages, such as MATLAB or Mathematica. Until recently, the vendors of these high-level computing languages did not offer high-level tools for prototyping and developing technical computing applications for distributed environments. As a result, technical computing applications developed in high-level languages had to be rewritten in C or Fortran for distributed computing, an arduous and time-consuming task for which many engineers and scientists do not possess the required skills. Also, few C or Fortran programmers have the necessary domain experience.

Also, converting an application from a high-level language to a low-level language results in two versions of the same application. If requirements change after this point, it is difficult and inefficient to go back to the original high-level language version. Also, if you implement code changes in the low-level program, you lose the advantages of working in the intuitive, high-level environment.

Moreover, to achieve maximum performance, programmers usually tune applications written in a low-level language to the specific hardware where they run. So if the hardware changes, as it frequently does, you have to retool the program. The processing time you gain with distributed computing is thereby offset by the time you spend retooling the program to run in a distributed environment.

These problems could be avoided if the high-level languages that domain experts use directly supported a distributed computing environment. Domain experts could then prototype and develop distributed applications from within one environment, with considerable reductions in time and cost. These applications could then be redeployed to any hardware platform the high-level language supports, with less frustration and fewer errors. While a program written and implemented in a high-level language might perform less efficiently than a program written by an expert C or Fortran programmer, the reduction in time-to-solution far outweighs the minor loss of computing speed.

Distributed Computing Toolkits

Several third-party toolkits for high-level languages are already available for developing distributed and parallel applications. In a recent survey, the Supercomputing Technologies Group at Massachusetts Institute of Technology (http://supertech.lcs.mit.edu/~cly/survey.html) identified almost 30 toolkits for use with MATLAB. Most were developed for research projects and are in the public domain. Few are commercially supported, which means that no technical support or help is available. Most are no longer being developed, and so do not work with the latest versions of MATLAB and associated application-specific toolboxes. While the toolkits themselves are often free, most require that each computer in the distributed computing environment have a full license for MATLAB and associated toolboxes, raising the cost of ownership.

The MathWorks has responded by providing products to enable domain experts to prototype and develop distributed computing applications in MATLAB. Initial releases supported coarse-grained, or embarrassingly parallel applications in which the same algorithm runs independently on several nodes, without communication, shared data, or synchronization points between the nodes. Sample applications include parameter sweeps and Monte Carlo simulations. The current version addresses finer-grained applications containing interdependent tasks that require communication between tasks during processing. It implements key Message Passing Interface (MPI) function calls, the industry-wide standard protocol for point-to-point communication in a parallel program.

These products enable users to prototype and develop distributed application interactively and to access distributed computing resources at development time. Version 2 adds support for pluggable schedulers, such as LSF from Platform Computing, which also help users to send applications to a batch queue and receive resources for execution only when the scheduler allocates them. Pluggable schedulers provide support for integrating the distributed computing tools into users’ existing distributed computing environments.

Simplifying the Parallelization Task

Using MathWorks distributed computing tools, domain experts can develop applications in MATLAB and then divide them into tasks that are evaluated remotely on cluster nodes. In the simplest case, where you can divide a problem into tasks consisting of the same function with the same number of input and output variables, a single function call parallelizes the problem for a distributed computing environment -- so one line of code is sufficient. More complex cases require a few functions, or several lines of code.

Distributed computing tools can execute algorithms that include any toolbox for which a user is licensed, so users do not need to purchase additional toolbox licenses for the cluster nodes. As a result, the cost of ownership is a fraction of what it would be with third-party toolkits, which require purchasing licenses of each toolbox for every node where it might be used.

Just as COTS clusters have reduced the cost of distributed computing, so will programs written by domain experts in high-level languages reduce the time and effort required for application development. This shift is being driven by users who expect not only the performance of high-performance computing applications but also their programmability and portability. High-productivity programming tools can dramatically reduce the time-to-solution by enabling domain experts to develop their own distributed computing applications. The more prevalent these tools become, the more companies will invest in hardware, software applications, and training required to support the development of distributed computing applications.