Next | Section 5: Analyzing CPU-bound code.
Return | Introduction and Table of Contents

4.0 Optimizing I/O-bound Code

This chapter describes specific techniques for optimizing I/O-bound code. To determine which of these techniques to use for I/O-bound code, see Section 3. The following techniques are described in this chapter:

• 4.1 Optimizing formatted I/O
• 4.2 Optimizing large, sequential, unformatted I/O requests
• 4.3 Optimizing small, sequential, unformatted I/O requests
• 4.4 Optimizing for direct access I/O
• 4.5 Optimizing asynchronous I/O
• 4.6 Using an optimal storage device
• 4.7 Minimizing system calls

4.1 Optimizing formatted I/O

Formatted I/O is the slowest I/O and is useful only when the files must be viewed by people or transferred to systems other than Cray Research systems. However, if you are transferring the data to a system other than a Cray Research system, you can easily send the unformatted (binary) version instead of the formatted ASCII version by using the Cray foreign file conversion facility provided by the Flexible File I/O (FFIO) library. Use the techniques described in the following sections to optimize code that contains formatted I/O.

4.1.1 Changing to unformatted I/O

• In Fortran code, remove references to the FORMAT statement label and modify the Fortran OPEN statement to include FORM='UNFORMATTED'.

  Example:

  
  OPEN (10,FORM='UNFORMATTED')
  

• For C++ codes, cout < < and cin > > are formatted read and write member functions of the iostream class. Also, the scanf(3) and printf(3) function calls (including fscanf, sscanf, fprintf, and sprintf) require formatting to human- readable ASCII. Convert these functions to call unformatted I/O functions such as fread and fwrite instead. You can access FFIO by using the ffread(3) and ffwrite(3) functions in your code in conjunction with the UNICOS assign(1) command.
• To access the I/O layers provided by the FFIO libraries, use the -F command- line option with the UNICOS assign command. This will provide access to the automated foreign file conversion.

4.1.2 Reducing the amount of formatted I/O

• Change the code to show final results instead of many intermediate results.
• Change the code to show a checksum instead of the data itself.
• If the program sends data to another computer system (or printer), revise the program so that only the final version of the data is formatted.
• If you need to view the data, consider shipping it unformatted to a graphics postprocessor.

4.1.3 Increasing formatted I/O efficiency for Fortran programs

4.1.3.1 Minimizing the number of data items in the I/O list

DIMENSION X(20), Y(10), Z(5,30) 
WRITE (6,101) Y, (X(I), I=1,20), Z(M,J)

With vectorization turned off, this WRITE statement represents 22 data items. In this case, the WRITE operation would require 22 calls to the library routines that drive the WRITE statement. When vectorization is turned on, the compiler treats each innermost implicit DO loop as a single data item, so that the preceding WRITE statement requires only 3 calls.

If you rewrite the statement as follows, the parameter list always represents 3 calls, even if all optimization is turned off:

WRITE (6,101) Y, X, Z(M,J)

4.1.3.2 Using a single READ, WRITE, or PRINT statement

    DO J = 1, M
    DO I = 1, N 
    WRITE (42, 100) X(I,J) 
100 FORMAT (E25.15) 
    ENDDO 
    ENDDO

It is more efficient to write the entire array with a single WRITE statement, as follows:

    WRITE (42, 100) ((X(I,J),I=1,N),J=1,M) 
100 FORMAT (E25.15)

The following statement is even more efficient:

    WRITE (42, 100) X 
100 FORMAT (E25.15)

Each of these three code fragments produce exactly the same output; however, the latter two examples are about twice as fast as the first. Also, the latter two examples are equivalent only if the implied DO loops write out the entire array, in order, and without omitting any items. You can use the format to control how much data is written per record.

4.1.3.3 Using longer records

    WRITE (42, 100) X 
100 FORMAT (E25.15)

If you change it as follows, the resulting file will have 80% fewer records and, more importantly, the program will execute faster:

WRITE (42, 101) X 
101 FORMAT (5E25.15)

Be careful to ensure that the resulting file does not contain records that are too long for the intended application. For example, certain text editors and utilities cannot process lines that are longer than a predetermined limit. Generally, lines that are not longer than 128 characters are safe to use in most applications.

4.1.3.4 Using repeated edit descriptors

200 FORMAT (16(X,I4))

Instead, use a format of the following form:

201 FORMAT (16I5)

4.1.3.5 Using data edit descriptors that are the same width as the character data

4.1.4 Increasing formatted I/O efficiency for C++ programs

for (i=0; i < N;) {
    fprintf(o1,"%d ",a[i];
    ++i;
    if (i%5 == 0) fprintf(o1,"\n"); 
}

If you change it as follows, the resulting code will make 80% fewer calls to fprintf and, more importantly, the program will execute faster:

for (i=0; i < N; i+=5) {
   fprintf(o2, "%d %d %d %d %d\n"
      a[i], a[i+1], a[i+2], a[i+3], a[i+4]); 
}

4.1.5 Increasing library buffer sizes for formatted I/O requests

assign -b sz

For C++, use the setvbuf(3) library function.

4.2 Optimizing large, sequential, unformatted I/O requests

Sequential access indicates that data items in a file have an implicit order. Unless the code issues positioning requests such as fseek(3) or rewind(3), the system always accesses the next record automatically. If the code is issuing sequential, unformatted I/O requests larger than 1 Mword, use the techniques described in the following sections to optimize its I/O.

4.2.1 Changing I/O file format to unbuffered and unblocked

If the code is issuing sequential, unformatted I/O requests larger than 1 Mword (8 Mbyte), change the I/O file format to unbuffered and unblocked by using the -s u option, or by specifying the FFIO system, or syscall layer, as shown in the following assign(1) command examples:

assign -s u f:filename
assign -F system f:filename
assign -F syscall f:filename

C++ codes can access the FFIO libraries by using the ffread(3) and ffwrite(3) I/O function calls in conjunction with the UNICOS assign command.

Using unbuffered, unblocked I/O file format requires you to construct well-formed I/O requests in the code. These are simply I/O requests that begin and end on disk sector boundaries, usually 512 words (4096 bytes) or a multiple of 512 words. This unit of measurement is also known as a UNICOS block or click. See your system administrator to determine the sector size of the disks you are using.

4.2.2 Converting to asynchronous I/O

• Repetitive patterns of input, computation on that data, output, then input again
• I/O that appears in a loop

Most prominent sequential, unformatted I/O requests that consume a majority of the code's elapsed time will benefit from code conversion to asynchronous I/O. You can convert to asynchronous I/O by using the assign(1) command or by modifying your source code.

4.2.2.1 Using the assign command to convert code to asynchronous I/O

assign -F cachea:bs:nbufs f:filename 
assign -F bufa:bs:nbufs f:filename

The bs argument specifies the size in 512-word blocks of each cache page or buffer. The nbue argument specifies the number of cache pages or buffers to use. You can tune these arguments to better suit the I/O activities of the code.

If the code requires the use of COS blocked format, you can establish a specialized FFIO layer to provide asynchronous access by using the following UNICO assign command:

assign -F cos.async f:filename

4.2.2.2 Modifying source code to convert code to asynchronous

With double-buffering, two sets of data (buffers) are active at any given moment for each stream of input or output data. One buffer is active in CPU work, while the other is active in I/O (reading or writing). In a typical double buffer scheme, the I/O and CPU work sets are staggered, as in the following algorithm:

1. The first set of input data is read.

2. The second set of input data is read while the CPU works on the first set of input data.

3. The third set of input data is read while the CPU works on the second set of input data and the first set of data is output.

4. This sequence continues until all data is read. As the last data set is read, the next-to-last CPU work is in progress, and the third-from-last data set is output.

5. The CPU works on the last data set and the next-to-last data set is output.

6. The final data set is output.

The following Fortran 90 example shows a double-buffering code example with the older alternatives to the cachea and bufa FFIO layers, BUFFERIN and BUFFEROUT. The first input is the BUFFERIN statement before the DO loop. Inside the loop, each BUFFERIN statement synchronizes the previous BUFFERIN statement, and each BUFFEROUT statement synchronizes the previous BUFFEROUT statement. This is called blocking asynchronous I/O, because each request to the same unit blocks execution until the previous request is complete. The last BUFFERIN statement is synchronized by the call to UNIT in the last iteration (I.EQ.M).

PROGRAM DBUF 
PARAMETER (N=1001472,M=1000) 
REAL A(N,0:1), B(N,0:1) 

CALL ASNUNIT (10,'-s u',IERR) 
CALL ASNUNIT (11,'-s u',IERR) 
IRD=0 
BUFFERIN (10,0) (A(1,IRD),A(N,IRD)) 
DO 10 I=1,M 
IWK=IRD 
IRD=MOD(IRD+1,2) 
IF (I.NE.M) BUFFERIN (10,0) (A(1,IRD),A(N,IRD)) 
IF (I.EQ.M) FERR=UNIT(10) 
CALL WORK(A(1,IWK),B(1,IWK)) 
BUFFEROUT (11,0) (B(1,IWK),B(N,IWK)) 
10 CONTINUE 
END

4.2.2.3 Using effective library buffer sizes for large, sequential, unformatted I/O

assign -b sz f:filename

For C++, use the setvbuf(3) library function.

4.3 Optimizing small, sequential, unformatted I/O requests

If the code is issuing sequential, unformatted I/O requests that are 1 Mword or smaller, use the techniques described in the following sections to optimize I/O.

4.3.1 Using effective library buffer sizes for small, sequential, unformatted I/O requests

4.3.2 Increasing I/O request size and issuing fewer requests

• Read or write larger array sections instead of one element at a time.
• Combine read requests and write requests into a single read request or single write request.
• Extract I/O from inner loops.

4.3.3 Using the memory-resident (MR) FFIO layer

4.4 Optimizing techniques for direct access I/O

Direct access indicates that a program can access records or data at any point in the file. This also can be called nonsequential or random access I/O.

4.4.1 CF90 direct access I/O

CF90 direct access example

OPEN (22,ACCESS='DIRECT',RECL=8000) 
READ (22,REC=10) (DATA(I),I=1,1000) 
WRITE (22,REC=2) (OUTNUM(J),J=1,150)

4.4.2 C++ direct access I/O

C++ direct access example

stream = fopen ("file","r+"); 
 
bytes_per_word = 8; 
nwords = 1000; 
 
lrec = bytes_per_word*nwords; 
fseek (stream,9*lrec,SEEK_SET); 
fread (data,bytes_per_word,nwords,stream); 
fseek (stream,1*lrec,SEEK_SET); 
fwrite (outnum,bytes_per_word,150,stream); 
fseek (stream,  lrec - bytes_per_word*150,SEEK_CUR); 
fread (data,bytes_per_word,nwords,stream);

4.4.3 Optimizing techniques for direct access code

• Ensure that the files are in binary file format and that they bypass the system cache by using the assign -s bin command.

• Ensure that the code is not using formatted or COS blocked file formats.

• Set the library buffer size as close to the length of a record (request) as possible without going under the length. This minimizes unnecessary data transfers, which are not useful for random I/O.

• For small, random I/O requests, use a smaller library buffer than the default. Limit its size to the record length of the code. This might improve performance by avoiding excessive unused data movement when filling the unused portion of the buffer.

• For large I/O requests, the library buffer size should be set equal to the length of the fixed-size record (request). To specify the library buffer size for Fortran, use the assign -b sz command. For C++, use the setvbuf(3) library function.

• If the code makes repeated references to the same place in the data file, a memory-resident (MR) buffer might help if it can include the most frequently used area of data. For information on the MR layer, see Section 4.6.

• If the code uses word-addressable data, you can transfer the data faster with a binary file format (using the assign -s bin command), which also bypasses the system cache and forces use of the GETWA and PUTWA I/O routines without changing the source code. The GETWA and PUTWA I/O routines are among the fastest types of random-access I/O on Cray PVP systems, but they place the burden of record keeping and indexing on you.

• Rearrange the data file so that the code can process it sequentially. Sequential I/O is usually faster than direct-access I/O. You might be able to use separate files to accomplish the same effect.

4.5 Optimizing asynchronous I/O requests

In most code, synchronous I/O is used more often than asynchronous I/O (also known as raw I/O) is used. Synchronous I/O indicates that control is returned to the calling program after all requested data is transferred. The I/O transfer runs serially with respect to the CPU work.

Asynchronous I/O indicates that control is returned to the calling program after the I/O process has started, but before the I/O is completed. The I/O transfer runs in parallel with respect to the CPU work. The user program continues executing at the same time the I/O operation is executing.

If the code is using asynchronous I/O, use the techniques described in the following sections. Some of these methods increase CPU overhead but decrease total elapsed time if there is significant work to do during the I/O transfer.

4.5.1 Using unblocked file format for asynchronous I/O requests

For example, the following assign statements specify the unblocked file structure:

assign -s unblocked f:filename 
assign -s u f:filename 
assign -s bin f:filename

Setting the library buffer size to an even number greater than 63 blocks causes COS blocked files to perform double-buffered asynchronous I/O by dividing the library buffer in half. When the library buffer size is an even number of disk sectors, each half of the buffer is well-formed. Thus, I/O requests for either half-buffer do not need to be rerouted through the system cache.You can change the buffer size by using the SEGLDR directive SET, as follows:

SET=_def_cos_thrsh:size

You can also change the buffer size by using the assign(1) command to specify a special FFIO layer, as follows:

assign -F cos.async:size f:filename

4.6 Using an optimal storage device

If you have some flexibility with the storage devices your code uses, ensure that it uses the fastest devices available for the appropriate situations. The following sections describe storage devices and the situations in which they are best used.

4.6.1 Memory-resident files

4.7 Minimizing system calls

With few exceptions, system calls are required for all physical I/O requests and data movement to or from the library buffer. The following options minimize system calls:

• Ensure that I/O requests are as large as possible. For example, write whole arrays rather than one row at a time. Group multiple arrays into one write statement.

• Use larger buffers (or user cache) to capture many I/O requests in the user process space before the I/O library transfers the data out.

• Use scratch files for intermediate data that you no longer need after the code completes execution. This can eliminate unnecessary data movement and might avoid the device entirely.

• Scratch files are temporary and are deleted when they are closed. To create a Fortran scratch file, open a file with STATUS='SCRATCH' and use STATUS='DELETE'.

• Use the MR layer when appropriate (see Section 4.6).

• Use SDS if available (see Section 4.6).

    Next | Section 5: Analyzing CPU-bound code.
Previous | Section 3: Analyzing I/O-bound code
  Return | Introduction and Table of Contents