gorge - Online in the Cloud

PROGRAM:

NAME

spew - measures I/O performance and/or generates I/O load

SYNOPSIS

spew --write|--read|--read-after-write [OPTION]... TRANSFER_SIZE[kKmMgG] FILE

DESCRIPTION

An I/O performance measurement and load generation tool. Writes and/or reads generated
data to or from a character device, block device, or regular file.

-B, --max-buffer-size=BUFFER_SIZE
Each read(2)/write(2) call uses a maximum buffer of size BUFFER_SIZE.

-b, --min-buffer-size=BUFFER_SIZE
Each read(2)/write(2) call uses a minimum buffer of size BUFFER_SIZE.

-c, --continue-after-error
Continue after data integrity errors.

-d, --direct
Use direct I/O. Should only be used on block device files. Not all operating
systems support direct I/O

-g, --generate-load
Equivalent to: -v -t -P -p random -i 0.

-i, --iterations=COUNT
Write/read data COUNT times. If count is 0, repeats forever.

-l, --logfile=LOGFILE
Send log messages to LOGFILE.

--no-progress
Don't show progress (default).

--no-rcfiles
Don't use standard rcfiles.

-q, --no-statistics
Don't output statistics.

--no-tui
Don't use TUI interface.

-o, --offset=OFFSET
Seek to OFFSET before starting I/O.

-P, --progress
Show progress.

-p, --pattern=PATTERN
Use data pattern PATTERN when reading or writing data.

-r, --random
Read/Write buffers to random offsets.

--raw An alias for --read-after-write.

--rcfile=RCFILE
Read command-line options from RCFILE.

--read Read date from FILE.

--read-after-write
Read back data after writing to FILE.

-S, --seed=SEED
Use SEED for random number seed.

-s, --sync
Use synchronous I/O.

--statistics
Output statistics (default).

-t, --tui
Use curses-based, terminal user interface.

-u, --units=UNITS
Show transfer rate in UNITS units.

--usage
Show brief usage message and exit.

-V, --version
Output version information and exit.

-v, --detailed-statistics
Output detailed statistics.

--write
Write data to FILE.

-?, --help
Show this help and exit.

FILE Regular or device file to write data to.

LOGFILE
Path to a file used for logging.

MAX_BUFFER_SIZE
Minimum buffer size used in each read(2)/write(2) call (default is MIN_BUFFER_SIZE
bytes). MAX_BUFFER_SIZE. Must be an even multiple of 512 bytes and can be
specified in bytes, kibibytes(k), kilobytes(K), mebibytes(m), megabytes(M),
gibibytes(g), gigabytes(G). tebibytes(t), or terabytes(T). MAX_BUFFER_SIZE
defaults to MIN_BUFFER_SIZE. If MAX_BUFFER_SIZE > MIN_BUFFER_SIZE, random buffers
sizes between the two limits are used. MAX_BUFFER_SIZE must be an even multiple of
MIN_BUFFER_SIZE.

MIN_BUFFER_SIZE
Minimum buffer size used in each read(2)/write(2) call (default is 512 bytes).
MIN_BUFFER_SIZE. Must be an even multiple of 512 bytes and can be specified in
bytes, kibibytes(k), kilobytes(K), mebibytes(m), megabytes(M), gibibytes(g),
gigabytes(G). tebibytes(t), or terabytes(T).

OFFSET Position to seek to in the file before starting I/O (default is 0). OFFSET must be
an even multiple of 512 bytes and can be specified in bytes, kibibytes(k),
kilobytes(K), mebibytes(m), megabytes(M), gibibytes(g), gigabytes(G).
tebibytes(t), or terabytes(T).

PATTERN
Data pattern used when writing/reading data. Available patterns are: none, zeros,
ones, alt, random, numbers, and "#" (where "#" is a number between 0-255). The
default pattern is "none".

RCFILE Read additional command-line options from RCFILE. Other options on the
command-line will override options in RCFILE.

SEED Used to seed the random number generator Must be >= 1 and <= 2^32.

TRANSFER_SIZE
Total number of bytes to transfer (must be an even multiple of both MIN_BUFFER_SIZE
and MAX_BUFFER)SIZE). TRANSFER_SIZE can be specified in bytes, kilobytes,
megabytes, or gigabytes.

UNITS Kibibytes(k), kilobytes(K), mebibytes(m), megabytes(M), gibibytes(g), gigabytes(G).
tebibytes(t), or terabytes(T).

EXAMPLES

spew --write -b 16k 1m /tmp/bigfile

or

spew -b 16k 1m /tmp/bigfile

Writes 1 mebibyte (1 mebibyte = 1024*1024 bytes) using 16 kibibytes (1 kibibyte = 1024
bytes) requests to the file /tmp/bigfile using the default pattern (random). Displays the
write transfer rate in kibibytes per second and the write transfer time in seconds.

spew --write -u m -i 10 -b 1k 256k /dev/sda1

or

spew -u m -i 10 -b 1k 256k /dev/sda1

Writes 256 kibibytes using 1 kibibyte requests to the block device file /dev/sda1 10 times
using the default pattern (random). The iteration and cumulative write transfer rates are
displayed in mebibytes per second and the iteration and cumulative write transfer times
are displayed in seconds.

spew --raw -d -o 1m -b 16m 1g /tmp/bigfile

or

regorge -s -o 1m -b 16m 1g /tmp/bigfile

Write 1 gibibyte (1 gibibyte = 1024*1024*1024 bytes) starting at an offset of 1 mebibyte
using 16 mebibyte requests to the file /tmp/bigfile using the default pattern (random).
The data is written synchronously and flushed at file close. Then read in the same data
using the same request size and offset. The data is checked to ensure that the data read
in matches the data read out. Write and read transfer rates are displayed in
kibibytes/second. Read and write transfer times are displayed in seconds.

spew --read -i 0 -u M -p zeros -b 512 1m /dev/zero

or

gorge -i 0 -u M -p zeros -b 512 1m /dev/zero

Read 1 mebibyte of data using 512 byte requests from the file /dev/zero an infinite number
of times using the zeros pattern (don't check the data). The iteration and cumulative
read transfer rates are displayed in megabytes (1 megabyte = 1,000,000 bytes) per second
and the iteration and cumulative read transfer times are displayed in seconds.

spew --raw -g -r -b 1k -B 256K 1t /dev/md1

or

regorge -g -r -b 1k -B 256K 1t /dev/md1

Write 1 tebibyte (1 tebibyte = 1024*1024*1024*1024 bytes) using 1-256 kibibyte requests to
the block device /dev/md1 using the random pattern. Random seeks are performed before
each transfer, but each block between the start and end of the data is written exactly
once. The request sizes are chosen randomly. Then read in the same data using the same
request sizes and seeks in the same sequence. Repeat the above sequence an infinite
number of times until told to quit (via signal or TUI command).

The data is checked to ensure that the data read in matches the data read out. A curses-
based TUI is used to display iteration and cumulative transfer rates, transfer times, and
bytes transferred. Display verbose statistics after quitting.

ALGORITHMS

The following algorithm is used to determine blocksize when MAX_BUFFER_SIZE >
MIN_BUFFER_SIZE:

Find the largest blocksize that is a power of 2 and is >= MAX_BUFFER_SIZE. Call this
maxbuffersize. Find the smallest buffersize that is a power of 2 and is >=
MIN_BUFFER_SIZE. Call this minbuffersize. Chunksize = maxbuffersize * (difference in
number of bits between min and maxbuffersize + 1). The number of chunks is
TRANSFER_SIZE/chunksize. Call this numchunks. There will be numchunks number of
read/writes that are of size maxbuffersize, 2 * numchunks number of read/writes that are
of size maxbuffersize/2, 4 * numchunks number of read/writes that are of size
maxbuffersize/4, and so on until minbuffersize is reached. MIN_BUFFER_SIZE is used for
any remaining transfers if TRANSFER_SIZE is not evenly divisible by chunksize.

Example: Using a MIN_BLOCK_SIZE of 1k, a MAX_BLOCK_SIZE of 8k and a TRANSFER_SIZE of 264k
results in 8 8k read/writes, 16 4k read/write, 32 2k read/writes, and 64 + 8 1k
read/writes.

PATTERNS

Use the following od(1) command to examine spew generated files where a data integrity
problem has occurred:

od -A d --format=d8 --width=32 <file>

Use gorge online using onworks.net services