Overview

The 3D Raster Volume Library is used for the r3.* and vector volume tools. The library uses a tile cache based approach to store floating point values in abritrary order in a volume. The coordinate system of a volume is column and row compatible to the raster library and counts from the bottom to the top of the cube.

The volume coordinate system and tile layout of the 3D Raster Library

TODO: update for GRASS7

Directory Structure

The file format consists of a mapset element grid3 which contains a directory for every map. The elements for each map are

        3d region file
        color file (color)
        categories file (cats)
        range file (range)
        timestamp file /* not yet implemented */
        cell file (cell)
        header file (cellhd)
        a directory containing display files (dsp)

There is also a colr2 mechanism provided. colr2 color tables are stored in grid3/colr2/MAPSET/MAP.

Note: color, categories, and the range can be used in the same way as in 2d GRASS with the exception of reading and writng. 3d read and write functions have to be used for this purpose.

Data File Format

Cell-values can be either double or float.
Values are written in XDR-format.
NULL-values are stored in an embedded fashion.
The cell-values are organized in 3d-tiles.
The tile dimensions can be chosen when a new map is opened.
Every tile of a map has the same dimension except those which overlap the region boundaries.
Compression is used to store tiles.

The data file has the following format:

        xdr_int nofBytesLong;
        xdr_int nofBytesUsed;
        encoded_long indexOffset;
        compressed_tile[] tiles;
        compressed_encoded_long[] index;

Transportability of data file

All numbers stored in the data file are either XDR-encoded or encoded by some other method (for variables of type long only).

Tile Data NULL-values

RASTER3D uses the same functions as 2d GRASS to set and test NULL-values. The storage in the file is different though. NULL-values are stored with a special bit-pattern if maximum precision is chosen. They are stored by adding an additional bit if the precision is smaller.

Tile Data Compression

There are three methods of compression provided. The compression methods can either be those defined by default, set by environment variables or explicitly set at run-time.

        Precision
        RLE

Precision indicates how many of the mantissa bits should be stored on file. This number can be any value between 0 and 23 for floats and between 0 and 52 for doubles. Choosing a small precision is the most effective way to achieve good compression.

RLE takes advantage of possible repetitions of the exponents and the NULL-bit structure. Using RLE does not significantly increase the running time. If for some tile the non-RLEed version is smaller in size, RLE is not used for this tile.

The default and suggested setting is to use precision and RLE.

Additional compression is achieved by storing the extra NULL-bit in a separate bit-array. Using this scheme NULL-values need not actually be represented in the array of cell values. This array is stored together with the cell-values of the tile.

Tile Cache

Tiles can either be read and written directly or use an intermediate cache instead.

In non-cache mode the application should only use the functions

int Rast3d_readTile()

and

int Rast3d_writeTile()

to read and write tiles. The application can use one tile provided by the map structure as buffer. See Rast3d_getTilePtr().

In cache mode the application can access cell-values directly by their coordinates. The corresponding functions are

int Rast3d_getValue()

and

int Rast3d_putValue()

and their corresponding typed versions.

If the map is new then in addition to the memory-cache a file-cache is provided. This allows the application to write the cell-values in any arbitrary order. Tiles are written (flushed) to the data-file either at closing time or if explicitly requested.

If the map is new Rast3d_getValue() can be used even if the tile which contains the cell has already been flushed to the data file. In this case the tile is simply read back into the memory-cache from the data file.

Explicitly flushing tiles can have the advantage that less disk space is occupied since tiles are stored in a uncompressed fashion in the file-cache. Flushing tiles explicitly can cause problems with accuracy though if precision is less than the maximum precision and an already flushed value is used for computations later in the program.

The type of the cell-values of the tiles in memory can be chosen independently of the type of the tiles in the file. Here, once again one has to consider possible problems arising from mixing different precisions.

As an example consider the case where the data is stored in the file with double precision and the tiles are stored in memory in single precision. Then using Rast3d_getValue() will actually return a double precision number whose precision is only 23 bits. It is therefore a good idea to use the types in the memory consistently.

Header File

The header file has the following format:

Proj: 1
Zone: 1
North: 2.0000000000000
South: 0.5000000000000
East: 4.0000000000000
West: 3.0000000000000
Top: 6.0000000000000
Bottom: 5.0000000000000
nofRows: 30
nofCols: 20
nofDepths: 14
e-w resol: 0.05
n-s resol: 0.05
t-b resol: 0.071428571
TileDimensionX: 8
TileDimensionY: 8
TileDimensionZ: 8
CellType: double
useCompression: 1
useRle: 1
Precision: -1
nofHeaderBytes: 12
useXdr: 1
hasIndex: 1
Units: none
VerticalUnits: 1

Except for the first 14 fields the entries of the header file should not be modified. The precision value -1 indicates that maximum precision is used.

Binary files not in RASTER3D format can be read by the library. The following actions have to be taken:

Make a new map directory in the grid3 element of the mapset (say mymap). Copy the file into mymap/cell and generate a header file mymap/cellhd.

In the following example the relevant values of mymap/cellhd are shown:

TileDimensionX: A
TileDimensionY: B
TileDimensionZ: C
useCompression: 0
useRle: 0
Precision: -1
nofHeaderBytes: X
useXdr: 0
hasIndex: 0

The values of A, B, and C have to be chosen according to one of the following patterns:

A &gt;= 1, B == 1, C == 1, or
A &gt;= nofRows, B &gt;= 1, C == 1, or
A &gt;= nofRows, B &gt;= nofCols, C &gt;= 1.

A larger tile size reduces the number of tile-reads. If in the third pattern C is chosen larger than or equal to nofDepths, the entire region is considered one large tile.

The value nofHeaderBytes indicates the offset in the file to the first data entry.

For performance reasons it is a good idea to use function Rast3d_retile() before using the file in other applications.

Region Structure

typedef struct{

    double north, south;
    double east, west;
    double top, bottom;
  
    int rows, cols, depths;/* data dimensions in cells */

    double ns_res, ew_res, tb_res;

    int proj;  /* Projection (see gis.h) */
    int zone;  /* Projection zone (see gis.h) */

} RASTER3D\_Region;

Windows

Window capability similar to that of 2d GRASS is provided (compare Region). Additional features are the window for the third dimension as well as the possibility to choose a different window for every map. The window can be specified at the time of opening an old map. It can be modified at any time later in the program. The resampling method can be the default nearest neighbor method as well as an application provided method.

The default 3d window file is WIND3 located in the mapset. Application programs should use Rast3d_useWindowParams() to allow the user to overwrite this default.

The window file has the following format:

Proj: 1
Zone: 1
North: 2.0
South: 0.5
East: 4.0
West: 3.0
Top: 5.0
Bottom: 6.0
nofRows: 30
nofCols: 20
nofDepths: 14
e-w resol: 0.05000000000000000
n-s resol: 0.05000000000000000
t-b resol: 0.07142857142857142

Note: after reading the window file the fields e-w, n-s, and t-b are recomputed internally.

A note about windows and caching. Caching is performed on the level of tiles read from the file. There is no caching performed on resampled data. This is different from 2d GRASS since resampling for a specific value is performed every time it is being accessed.

Masks

RASTER3D provides a mask for the 3d region. The mask structure is automatically initialized at the time the first file is opened. The same structure is used for all the files. The default for every file is that the mask is turned off. If masking should be performed, the application program has to turn on masking explicitly. If masking is turned on for a file, the cell-values of a tile are automatically checked against the mask. Values which are masked out, are set to NULL.

Note: changing the status of masking after one or more tiles have already been read does not affect the tiles which are already stored in the cache.

Any arbitrary 3d raster map can be used as mask file: NULL-values are interpreted as "mask-out", all other values are interpreted as "don't mask out". Using r3.mask to convert a 3d raster map into a mask file instead of simply copying (or renaming) the directory will significantly reduce to amount of disk space and the access time for the mask.

Include File

Exported RASTER3D constants and structures can be found in G3d.h.

RASTER3D Defaults

There are three methods to set default variables. First, the default can be set at compile time in g3ddefault.c. This value has lowest priority.

Second, the default can be set via an environment variable. Third, the value can be set explicitly set at run time. This value has highest priority.

There are also functions provided to query the value.

Cache Mode

Limiting the maximum cache size

The limit is specified in bytes. It is a limit on the size of cell-data stored in the cache and does not include the support structure.

Default RASTER3D_CACHE_SIZE_MAX_DEFAULT. This is currently set to 16meg and can be changed at compilation time of the library.

Environment variable RASTER3D_MAX_CACHE_SIZE.

void Rast3d_setCacheLimit(int nBytes)Set cache limit

int Rast3d_getCacheLimit(int nBytes)Get cache limit

Setting the cache size

This value specifies the number of tiles stored in the cache. It is the value used if at opening time of a map RASTER3D_USE_CACHE_DEFAULT is used for the cache mode. Any other value used at opening time will supersede the default value. A default value of 0 indicates that non-cache mode should be used by default.

Default RASTER3D_CACHE_SIZE_DEFAULT. This is currently set to 1000 and can be changed at compilation time of the library.

Environment variable RASTER3D_DEFAULT_CACHE_SIZE.

void Rast3d_setCacheSize(int nTiles)

int Rast3d_getCacheSize()

Compression

Toggling compression mode

This value specifies whether compression should be used while writing a new map. It does not have any effect on old maps.

Default RASTER3D_COMPRESSION_DEFAULT. This is set to RASTER3D_COMPRESSION. This default should not be changed.

Environment variables RASTER3D_USE_COMPRESSION and RASTER3D_NO_COMPRESSION.

See functions Rast3d_setCompressionMode() (cf. Section 22.3.2.3 ) and Rast3d_getCompressionMode() (cf. Section 22.3.2.3 ).

Toggling RLE compression

This value specifies whether RLE compression should be used (in addition to precision).

Default RASTER3D_USE_RLE_DEFAULT. This is currently set to RASTER3D_USE_RLE and can be changed at compilation time of the library.

Environment variables RASTER3D_USE_RLE and RASTER3D_NO_RLE.

See functions Rast3d_setCompressionMode() (cf. Section 22.3.2.3) and Rast3d_getCompressionMode() (cf. Section 22.3.2.3).

Setting the precision

This number specifies how many mantissa bits should be used when writing a cell value. The minimum value is 0. The maximum value is 23 or RASTER3D_MAX_PRECISION for type FCELL_TYPE, it is 52 or RASTER3D_MAX_PRECISION for type DCELL_TYPE.

Default RASTER3D_PRECISION_DEFAULT. This is set to RASTER3D_MAX_PRECISION. This default should not be changed.

Environment variables RASTER3D_PRECISION and RASTER3D_MAX_PRECISION.

void Rast3d_setCompressionMode(int doCompress, int doLzw, int doRle, int precision) doCompress should be one of RASTER3D_NO_COMPRESSION and RASTER3D_COMPRESSION, doRle should be either RASTER3D_NO_RLE or RASTER3D_USE_RLE, and precision should be either RASTER3D_MAX_PRECISION or a positive integer.

void Rast3d_getCompressionMode(int *doCompress, int *doLzw, int *doRle, int *precision)

Tiles

Setting the tile dimensions

The dimensions are specified in number of cell.

Defaults RASTER3D_TILE_X_DEFAULT, RASTER3D_TILE_Y_DEFAULT, and RASTER3D_TILE_Z_DEFAULT. These are currently set to 8 and can be changed at compilation time of the library.

Environment variables RASTER3D_TILE_DIMENSION_X, RASTER3D_TILE_DIMENSION_Y, and RASTER3D_TILE_DIMENSION_Z.

void Rast3d_setTileDimension(int tileX, int tileY, int tileZ)

void Rast3d_getTileDimension(int *tileX, int *tileY, int *tileZ)

Setting the tile cell-value type

Specifies which type is used to write cell-values on file. This type can be chosen independently of the type used to store cell-values in memory.

Default RASTER3D_FILE_TYPE_DEFAULT. This is set to DCELL_TYPE. This default should not be changed.

Environment variables RASTER3D_WRITE_FLOAT and RASTER3D_WRITE_DOUBLE.

void Rast3d_setFileType(int type)

int Rast3d_getFileType(int type)

Setting the window

The window is set from a 3d window file.

The default 3d window file is WIND3 located in the current mapset.

Possible choices for 3d window files are name which refers to a window file in the 3d window database located at windows3d of the current mapset; or file names which are identified by a leading "/" or "."; or fully qualified names, i.e. file which refer to window files in the 3d window database of mapset. Note, that names WIND3 and WIND3 do not specify the default window name in the (current) mapset but rather a window file in the window database of the (current) mapset.

Environment variable RASTER3D_DEFAULT_WINDOW3D.

See functions

Rast3d_useWindowParams(),

Rast3d_setWindow(), and

Rast3d_setWindowMap().

Setting the Units

The data untis of a map can be described using a string. The vertical units are defined in gis.h and can be spatial or temporal.

No environment variable.

void Rast3d_set_unit (map, unit) RASTER3d_Map; /* the map