PostGIS

The functions given below are the ones which a user of PostGIS Raster is likely to need and which are currently available in PostGIS Raster. There are other functions which are required support functions to the raster objects which are not of use to a general user.

raster is a new PostGIS type for storing and analyzing raster data.

For loading rasters from raster files please refer to Section 11.1, “Loading and Creating Rasters”

For the examples in this reference we will be using a raster table of dummy rasters - Formed with the following code

CREATE TABLE dummy_rast(rid integer, rast raster);
INSERT INTO dummy_rast(rid, rast)
VALUES (1,
('01' -- little endian (uint8 ndr)
||
'0000' -- version (uint16 0)
||
'0000' -- nBands (uint16 0)
||
'0000000000000040' -- scaleX (float64 2)
||
'0000000000000840' -- scaleY (float64 3)
||
'000000000000E03F' -- ipX (float64 0.5)
||
'000000000000E03F' -- ipY (float64 0.5)
||
'0000000000000000' -- skewX (float64 0)
||
'0000000000000000' -- skewY (float64 0)
||
'00000000' -- SRID (int32 0)
||
'0A00' -- width (uint16 10)
||
'1400' -- height (uint16 20)
)::raster
),
-- Raster: 5 x 5 pixels, 3 bands, PT_8BUI pixel type, NODATA = 0
(2,  ('01000003009A9999999999A93F9A9999999999A9BF000000E02B274A' ||
'41000000007719564100000000000000000000000000000000FFFFFFFF050005000400FDFEFDFEFEFDFEFEFDF9FAFEF' ||
'EFCF9FBFDFEFEFDFCFAFEFEFE04004E627AADD16076B4F9FE6370A9F5FE59637AB0E54F58617087040046566487A1506CA2E3FA5A6CAFFBFE4D566DA4CB3E454C5665')::raster);

12.1. Raster Support Data types

Abstract

This section lists the PostgreSQL data types specifically created to support raster functionality.

geomval[.refpurpose]# — A spatial datatype with two fields - geom (holding a geometry object) and val (holding a double precision pixel value from a raster band).#

addbandarg[.refpurpose]# — A composite type used as input into the ST_AddBand function defining the attributes and initial value of the new band.#

rastbandarg[.refpurpose]# — A composite type for use when needing to express a raster and a band index of that raster.#

raster[.refpurpose]# — raster spatial data type.#

reclassarg[.refpurpose]# — A composite type used as input into the ST_Reclass function defining the behavior of reclassification.#

summarystats[.refpurpose]# — A composite type returned by the ST_SummaryStats and ST_SummaryStatsAgg functions.#

unionarg[.refpurpose]# — A composite type used as input into the ST_Union function defining the bands to be processed and behavior of the UNION operation.#

12.2. Raster Management

AddRasterConstraints[.refpurpose]# — Adds raster constraints to a loaded raster table for a specific column that constrains spatial ref, scaling, blocksize, alignment, bands, band type and a flag to denote if raster column is regularly blocked. The table must be loaded with data for the constraints to be inferred. Returns true if the constraint setting was accomplished and issues a notice otherwise.#

DropRasterConstraints[.refpurpose]# — Drops PostGIS raster constraints that refer to a raster table column. Useful if you need to reload data or update your raster column data.#

AddOverviewConstraints[.refpurpose]# — Tag a raster column as being an overview of another.#

DropOverviewConstraints[.refpurpose]# — Untag a raster column from being an overview of another.#

PostGIS_GDAL_Version[.refpurpose]# — Reports the version of the GDAL library in use by PostGIS.#

PostGIS_Raster_Lib_Build_Date[.refpurpose]# — Reports full raster library build date.#

PostGIS_Raster_Lib_Version[.refpurpose]# — Reports full raster version and build configuration infos.#

ST_GDALDrivers[.refpurpose]# — Returns a list of raster formats supported by PostGIS through GDAL. Only those formats with can_write=True can be used by ST_AsGDALRaster#

ST_Contour[.refpurpose]# — Generates a set of vector contours from the provided raster band, using the GDAL contouring algorithm.#

ST_InterpolateRaster[.refpurpose]# — Interpolates a gridded surface based on an input set of 3-d points, using the X- and Y-values to position the points on the grid and the Z-value of the points as the surface elevation. #

UpdateRasterSRID[.refpurpose]# — Change the SRID of all rasters in the user-specified column and table. #

ST_CreateOverview[.refpurpose]# — Create an reduced resolution version of a given raster coverage. #

12.3. Raster Constructors

ST_AddBand[.refpurpose]# — Returns a raster with the new band(s) of given type added with given initial value in the given index location. If no index is specified, the band is added to the end. #

ST_AsRaster[.refpurpose]# — Converts a PostGIS geometry to a PostGIS raster.#

ST_Band[.refpurpose]# — Returns one or more bands of an existing raster as a new raster. Useful for building new rasters from existing rasters.#

ST_MakeEmptyCoverage[.refpurpose]# — Cover georeferenced area with a grid of empty raster tiles.#

ST_MakeEmptyRaster[.refpurpose]# — Returns an empty raster (having no bands) of given dimensions (width & height), upperleft X and Y, pixel size and rotation (scalex, scaley, skewx & skewy) and reference system (srid). If a raster is passed in, returns a new raster with the same size, alignment and SRID. If srid is left out, the spatial ref is set to unknown (0).#

ST_Tile[.refpurpose]# — Returns a set of rasters resulting from the split of the input raster based upon the desired dimensions of the output rasters.#

ST_Retile[.refpurpose]# — Return a set of configured tiles from an arbitrarily tiled raster coverage. #

ST_FromGDALRaster[.refpurpose]# — Returns a raster from a supported GDAL raster file.#

12.4. Raster Accessors

ST_GeoReference[.refpurpose]# — Returns the georeference meta data in GDAL or ESRI format as commonly seen in a world file. Default is GDAL.#

ST_Height[.refpurpose]# — Returns the height of the raster in pixels.#

ST_IsEmpty[.refpurpose]# — Returns true if the raster is empty (width = 0 and height = 0). Otherwise, returns false.#

ST_MemSize[.refpurpose]# — Returns the amount of space (in bytes) the raster takes.#

ST_MetaData[.refpurpose]# — Returns basic meta data about a raster object such as pixel size, rotation (skew), upper, lower left, etc.#

ST_NumBands[.refpurpose]# — Returns the number of bands in the raster object.#

ST_PixelHeight[.refpurpose]# — Returns the pixel height in geometric units of the spatial reference system.#

ST_PixelWidth[.refpurpose]# — Returns the pixel width in geometric units of the spatial reference system.#

ST_ScaleX[.refpurpose]# — Returns the X component of the pixel width in units of coordinate reference system.#

ST_ScaleY[.refpurpose]# — Returns the Y component of the pixel height in units of coordinate reference system.#

ST_RasterToWorldCoord[.refpurpose]# — Returns the raster’s upper left corner as geometric X and Y (longitude and latitude) given a column and row. Column and row starts at 1.#

ST_RasterToWorldCoordX[.refpurpose]# — Returns the geometric X coordinate upper left of a raster, column and row. Numbering of columns and rows starts at 1.#

ST_RasterToWorldCoordY[.refpurpose]# — Returns the geometric Y coordinate upper left corner of a raster, column and row. Numbering of columns and rows starts at 1. #

ST_Rotation[.refpurpose]# — Returns the rotation of the raster in radian.#

ST_SkewX[.refpurpose]# — Returns the georeference X skew (or rotation parameter).#

ST_SkewY[.refpurpose]# — Returns the georeference Y skew (or rotation parameter).#

ST_SRID[.refpurpose]# — Returns the spatial reference identifier of the raster as defined in spatial_ref_sys table.#

ST_Summary[.refpurpose]# — Returns a text summary of the contents of the raster.#

ST_UpperLeftX[.refpurpose]# — Returns the upper left X coordinate of raster in projected spatial ref.#

ST_UpperLeftY[.refpurpose]# — Returns the upper left Y coordinate of raster in projected spatial ref.#

ST_Width[.refpurpose]# — Returns the width of the raster in pixels.#

ST_WorldToRasterCoord[.refpurpose]# — Returns the upper left corner as column and row given geometric X and Y (longitude and latitude) or a point geometry expressed in the spatial reference coordinate system of the raster.#

ST_WorldToRasterCoordX[.refpurpose]# — Returns the column in the raster of the point geometry (pt) or a X and Y world coordinate (xw, yw) represented in world spatial reference system of raster.#

ST_WorldToRasterCoordY[.refpurpose]# — Returns the row in the raster of the point geometry (pt) or a X and Y world coordinate (xw, yw) represented in world spatial reference system of raster.#

12.5. Raster Band Accessors

ST_BandMetaData[.refpurpose]# — Returns basic meta data for a specific raster band. band num 1 is assumed if none-specified.#

ST_BandNoDataValue[.refpurpose]# — Returns the value in a given band that represents no data. If no band num 1 is assumed.#

ST_BandIsNoData[.refpurpose]# — Returns true if the band is filled with only nodata values.#

ST_BandPath[.refpurpose]# — Returns system file path to a band stored in file system. If no bandnum specified, 1 is assumed.#

ST_BandFileSize[.refpurpose]# — Returns the file size of a band stored in file system. If no bandnum specified, 1 is assumed.#

ST_BandFileTimestamp[.refpurpose]# — Returns the file timestamp of a band stored in file system. If no bandnum specified, 1 is assumed.#

ST_BandPixelType[.refpurpose]# — Returns the type of pixel for given band. If no bandnum specified, 1 is assumed.#

ST_MinPossibleValue[.refpurpose]# — Returns the minimum value this pixeltype can store.#

ST_HasNoBand[.refpurpose]# — Returns true if there is no band with given band number. If no band number is specified, then band number 1 is assumed.#

12.6. Raster Pixel Accessors and Setters

ST_PixelAsPolygon[.refpurpose]# — Returns the polygon geometry that bounds the pixel for a particular row and column.#

ST_PixelAsPolygons[.refpurpose]# — Returns the polygon geometry that bounds every pixel of a raster band along with the value, the X and the Y raster coordinates of each pixel.#

ST_PixelAsPoint[.refpurpose]# — Returns a point geometry of the pixel’s upper-left corner. #

ST_PixelAsPoints[.refpurpose]# — Returns a point geometry for each pixel of a raster band along with the value, the X and the Y raster coordinates of each pixel. The coordinates of the point geometry are of the pixel’s upper-left corner. #

ST_PixelAsCentroid[.refpurpose]# — Returns the centroid (point geometry) of the area represented by a pixel. #

ST_PixelAsCentroids[.refpurpose]# — Returns the centroid (point geometry) for each pixel of a raster band along with the value, the X and the Y raster coordinates of each pixel. The point geometry is the centroid of the area represented by a pixel. #

ST_Value[.refpurpose]# — Returns the value of a given band in a given columnx, rowy pixel or at a particular geometric point. Band numbers start at 1 and assumed to be 1 if not specified. If exclude_nodata_value is set to false, then all pixels include nodata pixels are considered to intersect and return value. If exclude_nodata_value is not passed in then reads it from metadata of raster.#

ST_NearestValue[.refpurpose]# — Returns the nearest non-NODATA value of a given band’s pixel specified by a columnx and rowy or a geometric point expressed in the same spatial reference coordinate system as the raster. #

ST_SetZ[.refpurpose]# — Returns a geometry with the same X/Y coordinates as the input geometry, and values from the raster copied into the Z dimension using the requested resample algorithm.#

ST_SetM[.refpurpose]# — Returns a geometry with the same X/Y coordinates as the input geometry, and values from the raster copied into the Z dimension using the requested resample algorithm.#

ST_Neighborhood[.refpurpose]# — Returns a 2-D double precision array of the non-NODATA values around a given band’s pixel specified by either a columnX and rowY or a geometric point expressed in the same spatial reference coordinate system as the raster. #

ST_SetValue[.refpurpose]# — Returns modified raster resulting from setting the value of a given band in a given columnx, rowy pixel or the pixels that intersect a particular geometry. Band numbers start at 1 and assumed to be 1 if not specified.#

ST_SetValues[.refpurpose]# — Returns modified raster resulting from setting the values of a given band.#

ST_DumpValues[.refpurpose]# — Get the values of the specified band as a 2-dimension array. #

ST_PixelOfValue[.refpurpose]# — Get the columnx, rowy coordinates of the pixel whose value equals the search value. #

12.7. Raster Editors

ST_SetGeoReference[.refpurpose]# — Set Georeference 6 georeference parameters in a single call. Numbers should be separated by white space. Accepts inputs in GDAL or ESRI format. Default is GDAL.#

ST_SetRotation[.refpurpose]# — Set the rotation of the raster in radian.#

ST_SetScale[.refpurpose]# — Sets the X and Y size of pixels in units of coordinate reference system. Number units/pixel width/height.#

ST_SetSkew[.refpurpose]# — Sets the georeference X and Y skew (or rotation parameter). If only one is passed in, sets X and Y to the same value.#

ST_SetSRID[.refpurpose]# — Sets the SRID of a raster to a particular integer srid defined in the spatial_ref_sys table.#

ST_SetUpperLeft[.refpurpose]# — Sets the value of the upper left corner of the pixel of the raster to projected X and Y coordinates.#

ST_Resample[.refpurpose]# — Resample a raster using a specified resampling algorithm, new dimensions, an arbitrary grid corner and a set of raster georeferencing attributes defined or borrowed from another raster. #

ST_Rescale[.refpurpose]# — Resample a raster by adjusting only its scale (or pixel size). New pixel values are computed using the NearestNeighbor (english or american spelling), Bilinear, Cubic, CubicSpline or Lanczos resampling algorithm. Default is NearestNeighbor. #

ST_Reskew[.refpurpose]# — Resample a raster by adjusting only its skew (or rotation parameters). New pixel values are computed using the NearestNeighbor (english or american spelling), Bilinear, Cubic, CubicSpline or Lanczos resampling algorithm. Default is NearestNeighbor. #

ST_SnapToGrid[.refpurpose]# — Resample a raster by snapping it to a grid. New pixel values are computed using the NearestNeighbor (english or american spelling), Bilinear, Cubic, CubicSpline or Lanczos resampling algorithm. Default is NearestNeighbor. #

ST_Resize[.refpurpose]# — Resize a raster to a new width/height#

ST_Transform[.refpurpose]# — Reprojects a raster in a known spatial reference system to another known spatial reference system using specified resampling algorithm. Options are NearestNeighbor, Bilinear, Cubic, CubicSpline, Lanczos defaulting to NearestNeighbor.#

12.8. Raster Band Editors

ST_SetBandNoDataValue[.refpurpose]# — Sets the value for the given band that represents no data. Band 1 is assumed if no band is specified. To mark a band as having no nodata value, set the nodata value = NULL.#

ST_SetBandIsNoData[.refpurpose]# — Sets the isnodata flag of the band to TRUE.#

ST_SetBandPath[.refpurpose]# — Update the external path and band number of an out-db band#

ST_SetBandIndex[.refpurpose]# — Update the external band number of an out-db band#

12.9. Raster Band Statistics and Analytics

ST_Count[.refpurpose]# — Returns the number of pixels in a given band of a raster or raster coverage. If no band is specified defaults to band 1. If exclude_nodata_value is set to true, will only count pixels that are not equal to the nodata value.#

ST_CountAgg[.refpurpose]# — Aggregate. Returns the number of pixels in a given band of a set of rasters. If no band is specified defaults to band 1. If exclude_nodata_value is set to true, will only count pixels that are not equal to the NODATA value. #

ST_Histogram[.refpurpose]# — Returns a set of record summarizing a raster or raster coverage data distribution separate bin ranges. Number of bins are autocomputed if not specified.#

ST_Quantile[.refpurpose]# — Compute quantiles for a raster or raster table coverage in the context of the sample or population. Thus, a value could be examined to be at the raster’s 25%, 50%, 75% percentile.#

ST_SummaryStats[.refpurpose]# — Returns summarystats consisting of count, sum, mean, stddev, min, max for a given raster band of a raster or raster coverage. Band 1 is assumed is no band is specified.#

ST_SummaryStatsAgg[.refpurpose]# — Aggregate. Returns summarystats consisting of count, sum, mean, stddev, min, max for a given raster band of a set of raster. Band 1 is assumed is no band is specified.#

ST_ValueCount[.refpurpose]# — Returns a set of records containing a pixel band value and count of the number of pixels in a given band of a raster (or a raster coverage) that have a given set of values. If no band is specified defaults to band 1. By default nodata value pixels are not counted. and all other values in the pixel are output and pixel band values are rounded to the nearest integer.#

12.10. Raster Inputs

ST_RastFromWKB[.refpurpose]# — Return a raster value from a Well-Known Binary (WKB) raster.#

ST_RastFromHexWKB[.refpurpose]# — Return a raster value from a Hex representation of Well-Known Binary (WKB) raster.#

12.11. Raster Outputs

ST_AsBinary/ST_AsWKB[.refpurpose]# — Return the Well-Known Binary (WKB) representation of the raster.#

ST_AsHexWKB[.refpurpose]# — Return the Well-Known Binary (WKB) in Hex representation of the raster.#

ST_AsGDALRaster[.refpurpose]# — Return the raster tile in the designated GDAL Raster format. Raster formats are one of those supported by your compiled library. Use ST_GDALDrivers() to get a list of formats supported by your library.#

ST_AsJPEG[.refpurpose]# — Return the raster tile selected bands as a single Joint Photographic Exports Group (JPEG) image (byte array). If no band is specified and 1 or more than 3 bands, then only the first band is used. If only 3 bands then all 3 bands are used and mapped to RGB.#

ST_AsPNG[.refpurpose]# — Return the raster tile selected bands as a single portable network graphics (PNG) image (byte array). If 1, 3, or 4 bands in raster and no bands are specified, then all bands are used. If more 2 or more than 4 bands and no bands specified, then only band 1 is used. Bands are mapped to RGB or RGBA space.#

ST_AsTIFF[.refpurpose]# — Return the raster selected bands as a single TIFF image (byte array). If no band is specified or any of specified bands does not exist in the raster, then will try to use all bands.#

12.12. Raster Processing: Map Algebra

ST_Clip[.refpurpose]# — Returns the raster clipped by the input geometry. If band number not is specified, all bands are processed. If crop is not specified or TRUE, the output raster is cropped.#

ST_ColorMap[.refpurpose]# — Creates a new raster of up to four 8BUI bands (grayscale, RGB, RGBA) from the source raster and a specified band. Band 1 is assumed if not specified.#

ST_Grayscale[.refpurpose]# — Creates a new one-8BUI band raster from the source raster and specified bands representing Red, Green and Blue#

ST_Intersection[.refpurpose]# — Returns a raster or a set of geometry-pixelvalue pairs representing the shared portion of two rasters or the geometrical intersection of a vectorization of the raster and a geometry.#

ST_MapAlgebra (callback function version)[.refpurpose]# — Callback function version - Returns a one-band raster given one or more input rasters, band indexes and one user-specified callback function. #

ST_MapAlgebra (expression version)[.refpurpose]# — Expression version - Returns a one-band raster given one or two input rasters, band indexes and one or more user-specified SQL expressions. #

ST_MapAlgebraExpr[.refpurpose]# — 1 raster band version: Creates a new one band raster formed by applying a valid PostgreSQL algebraic operation on the input raster band and of pixeltype provided. Band 1 is assumed if no band is specified.#

ST_MapAlgebraExpr[.refpurpose]# — 2 raster band version: Creates a new one band raster formed by applying a valid PostgreSQL algebraic operation on the two input raster bands and of pixeltype provided. band 1 of each raster is assumed if no band numbers are specified. The resulting raster will be aligned (scale, skew and pixel corners) on the grid defined by the first raster and have its extent defined by the "extenttype" parameter. Values for "extenttype" can be: INTERSECTION, UNION, FIRST, SECOND. #

ST_MapAlgebraFct[.refpurpose]# — 1 band version - Creates a new one band raster formed by applying a valid PostgreSQL function on the input raster band and of pixeltype prodived. Band 1 is assumed if no band is specified.#

ST_MapAlgebraFct[.refpurpose]# — 2 band version - Creates a new one band raster formed by applying a valid PostgreSQL function on the 2 input raster bands and of pixeltype prodived. Band 1 is assumed if no band is specified. Extent type defaults to INTERSECTION if not specified.#

ST_MapAlgebraFctNgb[.refpurpose]# — 1-band version: Map Algebra Nearest Neighbor using user-defined PostgreSQL function. Return a raster which values are the result of a PLPGSQL user function involving a neighborhood of values from the input raster band.#

ST_Reclass[.refpurpose]# — Creates a new raster composed of band types reclassified from original. The nband is the band to be changed. If nband is not specified assumed to be 1. All other bands are returned unchanged. Use case: convert a 16BUI band to a 8BUI and so forth for simpler rendering as viewable formats.#

ST_Union[.refpurpose]# — Returns the union of a set of raster tiles into a single raster composed of 1 or more bands.#

12.13. Built-in Map Algebra Callback Functions

ST_Distinct4ma[.refpurpose]# — Raster processing function that calculates the number of unique pixel values in a neighborhood.#

ST_InvDistWeight4ma[.refpurpose]# — Raster processing function that interpolates a pixel’s value from the pixel’s neighborhood.#

ST_Max4ma[.refpurpose]# — Raster processing function that calculates the maximum pixel value in a neighborhood.#

ST_Mean4ma[.refpurpose]# — Raster processing function that calculates the mean pixel value in a neighborhood.#

ST_Min4ma[.refpurpose]# — Raster processing function that calculates the minimum pixel value in a neighborhood. #

ST_MinDist4ma[.refpurpose]# — Raster processing function that returns the minimum distance (in number of pixels) between the pixel of interest and a neighboring pixel with value.#

ST_Range4ma[.refpurpose]# — Raster processing function that calculates the range of pixel values in a neighborhood.#

ST_StdDev4ma[.refpurpose]# — Raster processing function that calculates the standard deviation of pixel values in a neighborhood.#

ST_Sum4ma[.refpurpose]# — Raster processing function that calculates the sum of all pixel values in a neighborhood.#

12.14. Raster Processing: DEM (Elevation)

ST_Aspect[.refpurpose]# — Returns the aspect (in degrees by default) of an elevation raster band. Useful for analyzing terrain.#

ST_HillShade[.refpurpose]# — Returns the hypothetical illumination of an elevation raster band using provided azimuth, altitude, brightness and scale inputs.#

ST_Roughness[.refpurpose]# — Returns a raster with the calculated "roughness" of a DEM.#

ST_Slope[.refpurpose]# — Returns the slope (in degrees by default) of an elevation raster band. Useful for analyzing terrain.#

ST_TPI[.refpurpose]# — Returns a raster with the calculated Topographic Position Index.#

ST_TRI[.refpurpose]# — Returns a raster with the calculated Terrain Ruggedness Index.#

12.15. Raster Processing: Raster to Geometry

Box3D[.refpurpose]# — Returns the box 3d representation of the enclosing box of the raster.#

ST_ConvexHull[.refpurpose]# — Return the convex hull geometry of the raster including pixel values equal to BandNoDataValue. For regular shaped and non-skewed rasters, this gives the same result as ST_Envelope so only useful for irregularly shaped or skewed rasters.#

ST_DumpAsPolygons[.refpurpose]# — Returns a set of geomval (geom,val) rows, from a given raster band. If no band number is specified, band num defaults to 1.#

ST_Envelope[.refpurpose]# — Returns the polygon representation of the extent of the raster.#

ST_MinConvexHull[.refpurpose]# — Return the convex hull geometry of the raster excluding NODATA pixels. #

ST_Polygon[.refpurpose]# — Returns a multipolygon geometry formed by the union of pixels that have a pixel value that is not no data value. If no band number is specified, band num defaults to 1.#

12.16. Raster Operators

&&[.refpurpose]# — Returns TRUE if A’s bounding box intersects B’s bounding box.#

&<[.refpurpose]# — Returns TRUE if A’s bounding box is to the left of B’s.#

&>[.refpurpose]# — Returns TRUE if A’s bounding box is to the right of B’s.#

=[.refpurpose]# — Returns TRUE if A’s bounding box is the same as B’s. Uses double precision bounding box.#

@[.refpurpose]# — Returns TRUE if A’s bounding box is contained by B’s. Uses double precision bounding box.#

~=[.refpurpose]# — Returns TRUE if A’s bounding box is the same as B’s.#

~[.refpurpose]# — Returns TRUE if A’s bounding box is contains B’s. Uses double precision bounding box.#

12.17. Raster and Raster Band Spatial Relationships

ST_Contains[.refpurpose]# — Return true if no points of raster rastB lie in the exterior of raster rastA and at least one point of the interior of rastB lies in the interior of rastA. #

ST_ContainsProperly[.refpurpose]# — Return true if rastB intersects the interior of rastA but not the boundary or exterior of rastA. #

ST_Covers[.refpurpose]# — Return true if no points of raster rastB lie outside raster rastA. #

ST_CoveredBy[.refpurpose]# — Return true if no points of raster rastA lie outside raster rastB. #

ST_Disjoint[.refpurpose]# — Return true if raster rastA does not spatially intersect rastB. #

ST_Intersects[.refpurpose]# — Return true if raster rastA spatially intersects raster rastB.#

ST_Overlaps[.refpurpose]# — Return true if raster rastA and rastB intersect but one does not completely contain the other. #

ST_Touches[.refpurpose]# — Return true if raster rastA and rastB have at least one point in common but their interiors do not intersect. #

ST_SameAlignment[.refpurpose]# — Returns true if rasters have same skew, scale, spatial ref, and offset (pixels can be put on same grid without cutting into pixels) and false if they don’t with notice detailing issue.#

ST_NotSameAlignmentReason[.refpurpose]# — Returns text stating if rasters are aligned and if not aligned, a reason why.#

ST_Within[.refpurpose]# — Return true if no points of raster rastA lie in the exterior of raster rastB and at least one point of the interior of rastA lies in the interior of rastB. #

ST_DWithin[.refpurpose]# — Return true if rasters rastA and rastB are within the specified distance of each other. #

ST_DFullyWithin[.refpurpose]# — Return true if rasters rastA and rastB are fully within the specified distance of each other. #

12.18. Raster Tips

Abstract

This section documents various gotchas and tips related to PostGIS Raster.

12.18.1. Out-DB Rasters

12.18.1.1. Directory containing many files

12.18.1.2. Maximum Number of Open Files

12.18.1.1. Directory containing many files

When GDAL opens a file, GDAL eagerly scans the directory of that file to build a catalog of other files. If this directory contains many files (e.g. thousands, millions), opening that file becomes extremely slow (especially if that file happens to be on a network drive such as NFS).

To control this behavior, GDAL provides the following environment variable: GDAL_DISABLE_READDIR_ON_OPEN. Set GDAL_DISABLE_READDIR_ON_OPEN to TRUE to disable directory scanning.

In Ubuntu (and assuming you are using PostgreSQL’s packages for Ubuntu), GDAL_DISABLE_READDIR_ON_OPEN can be set in /etc/postgresql/POSTGRESQL_VERSION/CLUSTER_NAME/environment (where POSTGRESQL_VERSION is the version of PostgreSQL, e.g. 9.6 and CLUSTER_NAME is the name of the cluster, e.g. maindb). You can also set PostGIS environment variables here as well.

# environment variables for postmaster process
# This file has the same syntax as postgresql.conf:
#  VARIABLE = simple_value
#  VARIABLE2 = 'any value!'
# I. e. you need to enclose any value which does not only consist of letters,
# numbers, and '-', '_', '.' in single quotes. Shell commands are not
# evaluated.
POSTGIS_GDAL_ENABLED_DRIVERS = 'ENABLE_ALL'

POSTGIS_ENABLE_OUTDB_RASTERS = 1

GDAL_DISABLE_READDIR_ON_OPEN = 'TRUE'

12.18.1.2. Maximum Number of Open Files

The maximum number of open files permitted by Linux and PostgreSQL are typically conservative (typically 1024 open files per process) given the assumption that the system is consumed by human users. For Out-DB Rasters, a single valid query can easily exceed this limit (e.g. a dataset of 10 year’s worth of rasters with one raster for each day containing minimum and maximum temperatures and we want to know the absolute min and max value for a pixel in that dataset).

The easiest change to make is the following PostgreSQL setting: max_files_per_process. The default is set to 1000, which is far too low for Out-DB Rasters. A safe starting value could be 65536 but this really depends on your datasets and the queries run against those datasets. This setting can only be made on server start and probably only in the PostgreSQL configuration file (e.g. /etc/postgresql/POSTGRESQL_VERSION/CLUSTER_NAME/postgresql.conf in Ubuntu environments).

...
# - Kernel Resource Usage -

max_files_per_process = 65536           # min 25
                                        # (change requires restart)
...

The major change to make is the Linux kernel’s open files limits. There are two parts to this:

Maximum number of open files for the entire system
Maximum number of open files per process

12.18.1.2.1. Maximum number of open files for the entire system

You can inspect the current maximum number of open files for the entire system with the following example:

$ sysctl -a | grep fs.file-max
fs.file-max = 131072

If the value returned is not large enough, add a file to /etc/sysctl.d/ as per the following example:

$ echo "fs.file-max = 6145324" >> /etc/sysctl.d/fs.conf

$ cat /etc/sysctl.d/fs.conf
fs.file-max = 6145324

$ sysctl -p --system
* Applying /etc/sysctl.d/fs.conf ...
fs.file-max = 2097152
* Applying /etc/sysctl.conf ...

$ sysctl -a | grep fs.file-max
fs.file-max = 6145324

12.18.1.2.2. Maximum number of open files per process

We need to increase the maximum number of open files per process for the PostgreSQL server processes.

To see what the current PostgreSQL service processes are using for maximum number of open files, do as per the following example (make sure to have PostgreSQL running):

$ ps aux | grep postgres
postgres 31713  0.0  0.4 179012 17564 pts/0    S    Dec26   0:03 /home/dustymugs/devel/postgresql/sandbox/10/usr/local/bin/postgres -D /home/dustymugs/devel/postgresql/sandbox/10/pgdata
postgres 31716  0.0  0.8 179776 33632 ?        Ss   Dec26   0:01 postgres: checkpointer process
postgres 31717  0.0  0.2 179144  9416 ?        Ss   Dec26   0:05 postgres: writer process
postgres 31718  0.0  0.2 179012  8708 ?        Ss   Dec26   0:06 postgres: wal writer process
postgres 31719  0.0  0.1 179568  7252 ?        Ss   Dec26   0:03 postgres: autovacuum launcher process
postgres 31720  0.0  0.1  34228  4124 ?        Ss   Dec26   0:09 postgres: stats collector process
postgres 31721  0.0  0.1 179308  6052 ?        Ss   Dec26   0:00 postgres: bgworker: logical replication launcher

$ cat /proc/31718/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             15738                15738                processes
Max open files            1024                 4096                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       15738                15738                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

In the example above, we inspected the open files limit for Process 31718. It doesn’t matter which PostgreSQL process, any of them will do. The response we are interested in is Max open files.

We want to increase Soft Limit and Hard Limit of Max open files to be greater than the value we specified for the PostgreSQL setting max_files_per_process. In our example, we set max_files_per_process to 65536.

In Ubuntu (and assuming you are using PostgreSQL’s packages for Ubuntu), the easiest way to change the Soft Limit and Hard Limit is to edit /etc/init.d/postgresql (SysV) or /lib/systemd/system/postgresql.service* (systemd).

Let’s first address the SysV Ubuntu case where we add ulimit -H -n 262144 and ulimit -n 131072 to /etc/init.d/postgresql.

...
case "$1" in
    start|stop|restart|reload)
        if [ "$1" = "start" ]; then
            create_socket_directory
        fi
    if [ -z "`pg_lsclusters -h`" ]; then
        log_warning_msg 'No PostgreSQL clusters exist; see "man pg_createcluster"'
        exit 0
    fi

    ulimit -H -n 262144
    ulimit -n 131072

    for v in $versions; do
        $1 $v || EXIT=$?
    done
    exit ${EXIT:-0}
        ;;
    status)
...

Now to address the systemd Ubuntu case. We will add LimitNOFILE=131072 to every /lib/systemd/system/postgresql.service* file in the [Service] section.

...
[Service]

LimitNOFILE=131072

...

[Install]
WantedBy=multi-user.target
...

After making the necessary systemd changes, make sure to reload the daemon

systemctl daemon-reload

Prev		Next
Chapter 11. Raster Data Management, Queries, and Applications	Home	geomval

Chapter 11. Raster Data Management, Queries, and Applications

Home

geomval