Update of "PROJ.6"
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview

Artifact ID: ce4e33e400b1782aa87042cb94c827a85ec73646
Page Name:PROJ.6
Date: 2019-04-24 18:00:48
Original User: sandro
Parent: 29f52ff47a527b1137ac252c9437ad5c0f878e2a (diff)
Next bfd67870269a2ce3f09374a52a45819cf1fa22db
Content

back



Introduction

PROJ is a well-known library for performing conversions between cartographic projections.
It's universally supported by almost all open source GIS-oriented applications and packages, so there is no need to waste time in futher presentations.
We just need a bit of history to fully understand the current state-of-the-art:

Timeline:
Short conclusion: the fourth version of PROJ (aka PROJ.4) lasted for about two decades, a very uncommon situation.
And a full generation of developers and users become sincerly convinced that PROJ.4 was the real name of the library.

The revolution comes

Timeline:
Note: the whole transition implies many relevant changes, so that a deeply revised API will be required. In other words, the old PROJ.4 and the new PROJ.7 will support two different APIs, thus abruptly breaking cross-version compatibility.
This is an umpleasant new, because it practically means that all software modules depending on PROJ (this including SpatiaLite) will require a not at all trivial rewrite in order to fulfill the new API requirements.
But when you consider that's the first time in its very long life that PROJ requires an extra effrot in order to introduce so many useful innovations, this unexpected API breakage looks fully justified and absolutely reasonable.

More details about the API breakage:

What's new in PROJ.6

Fully supporting ISO-19111:2019 WKT

Old versions of PROJ (including PROJ.4) required to define each CRS (Coordinate Reference System) by a corresponding proj-string. The following table exemplifies the case of few CRSes:

SRIDCRS Nameproj-string
3003Monte Mario / Italy zone 1 +proj=tmerc +lat_0=0 +lon_0=9 +k=0.9996 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs
4326WGS 84 +proj=longlat +datum=WGS84 +no_defs
32632WGS 84 / UTM zone 32N +proj=utm +zone=32 +datum=WGS84 +units=m +no_defs

New versions of PROJ (starting since PROJ.6) still continue to support the old proj-strings, but the preferred notation for defining any CRS is now conformant to the ISO-19111:2019 international standard
(OGC Abstract Specification Topic 2: “Referencing By Coordinates”).
The following table exemplifies the same CRSes as above in the ISO WKT notation:

SRID3003432632632
CRS NameMonte Mario / Italy zone 1WGS 84WGS 84 / UTM zone 32N
ISO-2018 WKT
PROJCRS["Monte Mario / Italy zone 1",
    BASEGEODCRS["Monte Mario",
        DATUM["Monte Mario",<br>
            ELLIPSOID["International 1924",6378388,297,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]]],
    CONVERSION["Italy zone 1",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",9,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",1500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["easting (X)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["northing (Y)",north,
            ORDER[2],<br>
            LENGTHUNIT["metre",1]],
    AREA["Italy - west of 12°E"],
    BBOX[36.53,5.94,47.04,12],
    ID["EPSG",3003]]
GEODCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    AREA["World"],
    BBOX[-90,-180,90,180],
    ID["EPSG",4326]]
PROJCRS["WGS 84 / UTM zone 32N",
    BASEGEODCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]]],
    CONVERSION["UTM zone 32N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",9,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    AREA["World - N hemisphere - 6°E to 12°E - by country"],
    BBOX[0,6,84,12],
    ID["EPSG",32632]]

As you can easily notice, the two notations are profoundly different. The old proj-string notation is extremely concise and rough, whilst the new ISO-WKT is verbose but exhaustive, detailed and precise.
There is no possible match: ISO-KWT is clearly superior and more sophisticated under any possible aspect.
Not at all surprisingly, coordinate transformations based on ISO-WKT definitions (instead of proj-strings) are usually expected to be more accurate.

And that's not all; the old proj-strings weren't formally defined by any standard, and only PROJ, GDAL and few other FLOSS / GFOSS implementations can understand them.
At the opposite, the new ISO-WKT is formally defined by an international standard, and many free and proprietary implementations can understand it.

Important notice: there are several dialects in the WKT notation describing CRSes.
All them share the same core and mainly differ in few minor details.
PROJ.6 can understand the following dialects:
  1. ESRI WKT: a dialect widely adopted by ESRI proprietary software.
  2. GDAL WKT1: a dialect historically supported by the open source GDAL library.
  3. WKT2:2015: as defined by ISO 19162:2015 standard specifications.
  4. WKT2:2018: as defined by ISO 19162:2018 standard specifications.

Transformation pipelines

There is a second astonishing improvement supported by PROJ.6, that are transformation pipelines.
When using a pipeline you can freely define any complex geodetic transformation by chaining together many elementary steps such as conversion, transformation, projection, axis swap and so on.
A pipeline is conceptually similar to a UNIX shell script, with a dataflow regularly proceding forward from step to step until producing the final result.
The following is a practial example of a pipeline corresponding to a transformation from NAD27 to NAD83,
+proj=pipeline 
+step +proj=axisswap +order=2,1 
+step +proj=unitconvert +xy_in=deg +xy_out=rad 
+step +proj=hgridshift +grids=conus 
+step +proj=unitconvert +xy_in=rad +xy_out=deg 
+step +proj=axisswap +order=2,1

The private SQLite database supporting PROJ.6

Starting since PROJ.6 the PROJ library requires an external SQLite database containing all definitions about supported CRSes, Ellipsoids, Prime Meridians, Unit of Measure, Transformations and so on.
Many of the new advanced PROJ's API will fail if such database is not available.

In order to learn more about the many interesting cool new features supported by PROJ.6 please consult the original documentation


SpatiaLite-5.0.0 supporting PROJ.6

Existing SQL functions affected by PROJ.6

SQL FunctionExtended signatureBehavior under PROJ.6
ST_Transform() ST_Transform ( geom Geometry , newSrid Integer , area Geometry ) : Geometry
ST_Transform ( geom Geometry , newSrid Integer , area Geometry , method Text ) : Geometry
ST_Transform ( geom Geometry , newSrid Integer , area Geometry , method Text , proj_string Text ) : Geometry
Note: all these extended signatures are supported only when libspatialite-5.0 has been built on the top of PROJ.6
If the library has been built instead on earlier versions of PROJ any attempt to call these extended signatures will just return a wrong number of arguments to function ST_Transform() error.

Any invalid argument passed to ST_Transform() (extended signatures) will raise an SQL Exception.

  • the optional argument area may be NULL (default setting).
    Otherwise it's expected to contain a valid Geometry in SRID=4326 (long/lat). When an area of use is explicitly supplied, then PROJ.6 may eventually use the corresponding BBOX for finely tuning the more accurate transformation.
  • the optional argument method is expected to contain one the following values (proj4text will be always assumed as the default setting):
    • proj4text: both CRSes (origin and destination) will be defined by passing to PROJ.6 the traditionals proj-string definitions for the corresponding SRIDs retrieved from column proj4text in Table spatial_ref_sys
      This exactly corresponds to the well extablished traditional behavior of all previous versions of libspatialite and still continues to be supported mainly for historical compatibility, but isn't any longer the best way for taking full profit of the more advanced capabilities of PROJ.6
    • srtext: both CRSes (origin and destination) will be defined by passing to PROJ.6 the WKT definitions for the corresponding SRIDs retrieved from column srtext in Table spatial_ref_sys
      Note: this method could effectively take full profit of the more advanced capabilities of PROJ.6, but only if the underlaying spatialite database has been created with PROJ.6 support.
      But it could easily be inadequate when the underlaying spatialite database was created by some earlier version of spatialite lacking PROJ.6 support.
    • auth_name:auth_srid: both CRSes (origin and destination) will be defined by ignoring the definitions stored in spatial_ref_sys and leaving PROJ.6 free to retrieve their corresponding definitions from its own private SQLite database.
      Note: this method will always take full profit of the more advanced capabilities of PROJ.6
    • proj_string: an user-defined proj-string (may be one definining a transformation pipeline) will be passed to PROJ.6; this is an advanced option reserved to skilled power users.
  • the optional argument proj_string may be NULL (default setting).
    Otherwise it's expected to contain a valid PROJ.6 transformation string, may well be defining an user-defined transformation pipeline.
    Meaningful only in the case of method=proj_string
InitSpatialMetaData()
InitSpatialMetaDataFull()
InsertEpsgSRID()
unchanged All these SQL Functions doesn't change their signatures, but behave in a significantly different way when PROJ.6 support is available.
  • traditional behavior on old PROJ.4: the spatial_ref_sys metadata Table will be populated by using the CRSes definitions internally inlined within the code of libspatialite.
  • new behavior on PROJ.6: the spatial_ref_sys metadata Table will be populated by inserting the CRSes definitions extracted from the SQLite private database currently connected to PROJ.6; more precisely, column proj4text will be populated by inserting the new proj-strings as supported by PROJ.6, whilst column srtext will be populated by inserting the ISO-2018 WKT definitions as supported by PROJ.6
    Important consequence: if there is no database currently connected to PROJ.6 the initialization of spatial_ref_sys will fail.

Few practical SQL examples:
SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 3003)); -- SRID=3003 - default settings; using proj-strings from spatial_ref_sys.proj4text
-------------------------------------
POINT(1732832.444892 4816350.146246)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 3003, NULL, 'srtext')); -- SRID=3003 - NULL area of use, using ISO-2018 WKT definitions from spatial_ref_sys.srtext
-------------------------------------
POINT(1732852.942717 4816277.617691)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 3003, NULL, 'auth_name:auth_srid')); -- SRID=3003 - NULL area of use, ignoring spatialite_ref_sys and using the PROJ.6 own private database
-------------------------------------
POINT(1732852.942717 4816277.617691)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 32632)); -- SRID=32632 - default settings; using proj-strings from spatial_ref_sys.proj4text
-------------------------------------
POINT(732821.718475 4816262.631245)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 32632, NULL, 'srtext')); -- SRID=32632 - NULL area of use, using ISO-2018 WKT definitions from spatial_ref_sys.srtext
-------------------------------------
POINT(732821.718475 4816262.631245)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 32632, NULL, 'auth_name:auth_srid')); -- SRID=32632 - NULL area of use, ignoring spatialite_ref_sys and using the PROJ.6 own private database
-------------------------------------
POINT(732821.718475 4816262.631245)
This first test is based on a new SpatiaLite dabase created with full PROJ.6 support. As you can easily notice:
SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 3003)); -- SRID=3003 - default settings; using proj-strings from spatial_ref_sys.proj4text
-------------------------------------
POINT(1732852.942717 4816277.617691)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 3003, NULL, 'srtext')); -- SRID=3003 - NULL area of use, using ISO-2018 WKT definitions from spatial_ref_sys.srtext
-------------------------------------
POINT(1732852.942717 4816277.617691)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 3003, NULL, 'auth_name:auth_srid')); -- SRID=3003 - NULL area of use, ignoring spatialite_ref_sys and using the PROJ.6 own private database
-------------------------------------
POINT(1732852.942717 4816277.617691)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 32632)); -- SRID=32632 - default settings; using proj-strings from spatial_ref_sys.proj4text
-------------------------------------
POINT(732821.718475 4816262.631245)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 32632, NULL, 'srtext')); -- SRID=32632 - NULL area of use, using ISO-2018 WKT definitions from spatial_ref_sys.srtext
-------------------------------------
POINT(732821.718475 4816262.631245)

SELECT ST_AsText ( ST_Transform( MakePoint( 11.878056 , 43.463056, 4326 ), 32632, NULL, 'auth_name:auth_srid')); -- SRID=32632 - NULL area of use, ignoring spatialite_ref_sys and using the PROJ.6 own private database
-------------------------------------
POINT(732821.718475 4816262.631245)
This second test is based instead on a legacy SpatiaLite database created by some earlier version lacking PROJ.6 support.
In this specific case there is no difference in the results returned by different methods.

Mistery unveiled: just a quick glance to the old and new proj-strings:

New auxiliary SQL functions specifically supporting PROJ.6

SQL FunctionSupported argumentsDescription
HasProj6( void ) : Boolean None Will return 1 (TRUE) if the library has been built on PROJ.6 (or any later version), otherwise 0 (FALSE).
PROJ_GetLastErrorMsg( void ) : String None Will return the most recent error message returned by PROJ (if any).
NULL will be returned if there is no curently pending PROJ error.
PROJ_GetDatabasePath( void : String) None Will return the currently set pathname leading to the private PROJ's SQLite database.
NULL will be returned if there is no private PROJ's SQLite database currently connected.
PROJ_SetDatabasePath ( new_path String ) : String
  • new_path: a relative or absolute pathname leading to a valid PROJ's SQLite database.
Will change the currently set pathname leading to the private PROJ's SQLite database.
NULL will be returned if the passed path is invalid, otherwise the path of the currently set private PROJ's SQLite database will be returned.
PROJ_AsWKT ( auth_name String , auth_srid Integer ) : String
PROJ_AsWKT ( auth_name String , auth_srid Integer , wkt_style String ) : String
PROJ_AsWKT ( auth_name String , auth_srid Integer , wkt_style String , indented Boolean ) : String
PROJ_AsWKT ( auth_name String , auth_srid Integer , wkt_style String , indented Boolean , indentation Integer ) : String
  • auth_name and auth_srid identify the intended CRS.
    auth_name can be NULL, and in this case EPSG will be assumed.
  • the optional argument style determines which specific WKT format should be adopted, and must be one between GDAL, ESRI, ISO-2015 or ISO-2018 (this latter being the default setting).
  • the optional argument indented if set to TRUE will nicely format a multiline WKT expression, otherwise a single monolithic line lacking any white-space or new-line will be printed (the default is TRUE).
  • the optional argument indentation determines how many white-spaces are to be used for indenting (only meaningful if indented=TRUE; the default values is 4).
Will return the WKT expression corresponding to a given CRS; the definitions will be taken directly from the private PROJ's own database.
NULL will be returned on failure or on invalid arguments.
PROJ_GuessSridFromWKT ( wkt_expr String ) : Integer
  • wkt_expr: the WKT expression to be evaluated.
Will possibly return the SRID value corresponding to a given WKT expression defining a CRS.
-1 will be returned if no CRS supported by PROJ.6 matches the WKT expression.
NULL will be returned on invalid argument.
PROJ_GuessSridFromSHP ( filename String ) : Integer
  • filename: the absolute or relative path leading to some Shapefile.
    Note: exactley as required by ImportSHP() filename must omit any .shp, .shx, .dbf or .prj suffix.
Will possibly return the SRID value corresponding to the CRS defined by the .PRJ member of the Shapefile.
-1 will be returned if no CRS supported by PROJ.6 matches PRJ member of the Shapefile.
NULL will be returned on invalid argument.
Please note well: this SQL function opend the door to many potential security issues, and thus is always disabled by default.
Explicitly setting the environment variable SPATIALITE_SECURITY=relaxed is absolutely required in order to enable this function.

Note: all the above SQL Functions will be available only when Spatialite-5.0.0 has been built against PROJ.6 (or any subsequent version).
If SpatiaLite-5.0.0 has been built instead against any previosus version (as e.g. PROJ.4) any attempt to call one of these SQL Functions will simply return a no such function SQL error.

Practical examples:
SELECT PROJ_GetDatabasePath();  -- retrieving the currently set PROJ's own database
------------------------------
/usr/local/share/proj/proj.db

SELECT PROJ_SetDatabasePath('/home/sandro/not_existing_proj.db'); -- non existing database
------------------------------
NULL

SELECT PROJ_GetLastErrorMsg(); -- retrieving the failure cause
------------------------------
proj_context_set_database_path: Open of /home/sandro/not_existing_proj.db failed

SELECT PROJ_SetDatabasePath('/home/sandro/valid_proj.db'); -- valid database
------------------------------
/home/sandro/valid_proj.db

SELECT PROJ_AsWKT('EPSG', 4326); -- default settings: ISO-2018 style, indented with indentation 4
------------------------------
GEODCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    AREA["World"],
    BBOX[-90,-180,90,180],
    ID["EPSG",4326]]

SELECT PROJ_AsWKT('EPSG', 4326, 'ESRI', 0, 0); -- ESRI style, monolithic line
------------------------------
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]

SELECT PROJ_GuessSridFromWKT('GEOGCS["GCS_Monte_Mario_Rome",DATUM["D_Monte_Mario",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Rome",12.4523333333333],UNIT["Degree",0.0174532925199433]]'); 
------------------------------
4806

SELECT PROJ_GuessSridFromSHP('/home/sandro/tuscany_shp/prov2011'); -- remember: requires to explicitly set SPATIALITE_SECURITY=relaxed
------------------------------
3003



Important notice for Windows users

PROJ.6 critically depends on its own private SQLite database (proj.db) containing all relevant definitions about Ellipsoids, Prime Meridians, CRSes, Transformations and alike.
If PROJ.6 is unable to correctly establish a connection to this database it will be severely limited and will not be able to correctly behave as expected.

  • Short conclusion: installing the software alone is not enough.
    The PROJ.6 private database proj.db must be properly installed as well, in order to ensure that anything runs smoothly.

This isn't usually a big issue on Linux and Unix-like platforms, where a rational and very clear filesystem layout exists.
On these operating systems the package manager (or make install) will automatically take care to install proj.db on the most appropriate directory (usually as /usr/share/proj/proj.db or as /usr/local/share/proj/proj.db) and that's all.

Things are unhappily a little bit more difficults on Windows platforms. The user itself is responsible for properly installing proj.db So it becomes critical understanding the basic rules adopted by libspatialite in order to properly locate proj.db on Windows.

Where proj.db is expected to be found on Windows

  1. the first place where proj.db will be searched is the same folder form where the EXE binary was initially loaded.
    Example: assuming that you are currently executing C:\myprogs\spatialite\bin\spatialite.exe
    • then an attempt will be made to connect C:\myprogs\spatialite\bin\proj.db
  2. a second attempt will be made on behalf of the Public folder.
    Example: an attempt will be made to connect C:\Users\Public\spatialite\proj\proj.db
  3. a third (and last) attempt will be made on behalf of the User folder.
    Example: an attempt will be made to connect C:\Users\sandro\spatialite\proj\proj.db

An useful diagnostic check

SELECT PROJ_GetDatabasePath();
  • if NULL is returned, then PROJ.6 is definitely unable to connect to its own private SQLite database proj.db
    You have to carefully verifiy that it's really installed on one of the expected standard locations.
  • otherwise the path leading to the currently connected proj.db will be returned.



Using the environment variable SPATIALITE_PROJ_DB_PATH

If the environment variable SPATIALITE_PROJ_DB_PATH is set, then libspatialite will simply attempt to connect the database pointed by this variable.
Such a capability is supported not only on Windows but also on Linux (and on any other platform).

Linux example:
  • export "SPATIALITE_PROJ_DB_PATH=/home/sandro/aux/proj.db"
    echo $SPATIALITE_PROJ_DB_PATH
    spatialite


Windows example:
  • SET SPATIALITE_PROJ_DB_PATH=C:\Users\sandro\aux\proj.db
    echo %SPATIALITE_PROJ_DB_PATH%
    spatialite_gui





back