Topology-Geometry: an advanced tutorial

TopoLayers and TopoFeatures

a conceptual reference framework

• an ISO Topology is just intended to support a consistent set of Nodes, Edges and Faces primitives fully covering the plane. Topology primitives correspond to pure geometries, and can never be directly associated to a corresponding set of informational attributes. Both Node and Edge primitives directly correspond to a real geometry, but Face primitives simply correspond to a virtual geometry thus necessarily requiring to be dynamically reconstructed every time that they are referenced (and this could easily become a real bottleneck).
• A GeoTable (alias layer, to use the common GIS terminology) on the other hand, is always based on many distinct features, and for each feature a Geometry with associated set of information attributes.
• there is a further difference to be taken into account: Topology primitives necessarily are elementary. Feature Geometries can usually represent some complex type: MULTIPOINT, MULTILINESTRING, MULTIPOLYGON and possibly a GEOMETRYCOLLECTION. So it is possible that a Feature Geometry does not necessarily corresponds to a single Topology primitive ; a Feature Geometry could easily be derived from the collection of many different Topology primitives.
• that's not all: many different GeoTables could eventually be derived from a single Topology; and the same Topology primitive could play a completely different role in each GeoTable.
  A simple sample: imagine a Topology corresponding to some land registry map where many potential layers such as: buildings, agricultural land, populated places, administrative boundaries, roads, ponds, rivers, fences will exist.
  In such a context a fence could easily correspond to a Feature within the "fences" layer, could, not only, separate two adjacent agricultural areas, but also, be part of some administrative boundary.
• TopoLayers and TopoFeatures represent a complex and flexible structure intended to establish a permanent relationship between Topology Primitives and GeoTable Features.
  • each TopoLayer is uniquely identified by its name and directly corresponds to a single GeoTable to be exported from the Topology
    An arbitrary number of TopoLayers can be created on the same Topology, and each one of them will act as a separate container.
  • each TopoFeature is uniquely identified by its fid (feature-id) and corresponds to a single Feature containing an individual set of informational attributes.
    The Geometry for each TopoFeature is always indirectly defined by specifying a list of Topology primitives individually referenced by their IDs.
    Such a geometry is considered to be virtual, meaning that it will be created through the collection/aggregation of all referenced Topology primitives, when exported to the destination GeoTable.
• The TopoLayers / TopoFeatures structure is practically implemented as a set of several closely related DB-tables (the respective layouts of which we'll examine later in full detail):
  • <topo-prefix>_topolayers: this first table simply is a catalog of all TopoLayers supporting the same Topology.
    Each TopoLayer is identified by its name and by its ID; both of which must be unique.
  • <topo-prefix>_topofeatures: this second table is intended to permanently store all cross-references existing between Topology primitives and TopoFeatures.
  • <topo-prefix>_topofeatures_<topolayer-id>: for each single TopoLayer a separate table is required. The actual relationship is established via the TopoLayer-ID suffix.
    All these tables are intended to permanently store the informational attributes for each TopoFeature on the same TopoLayer. Different TopoLayers will obviously support a different set of informational attributes thus requiring an individual table layout.
    A Primary Key of the INTEGER type named fid is always expected to declared and is intended to be an unique identifier for each TopoFeature.
• Several SQL functions are specifically intended to support processing operations based on TopoLayers and TopoFeatures; we'll examine them later in full detail.

1. download the sample DB-file from here
   It contains Census data (2011) kindly released by ISTAT under CC-BY license terms.
   The original datasets have been slightly rearranged in a more convenient form. All geometries are in the SRID 32632 (WGS 84 / UTM zone 32N):
   • census_2011: all Census Areas (2011) covering Tuscany.
     Note: several Census Areas are completely uninhabited (lakes, marshlands, high mountains and alike).
   • ppl_2011: Populated Places (defined as an aggregation of Census Areas).
     Note: not all Census Areas belong to a Populated Place: there are many self-standing dispersed rural areas.
   • com_2011: Tuscany Municipalities / Local Councils 2011 (defined as an aggregations of Census Areas).
   • com_2014_15: few new Municipalities created during the years 2014/15, through the merging of pre-existing smaller Municipalities.
   • prov_2011: Tuscany Provinces / Counties (defined as an aggregation of Municipalities).
   • reg_2011: Tuscany Region (defined as an aggregation of Provinces).
   • only the table census_2011 has geometries; other administration levels are defined by relation codes.
   • Attention: during this sample (as in others), we will never directly open this database, but only attach it to the database we are creating.
2. uncompress the downloaded db-file (tuscany-census-2011.sqlite).
3. now start a SpatiaLite session using your preferred SpatiaLite front end tool:
   • Connect to a new (empty) DB-file named tuscany-topo-2011.sqlite, to which tuscany-census-2011.sqlite will be attached to.

SELECT CreateTopology('census2011', 32632, 0, 0);
1

ATTACH DATABASE "./tuscany-census-2011.sqlite" AS istat;

SELECT TopoGeo_FromGeoTable('census2011', 'istat', 'census_2011', NULL, NULL, 512);
1

SELECT ST_ValidateTopoGeo('census2011');
NULL

SELECT * FROM TEMP.census2011_validate_topogeo;

SELECT TopoGeo_CreateTopoLayer('census2011', 'istat', 'census_2011', NULL, 'census_areas');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_ppl_2011', 'ppl');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_com_2011', 'com');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_prov_2011', 'prov');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_reg_2011', 'reg');
1

DETACH DATABASE istat;

• we'll create a new 2D Topology named census2011 and located into SRID 32632.
• then we'll attach the tuscany-topo-2011.sqlite external DB-file.
• after-which we'll populate the census2011 Topology by importing the istat.census_2011 GeoTable.
• As final task we'll check if this Topology is fully valid.

1. First invoke TopoGeo_CreateTopoLayer() in order to completely define a first TopoLayer representing Census Areas.
   Recall: the istat.census_2011 is an existing 'GeoTable', being used as our initial dataset, since it already contains both the information attributes and geometries needed, for which each 'Census Area' is expected to correspond to a single Topology Face. (later we'll examine in more depth, how the TopoLayer / TopoFeature relationships have actually been defined).
2. then we'll continue by defining several further TopoLayers corresponding to the Populated Places, Municipalities, Provinces and Region administration levels.
   Recall: all these administration levels in the initial dataset are simply defined by relation codes intended to collect/aggregate the lower level entities where, in this case, have no geometries at all. Being so, we cannot create a fully defined TopoLayer by calling TopoGeo_CreateTopolayer().
   By invoking TopoGeo_InitTopoLayer() only a partially defined TopoLayer will initialized, but which will be complement later any missing information - thus achieving, with a second step - a properly working TopoLayer (as we'll see later in full detail).
3. finally we'll detach the no longer needed external db-file.

SELECT * FROM census2011_topolayers;

SELECT * FROM census2011_topofeatures_2;

SELECT * FROM census2011_topofeatures_4;

SELECT * FROM census2011_topofeatures;

exporting the Census Areas TopoLayer

We'll start by exporting first the only completely defined TopoLayer we have at this point in the test DB-file, i.e. census_areas SELECT TopoGeo_ExportTopoLayer('census2011', 'census_areas', 'out_census_2011', 1); As you can easily check by yourself a new GeoTable named out_census_2011 has been created by TopoGeo_ExportTopoLayer(): this GeoTable contains all TopoFeatures defined into the TopoLayer. each TopoFeature faithfully preserves its initial information attributes. all Geometries are precisely built by aggregating the corresponding Topology primitives as specified by TopoLayer / TopoFeatures relationships. The image examplifies a choropleth centered around the Island of Elba and based on the out_census_2011 GeoTable: all Census Areas are displayed by adopting a colour directly corresponding to their population density.

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 2, a.fid
FROM census2011_topofeatures_2 AS a
JOIN census2011_topofeatures_1 AS b ON (b.cod_ppl = a.cod_ppl)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 1 AND c.fid = b.fid);

• census2011_topofeatures_1 is the TopoFeatures table corresponding to Census Areas.
• census2011_topofeatures_2 is the corresponds to Populated Places; the clause b.cod_ppl = a.cod_ppl will relationally join each Populated Place to its underlaying Census Areas.
• so we can duly insert into the TopoFeatures-geometry table (i.e. census2011_topofeatures) a new level of references to Topology primitives by simply copying all definitions already stored into the previous hierarchical level.
  And to do such a thing we simply have to read from topolayer=1 then inserting into topolayer=2 after setting the appropriate fid values.
  Really not a difficult task.

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 3, a.fid
FROM census2011_topofeatures_3 AS a
JOIN census2011_topofeatures_1 AS b ON (b.cod_com = a.cod_com)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 1 AND c.fid = b.fid);

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 4, a.fid
FROM census2011_topofeatures_4 AS a
JOIN census2011_topofeatures_3 AS b ON (b.cod_prov = a.cod_prov)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 3 AND c.fid = b.fid);

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 5, a.fid
FROM census2011_topofeatures_5 AS a
JOIN census2011_topofeatures_4 AS b ON (b.cod_reg = a.cod_reg)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 4 AND c.fid = b.fid);

step #1: exporting the Region TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'reg', 'out_reg_2011', 1); We are now definitely ready to export any other Administrative Level. For the sake of clarity we'll follow a top-bottom order, so we'll start by exporting first the whole Tuscany.
step #2: exporting the Provinces TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'prov', 'out_prov_2011', 1); We'll continue by exporting all the Tuscany Provinces.
step #3: exporting the Municipalities TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'com', 'out_com_2011', 1); Then we'll export all the Tuscany Municipalities. For better clarity the figure shows a magnified detail centered around the Island of Elba.
step #4: exporting the Populated Places TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'ppl', 'out_ppl_2011', 1); And we'll finally export all Populated Places. Conclusion: we started this advanced tutorial by importing into a Topology just a single layer (Census Areas) and now we've finished by producing a complete set of administrative boundaries at different hierarchical levels: Populated Places Municipalities Provinces Region

ATTACH DATABASE "./tuscany-census-2011.sqlite" AS istat;

CREATE TABLE com2015 AS
SELECT * FROM istat.pop_com_2011;

INSERT INTO com2015
SELECT a.cod_com, a.cod_prov, a.cod_istat, a.cod_cadastre, a.name, a.townhall_addr,
    Sum(b.pop_2011), Sum(b.m_2011), Sum(b.f_2011)
FROM istat.com_2014_15 AS a
JOIN istat.pop_com_2011 AS b ON (b.cod_com IN (a.old_cod_com_1, a.old_cod_com_2))
GROUP BY a.cod_com;

DETACH DATABASE istat;

1. we'll attach yet another time the initial tuscany-census-2011.sqlite DB-file.
2. we'll create a new com2015 table by just just copying the istat.pop_com_2011 View.
3. and finally we'll append to the same table the new Municipalities from the istat.com_2014_15 table; during this step we'll compute the Population 2011 census values.
   Note: after completing the above steps now the com2015 table contains both suppressed and new Municipalities

SELECT TopoGeo_InitTopoLayer('census2011', NULL, 'com2015', 'com2015');

INSERT INTO census2011_topofeatures
SELECT NULL, a.node_id, a.edge_id, a.face_id, 6, a.fid
FROM census2011_topofeatures AS a
JOIN census2011_topofeatures_3 AS b ON (a.topolayer_id = 3 AND a.fid = b.fid);

UPDATE census2011_topofeatures
SET fid = (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com = 51040)
WHERE topolayer_id = 6 AND fid IN (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com in (51009, 51029));

UPDATE census2011_topofeatures
SET fid = (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com = 51041)
WHERE topolayer_id = 6 AND fid IN (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com in (51032, 51036));

SELECT TopoGeo_ExportTopoLayer('census2011', 'com2015', 'out_com_ar_2015', 1, 1);

SELECT TopoGeo_InsertFeatureFromTopoLayer ('census2011', 'com2015', 'out_com_ar_2015', fid)
FROM census2011_topofeatures_6
WHERE cod_prov = 51 AND cod_com NOT IN (51009, 51029, 51032, 51036);

Municipalities in the Province of Arezzo: 2011	Municipalities in the Province of Arezzo: 2015