Topology-Geometry: an advanced tutorial

TopoLayers and TopoFeatures

a conceptual reference framework

• an ISO Topology is just intended to support a consistent set of Nodes, Edges and Faces primitives fully covering the plane. Topology primitives correspond to pure geometries, and can never be directly associated to a corresponding set of information attributes. Both Node and Edge primitives directly correspond to a real geometry, but Face primitives simply correspond to a virtual geometry thus necessarily requiring to be dynamically reconstructed every time that they are referenced (and this could easily become a real bottleneck).
• on the other hand a GeoTable (aka layer, as in the common GIS parlance) is always based on many distinct features, and for each feature a Geometry and an associated set of information attributes are expected to be declared.
• there is further difference to be taken in proper account: Topology primitives necessarily are elementary. Feature Geometries can usually represent some complex type: MULTIPOINT, MULTILINESTRING, MULTIPOLYGON and, may be, GEOMETRYCOLLECTION. So it should be absolutely clear that a Feature Geometry does not necessarily corresponds to a single Topology primitive ; a Feature Geometry could easily be derived from aggregating many different Topology primitives.
• that's not all: many different GeoTables could eventually be derived from a single Topology; and the same Topology primitive could play a very different role in everyone of these GeoTables.
  Just a basically simple example: imagine a Topology corresponding to some land registry map. We'll surely have many potential layers: buildings, agricultural land, populated places, administrative boundaries, roads, ponds, rivers, fences and so on.
  In such a context a fence could easily correspond to a Feature within the "fences" layer, could probably delimit two adjacent agricultural areas and could eventually be part of some administrative boundary.
• TopoLayers and TopoFeatures represent a complex and flexible structure intended to establish permanent explicit correspondences between Topology Primitives and GeoTable Features.
  • each TopoLayer is uniquely identified by its name and directly corresponds to a single GeoTable to be exported from the Topology
    An arbitrary number of TopoLayers can be created on the same Topology, and each one of them will act as a separate container.
  • each TopoFeature is uniquely identified by its fid (feature-id) and corresponds to a single Feature containing an individual set of information attributes.
    The Geometry for each TopoFeature is always indirectly defined by specifying a list of Topology primitives individually referenced by their IDs.
    Such a geometry obviously is a virtual one, and will be actually materialized (by aggregating all referenced Topology primitives) only when exporting the destination GeoTable.
• The TopoLayers / TopoFeatures structure is practically implemented as a set of several closely related DB-tables )we'll examine later their respective layouts in full detail):
  • <topo-prefix>_topolayers: this first table simply is a catalog of all TopoLayers supporting the same Topology.
    Each TopoLayer is identified by its name and by its ID; both identifiers are granted to be unique.
  • <topo-prefix>_topofeatures: this second table is intended to permanently store all cross-references existing between Topology primitives and TopoFeatures.
  • <topo-prefix>_topofeatures_<topolayer-id>: for each single TopoLayer a separate table is required. The actual correspondence is established via the TopoLayer-ID suffix.
    All these tables are intended to permanently store the information attributes for each TopoFeature in the same TopoLayer. Different TopoLayers will obviously support a different set of information attributes thus requiring an individual table layout.
    Anyway all these tables are always expected to declare a Primary Key of the INTEGER type named fid and intended to be an unique identifier for each TopoFeature.
• Several SQL functions are specifically intended to support processing operations based on TopoLayers and TopoFeatures; we'll examine later all them in full detail.

a quick practical exercise

1. download the sample DB-file from here
   It contains Census data (2011) kindly released by ISTAT under CC-BY license terms.
   The original datasets have been slightly rearranged in a more convenient form. All geometries are in the SRID 32632 (WGS 84 / UTM zone 32N):
   • census_2011: all Census Areas (2011) covering Tuscany.
     Note: several Census Areas are completely uninhabited (lakes, marshlands, high mountains and alike).
   • ppl_2011: Populated Places (defined as an aggregation of Census Areas).
     Note: not all Census Areas belong to a Populated Place: there are many self-standing dispersed rural areas.
   • com_2011: Tuscany Municipalities / Local Councils 2011 (defined as an aggregations of Census Areas).
   • com_2014_15: few new Municipalities created during years 2014/15 by merging pre-existing smaller Municipalities.
   • prov_2011: Tuscany Provinces / Counties (defined as an aggregation of Municipalities).
   • reg_2011: Tuscany Region (defined as an aggregation of Provinces).
   • just census_2011 alone supports geometries; any other administrative level is simply defined by relational codes.
2. uncompress the downloaded db-file (tuscany-census-2011.sqlite).
3. now you can finally start a SpatiaLite session using your preferred SpatiaLite front end tool:
   • Attention: you must not directly connect to tuscany-census-2011.sqlite; you are expected instead to connect a new (empty) DB-file named tuscany-topo-2011.sqlite

SELECT CreateTopology('census2011', 32632, 0, 0);
1

ATTACH DATABASE "./tuscany-census-2011.sqlite" AS istat;

SELECT TopoGeo_FromGeoTable('census2011', 'istat', 'census_2011', NULL, 0, 512, -1);
1

SELECT ST_ValidateTopoGeo('census2011');
NULL

SELECT * FROM TEMP.census2011_validate_topogeo;

SELECT TopoGeo_CreateTopoLayer('census2011', 'istat', 'census_2011', NULL, 'census_areas');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_ppl_2011', 'ppl');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_com_2011', 'com');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_prov_2011', 'prov');
1

SELECT TopoGeo_InitTopoLayer('census2011', 'istat', 'pop_reg_2011', 'reg');
1

DETACH DATABASE istat;

• we'll create a new 2D Topology named census2011 and located into SRID 32632.
• then we'll attach the tuscany-topo-2011.sqlite external DB-file.
• now we'll populate the census2011 Topology by importing the istat.census_2011 GeoTable, and then we'll immediately check if this Topology is fully valid.

1. so we'll duly invoke TopoGeo_CreateTopoLayer() in order to completely define a first TopoLayer representing Census Areas.
   Recall: the istat.census_2011 is the unique real GeoTable we have in our initial dataset, because it contains both information attributes and geometries; and each Census Area is expected to directly correspond to a single Topology Face. (we'll examine later in more depth how TopoLayer / TopoFeature relationships have been actually defined).
2. then we'll continue by defining several further TopoLayers respectively corresponding to Populated Places, Municipalities, Provinces and Region administrative levels.
   Recall: all these administrative levels in the initial dataset are simply defined by relational codes intended to aggregate lower level entities; in this case there are no explicit geometries at all. So are are not directly allowed to create a fully defined TopoLayer, and calling TopoGeo_CreateTopolayer() is just impossible because we lack any appropriate Geometry column.
   Anyway we can invoke TopoGeo_InitTopoLayer(); this will initialize an only partially defined TopoLayer, but we can easily complete in a second time any missing information so to get a properly working TopoLayer (as we'll see later in full detail).
3. finally we'll detach the now useless external db-file.

TopoLayers / TopoFeatures: layout of DB Tables and corresponding relationships

SELECT * FROM census2011_topolayers;

SELECT * FROM census2011_topofeatures_2;

SELECT * FROM census2011_topofeatures_4;

SELECT * FROM census2011_topofeatures;

TopoGeo_ExportTopoLayer: exporting a full TopoLayer into a GeoTable

exporting the Census Areas TopoLayer

We'll start by exporting first the only completely defined TopoLayer we have at this point in the test DB-file, i.e. census_areas SELECT TopoGeo_ExportTopoLayer('census2011', 'census_areas', 'out_census_2011', 1); As you can easily check by yourself a new GeoTable named out_census_2011 has been created by TopoGeo_ExportTopoLayer(): this GeoTable contains all TopoFeatures defined into the TopoLayer. each TopoFeature faithfully preserves its initial information attributes. all Geometries are precisely built by aggregating the corresponding Topology primitives as specified by TopoLayer / TopoFeatures relationships. The image examplifies a choropleth centered around the Island of Elba and based on the out_census_2011 GeoTable: all Census Areas are displayed by adopting a colour directly corresponding to their population density.

Building a full hierarchy of Administrative Levels by SQL statements

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 2, a.fid
FROM census2011_topofeatures_2 AS a
JOIN census2011_topofeatures_1 AS b ON (b.cod_ppl = a.cod_ppl)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 1 AND c.fid = b.fid);

• census2011_topofeatures_1 is the TopoFeatures table corresponding to Census Areas.
• census2011_topofeatures_2 is the corresponds to Populated Places; the clause b.cod_ppl = a.cod_ppl will relationally join each Populated Place to its underlaying Census Areas.
• so we can duly insert into the TopoFeatures-geometry table (i.e. census2011_topofeatures) a new level of references to Topology primitives by simply copying all definitions already stored into the previous hierarchical level.
  And to do such a thing we simply have to read from topolayer=1 then inserting into topolayer=2 after setting the appropriate fid values.
  Really not a difficult task.

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 3, a.fid
FROM census2011_topofeatures_3 AS a
JOIN census2011_topofeatures_1 AS b ON (b.cod_com = a.cod_com)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 1 AND c.fid = b.fid);

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 4, a.fid
FROM census2011_topofeatures_4 AS a
JOIN census2011_topofeatures_3 AS b ON (b.cod_prov = a.cod_prov)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 3 AND c.fid = b.fid);

INSERT INTO census2011_topofeatures
SELECT NULL, c.node_id, c.edge_id, c.face_id, 5, a.fid
FROM census2011_topofeatures_5 AS a
JOIN census2011_topofeatures_4 AS b ON (b.cod_reg = a.cod_reg)
JOIN census2011_topofeatures AS c ON (c.topolayer_id = 4 AND c.fid = b.fid);

step #1: exporting the Region TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'reg', 'out_reg_2011', 1); We are now definitely ready to export any other Administrative Level. For the sake of clarity we'll follow a top-bottom order, so we'll start by exporting first the whole Tuscany.
step #2: exporting the Provinces TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'prov', 'out_prov_2011', 1); We'll continue by exporting all the Tuscany Provinces.
step #3: exporting the Municipalities TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'com', 'out_com_2011', 1); Then we'll export all the Tuscany Municipalities. For better clarity the figure shows a magnified detail centered around the Island of Elba.
step #4: exporting the Populated Places TopoLayer SELECT TopoGeo_ExportTopoLayer('census2011', 'ppl', 'out_ppl_2011', 1); And we'll finally export all Populated Places. Conclusion: we started this advanced tutorial by importing into a Topology just a single layer (Census Areas) and now we've finished by producing a complete set of administrative boundaries at different hierarchical levels: Populated Places Municipalities Provinces Region

to be done

ATTACH DATABASE "./tuscany-census-2011.sqlite" AS istat;

CREATE TABLE com2015 AS
SELECT * FROM istat.pop_com_2011;

INSERT INTO com2015
SELECT a.cod_com, a.cod_prov, a.cod_istat, a.cod_cadrastre, a.name, a.townhall_addr,
    Sum(b.pop_2011), Sum(b.m_2011), Sum(b.f_2011)
FROM istat.com_2014_15 AS a
JOIN istat.pop_com_2011 AS b ON (b.cod_com IN (a.old_cod_com_1, a.old_cod_com_2))
GROUP BY a.cod_com;

DETACH DATABASE istat;

SELECT TopoGeo_InitTopoLayer('census2011', NULL, 'com2015', 'com2015');

INSERT INTO census2011_topofeatures
SELECT NULL, a.node_id, a.edge_id, a.face_id, 6, a.fid
FROM census2011_topofeatures AS a
JOIN census2011_topofeatures_3 AS b ON (a.topolayer_id = 3 AND a.fid = b.fid);

UPDATE census2011_topofeatures
SET fid = (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com = 51040)
WHERE topolayer_id = 6 AND fid IN (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com in (51009, 51029));

UPDATE census2011_topofeatures
SET fid = (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com = 51041)
WHERE topolayer_id = 6 AND fid IN (SELECT fid FROM census2011_topofeatures_6 WHERE cod_com in (51032, 51036));

SELECT TopoGeo_ExportTopoLayer('census2011', 'com2015', 'out_com_2015', 1, 1);

SELECT TopoGeo_InsertFeatureFromTopoLayer ('census2011', 'com2015', 'out_com_2015', fid)
FROM census2011_topofeatures_8
WHERE cod_prov = 51 AND cod_com NOT IN (51009, 51029, 51032, 51036);

AR 2011
AR 2015