BEGIN:VCALENDAR VERSION:2.0 PRODID:-//pretalx//talks.staging.osgeo.org//foss4g-2022//speaker//QS7Y88 BEGIN:VTIMEZONE TZID:CET BEGIN:STANDARD DTSTART:20001029T040000 RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10 TZNAME:CET TZOFFSETFROM:+0200 TZOFFSETTO:+0100 END:STANDARD BEGIN:DAYLIGHT DTSTART:20000326T030000 RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3 TZNAME:CEST TZOFFSETFROM:+0100 TZOFFSETTO:+0200 END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:pretalx-foss4g-2022-9G3YGD@talks.staging.osgeo.org DTSTART;TZID=CET:20220825T101000 DTEND;TZID=CET:20220825T101500 DESCRIPTION:When you care about data integrity of spatial data you need to know about the limitations/weaknesses of using simple feature datatype in your database. For instance https://land.copernicus.eu/pan-european/corine -land-cover/clc2018 contains 2\,377\,772 simple features among which we fi nd 852 overlaps and 1420 invalid polygons. For this test I used “ESRI FG DB” file and gdal for import to postgis. We find such minor overlaps an d gaps quite often\, which might not be visible for the human eye. The pro blem here is that it covers up for real errors and makes difficult to enfo rce database integrity constraints for this. Close parallel lines also se ems to cause Topology Exception in many spatial libraries.\n\nA core probl em with simple features is that they don't contain information about the r elation they have with neighbor features\, so integrity of such relations is hard to constraint. Another problem is mixing of old and new data in th e payload from the client. This makes it hard and expensive to create clie nts\, because you will need a full stack of spatial libraries and maybe a complete locked exact snapshot of your database on the client side. Anothe r thing is that a common line may differ from client to client depending o n spatial lib\, snapTo usage\, tolerance values and transport formats.\n\n In 2022 many system are depending on live updates also for spatial data. So it’s big advantage to be able to provide a simple and “secure” AP I’s with fast server side integrity constraints checks that can be used from a standard web browser. When we have this checks on server side we wi ll secure the equal rules across different clients.\n\nIs there alternativ es that can secure data integrity in a better way? Yes\, for instance Post gis Topology. The big difference is that Postgis Topology has more open st ructure that is realized by using standard database relational features. T his lower the complexity of the client and secures data integrity. In the talk “Use Postgis Topology to secure data integrity\, simple API and cle an up messy simple feature datasets.” we will dive more into the details off Postgis Topology\nBuilding an API for clients may be possible using s imple features\, but it would require expensive computations to ensure top ological integrity but to solve problem with mixing of new and old borders parts can not be solved without breaking the polygon up into logical part s. Another thing is attribute handling\, like if you place surface partly overlapping with another surface should that have an influence on the attr ibutes on the new surface.\n\nWe need to focus more on data integrity and the complexity and cost of creating clients when using simple feature\, be cause the demands for spatial data updated in real time from many differen t clients in a secure and consistent way will increase. This will be main focus in this talk. DTSTAMP:20260405T122050Z LOCATION:Room 9 SUMMARY:Data integrity risks when using simple feature - Lars Opsahl URL:https://talks.staging.osgeo.org/foss4g-2022/talk/9G3YGD/ END:VEVENT BEGIN:VEVENT UID:pretalx-foss4g-2022-H7ZEBF@talks.staging.osgeo.org DTSTART;TZID=CET:20220825T120000 DTEND;TZID=CET:20220825T123000 DESCRIPTION:In Postgis Topology a merge of two surfaces does not involve sp atial operations\, since \nthe surface to border relation has foreign key structures in the database. This means that the border of the new object i s spatially not touched/changed when two surfaces are merged. With simple feature the common border must be computed on the fly\, which again may in volve snapTo and cause tiny overlaps and gaps. \n\nWith Postgis Topology y ou can easily make an API where the client only sends new borders which is a key issue to secure data integrity. This secures that old border are no t are not moved by a client error or the by simple transport format\, beca use existing points are never not passed back to the server. Postgis Topol ogy makes it easy for the server to work with those new borders(delta)\, b ecause there are standard methods for this in Postgis Topology and all rel ations between border and surfaces are stored in the database. Postgis Top ology also has validation routines in addition to using standard database constraints to secure a healthy system. \n\nThe principles that Postgis To pology is based on was used in spatial system many years ago\, but one pro blem was to keep the border line work nice and clean and not end up in a s paghetti. So one of the first things we did together with Sandro Santilli was to create methods on top of Postgis Topology to avoid this\, by throw ing away any border parts that does not contribute to a new “valid” su rface.\n\nPostgis Topology is built on a relational database model that is based on SQL-MM part 3. Your own domain data are easily linked to border\ , surface objects with more. For instance to check domain attributes on a surface on the other side of a border is not spatial query but a standard relational query. \n\nThe following projects will also be touched in this talk:\n\nhttps://gitlab.com/nibioopensource/pgtopo_update_sql (Functions u sing Postgis Topology to make it easy to create spatial update clients.)\n \nhttps://github.com/strk/qgis_pgis_topoedit (Postgis Topology is very wel l integrated with QGIS.)\n\nhttps://github.com/larsop/resolve-overlap-and- gap (Show how we clean up\, simplify\, generalize simple feature tables with millions of rows using Postgis Topology)\n\nIs relational database st ructure a good choice for Postgis Topology? Yes I will mean and since it ’s also linked up SQL-MM part3 and not a random private structure and wi th all great Postgis functions available this is very good combination. Yo u may take take glance at https://www.ibm.com/ibm/history/ibm100/us/en/ico ns/reldb/ and other articles about relational databases. \n\nThe plan now is to build a full ecosystem around Postgis Topology with a generic client to support declarative rules\, where you can define attributes\, rules fo r attribute handling and how to deal with overlap and gap. \n\nAll the wor k NIBIO has done/is doing here would not have been possible with out the g reat support from Sandro Santilli. DTSTAMP:20260405T122050Z LOCATION:Room Limonaia SUMMARY:Postgis Topology to secure data integrity\, simple API and clean up messy simple feature datasets. - Lars Opsahl\, sandro santilli\, Mattia N atali URL:https://talks.staging.osgeo.org/foss4g-2022/talk/H7ZEBF/ END:VEVENT END:VCALENDAR