BEGIN:VCALENDAR VERSION:2.0 PRODID:-//pretalx//talks.staging.osgeo.org//foss4g-2024-academic-track//spe aker//LA3NZT BEGIN:VTIMEZONE TZID:-03 BEGIN:STANDARD DTSTART:20000101T000000 RRULE:FREQ=YEARLY;BYMONTH=1 TZNAME:-03 TZOFFSETFROM:-0300 TZOFFSETTO:-0300 END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:pretalx-foss4g-2024-academic-track-TWFHZB@talks.staging.osgeo.org DTSTART;TZID=-03:20241204T150000 DTEND;TZID=-03:20241204T153000 DESCRIPTION:1. Introduction & Related Work\nPedestrian mobility is crucial in urban environments\, and its promotion can contribute to the achievemen t of many UN SDGs (Adriazola-Steil et al.\, 2021). Mapping\, which enables public scrutiny and long-term optimized planning\, is indispensable in th is context. \nWith the widespread availability of a large set of Open Stre et-Level Imagery\, such as Mapillary\, there is now a significant opportun ity that presents significant challenges for data extraction (Ma et al.\, 2019). The richness of detail in these urban landscape representations can help us better understand the peculiarities of urban environments. The s cope of this project is to make use of them for the study of pathways\, fo cusing in particular on the verification of their existence\, their catego rization (road\, sidewalk\, or general footpath)\, and the identification of their surface material. Since pedestrian crossings are part of the car and pedestrian network\, and road characteristics (such as material and wi dth) significantly impact pedestrian safety\, it is worth noting that the study of roads is also fundamental to pedestrian infrastructure (Mesfin & Denbi\, 2022). However\, the central aspect remains sidewalks\, often the most ubiquitous type of pedestrian thoroughfare (Kim\, 2019)\, a valuable space for sociability (Osman\, 2016)\, whose "health" is symptomatic of ho w pedestrian-friendly the city is (Mesfin & Denbi\, 2022). \nDespite the importance of knowledge about them for understanding the urban environment \, pavements are often poorly mapped (Vestena et al.\, 2023). Even fewer w orks delve into the problem of pathway surface identification: Zhou et al. (2023) used conventional Convolutional Neural Networks (CNN) to identify pavement classes limited to asphalt\, gravel\, and cement\; Zhang et al. ( 2022) used a similar approach to identify asphalt-only damage such as "pot holes" and "patches"\; only Mesquita et al. (2022) and Hosseini et al. (20 22) made pixel-level identification\, albeit the first one was limited to only "paved" and "unpaved" categorization\, while the second one notwithst anding having a more wholesome approach has its categorization focused on a New-York centered classes and only classifies sidewalks. There is still a gap in approaches considering standardized surface types and generalized path detection.\n2. The Framework\n We propose the Deep Pavements Framewo rk to address these issues. It is a modular project\, with each part contr ibuting to the solution of the different challenges. The first module is t he Surface-patches Dataset\, labeled following the OpenStreetMap surface=* tags standard\, supporting the categories of "asphalt"\, "cobblestone"\, "compacted"\, "concrete plates"\, "concrete"\, "grass"\, "gravel"\, "groun d"\, "paving stones"\, and "sett" currently\, The second module is the Run ner\, to process the data for a given region. The third is the Sample-pick er\, which generates random samples for dataset generation. There is also the Sample-labeler to label samples interactively and a central module to guide the potential user into the project's usage. Each module relies upon a different set of dependencies\, thus reducing runtime issues. It is imp ortant to highlight the primary usage of containerizing engines\, i.e.\, D ocker.\nBeyond modularity\, Deep Pavements has as core design principles: 1) complete openness\, meaning that all its dependencies must have a broad ly permissive license that enables even for commercial usage\; 2) the ease of reproducibility by a straightforward setup with an as such and well-do cumented\, command line interface (CLI)\; 3) evolvability\, the State-of-T he-Art (SOTA) algorithms are constantly changing\, then at each new releas e of runner/sample-picker images a new set of tools can be employed while keeping the same CLI\; nevertheless the user would still be able to use a previous release\; 4) Standard-anchored with classes that had been agreed upon by the broad crowd-sourced knowledge base constituted by OSM communit y (Rahmig & Kludge\, 2013\; Mooney & Minghini\, 2017). \nThe implementatio n of the main modules (Runner/Sample Picker) uses open-vocabulary AI algor ithms to perform the data extraction\, following this workflow: 1) Groundi ng Dino (Liu et al.\, 2023) based on a free-input\, detects the bounding b ox of the detections\; 2) Segment Anything (Kirillov et al.\, 2023) transf orms it into a mask\; 3) 3 different versions of the CLIP algorithm (Radf ord et al.\, 2021) tests if the detection is not a hallucination\; 4) If c onfirmed\, a specialized version of CLIP is used for finally check the sur face material using the cheaply clipped biggest rectangle in the detection \, assuring the usage of the patch whose texture got less hindered by the effects of perspective (Lederman & Klatzky\, 1995) being no-data pixels\, free\, which is another potential source for classification jeopardy (Kang et al.\, 2019). \nThe workflow results from experiments for the presented design mainly point out the need for hallucination testing. This procedur e acts as a shield for one of the main drawbacks of open vocabulary algori thms (Ben-Kish et al.\, 2024). The use of this particular type of algorith m was essential due to its flexibility (Wang et al.\, 2024) and the potent ial for better semantic understanding of the scene due to its embedded lan guage model (Eichstaedt et al.\, 2021). There is also the possibility of a llowing the user to opt out of some or all of OSM standardized classes\, w hich can be helpful in some scenarios with regional uniqueness. \n3. Fina l Remarks\nDeep Pavements is an innovative and comprehensive toolset under continuous development with all modules maintained at Github\, with the c entral module available at . The framework enables creating pavement data that is seamlessly plugable into OSM.\n As future challenges\, we plan to filter lousy qual ity images that can happen on the primary data source (Ma et al.\, 2019) t o detect other visually-identifiable pavement traits such as its decay\, s tandardizing with OSM tags such as smoothness=*\; to integrate photogramme tric tools for obtaining additional modeling of pavements\, having as main interest in measuring pavement width\, one of the most relevant info for accessibility assessment (Kim et al.\, 2011). DTSTAMP:20260513T163002Z LOCATION:Room III SUMMARY:Deep Pavements Framework: Combining Ai Tools And Collaborative Terr estrial Imagery For Pathway Mapping - Kauê de Moraes Vestena URL:https://talks.staging.osgeo.org/foss4g-2024-academic-track/talk/TWFHZB/ END:VEVENT END:VCALENDAR