1st Year of the Project / Intermediate Results Presentation
To further improve the outcome of the automatic enrichment tools we created a generic dashboard for crowdsourced enrichment of multimedia collections. The general architecture of the Django[1]-based dashboard is shown in the figure below. This set-up allows users to vote for different types of datalinks and to add new datalinks. Each datalink consists of a data object (such as an image) and a metadata object (e.g., its location, time period, keyword). The votes and the quality of the datalinks of each user are used to generate a credibility score. Based on this score, his/her contributions will have more (or less) impact on the UGESCO metadata generation.
General architecture of the Ugesco generic dashboard
for crowdsourced enrichment of multimedia collections.
The user interface of the dashboard is shown below and can be evaluated at the UGESCO website[2]. The crowdsourced rephotography is not yet fully supported, but will be integrated soon.
UGESCO dashboard for crowdsourced enrichment – Street View localization task.
Crowdsourced rephotography – Aligning a historical photograph in a modern rephotograph[3] (or vice versa) can serve as a remarkable visualization of the passage of time and can be used across a plethora of application domains (e.g., tourism, media and interactive museum applications). Crowdsourced and computational rephotography modules try to accurately map images of the past on the buildings and landscapes of today. Estimation of FOV can be found automatically by following a random sample consensus (RANSAC) homography matching. Another approach, focusing on outlier filtering and viewpoint clustering, is proposed by Makantasis et al. . However, in order to optimize and validate the results of these approaches, a crowdsourced rephotography module will be integrated in the dashboard. The figure below shows an example of how this is currently done. It is important to mention, however, that the geometric alignment is still rather basic and requires more research/development. However, the tool already allows to zoom, navigate around and change transparency of the collection picture, which continuously is dynamically placed at its correct Street View location. These crowdsourced dynamic image overlays can be used to improve the computational algorithms, i.e., they can learn from the crowd’s actions.
Rephotography tool to align a historical photograph with its Street View location.
[1] https://www.djangoproject.com/
[3] https://en.wikipedia.org/wiki/Richard_Prince