First Mile / Last Mile

posted on 17 Mar 2019 by chek

The use of VR and AI technologies to create tools to help understand commuting experiences.

Brief

Evaluating commuter experiences using traditional methods of surveys, observations and interviews is laborious and costly. Moreover, when a researcher needs to collect real-world data in-the-wild and stops a busy commuter for a field survey for example, it may be disruptive and induce unwanted reactions that may confound the data collected.

The overarching aim of this project is to improve commuter experiences by investigating how aspects of VR and AI can help in building more conducive digital evaluation tools to understanding these commuting experiences better.

The project consists of 3 major parts - DataProc, VRCS and CRMA.

(We apologize that much of the data in the actual research artifacts cannot be shown here, hence the content in the images/videos shown in this article may not accurately reflect actual research outputs.)

DataProc

The Data Processing (DataProc) tool is a research artifact in the form of a graphical tool that investigates how to best process LiDAR-based terrestrial scans and photogrammetry-based aerial scans that captures real-life streetscape data to be translated for use in VR and mobile environments for VRCS and CRMA respectively.

There are many research problems that we work on the “translation” process, which includes 3D semantic segmentation for 3D scene understanding and usability design in our custom tool interface to manipulate these massive point clouds (file size of ~8GB per 200m² tile).

The video below demonstrates some features of the DataProc tool we created to perform some exploration and annotation on a sample 3D point cloud captured with our LiDAR terrestrial camera.

The processed data will be translated into high fidelity 3D assets for use in VRCS, e.g., in the sample images below (these are redacted synthetic environments made from the point clouds and not directly modeled after the real test site), and 2D topographical data for use in the CRMA.

VRCS

The Virtual Reality Commuting Simulator (VRCS) is a research artifact that helps us understand how to design a VR tool for evaluating commuter experiences. This addresses the pain points of data quality and costs when conducting physical surveys and observations in the field. A VR solution is also a much safer and detailed controlled environment for obtaining more nuanced responses from commuters.

Key research problems here include interaction design for infinite walking in VR, VR UI design as well as procedural generation of VR assets from the point cloud data from DataProc.

The video below shows the interaction modality for walking, in our pursuit of enabling a usable solution for infinite walking.

The other video below shows a collision guidance system to resolve inevitable situations where users will physically walk into virtual obstacles.

CRMA

The Commuter Routing Mobile App (CRMA) is a research artifact that helps us understand whether a highly localized (e.g., including HDB void decks for routing) routing app with personalized routing features (e.g., barrier-free routes that consider weather conditions) can provide novel propositions for Singaporean commuters.

Several of the research problems we are investigating includes the development of appropriate machine learning architectures for personalized routing, and whether such an app can entice commuters to provide meaningful data for transport planners to use.

The video below is a student-made prototype that demonstrates personalized routing which takes into account the user demographics and route preferences to generate routes that factor in localized features like slopes.

The video below is a another student-made prototype that now demonstrates the idea of using game elements to encourage commuters to explore their FMLM routes, and in turn provide more data points for transport planners to analyse commuting experiences.

Members

SIT

LTA

Aviation Virtual Pte Ltd

Outputs

Provided real-world project problem statements and industry interactions for the following SIT undergraduate student modules

Here are what some students have said about their learning experiences.

Acknowledgements

This work is funded by an industry grant from the Land Transport Authority of Singapore.