Student: “Nektarios I. Christou”

Program: “Data Science and Information Technologies”

Title: “A Spatial Attention Mechanism for Motion Prediction in Autonomous
Driving”

Abstract:
Accurate and efficient motion forecasting is essential for safe autonomous
driving, where an ego vehicle must anticipate the future trajectories of
surrounding agents in complex urban environments. Recent transformer based
architectures such as HPTR provide a strong and lightweight backbone for
trajectory prediction by jointly encoding map and agent information. In
this thesis we build directly on HPTR and study how to augment its
attention mechanism with an explicit spatial density prior. Our main
contribution is a Spatial Density Module (SDM) that models the interaction
space with a Gaussian Mixture Model and uses its normalized density as a
soft, spatially aware filter on attention maps. This gives a Spatial
Density Transformer (SDT) variant that keeps HPTR’s encoder-decoder
design, number of parameters, and training setup, and changes only the way
attention is distributed over possible interaction regions. We conduct
experiments on the Waymo Open Motion Dataset and focus the analysis on the
behavior of the Spatial Density Module. In an ablation study over the
number of mixture components and SDM hyperparameters, and using simple
diagnostic metrics for the mixtures (component utilization, spatial
variance, and entropy), we
observe that the learned density tends to place mass in regions where
there are agents and map elements instead of empty areas. This suggests
that the SDM does learn a meaningful notion of interaction density. In
terms of standard forecasting metrics on WOMD, SDT reaches reasonable
performance, but we do not carry out a full head to head comparison with
HPTR under identical conditions, so we cannot draw firm conclusions about
relative gains or losses. The main contribution of this work is therefore
the design and analysis of the density aware attention mechanism and its
diagnostics, rather than a clear improvement in benchmark scores. The
experiments indicate that adding a learned, density based attention filter
on top of an existing lightweight transformer is a viable direction that
could be explored further in future work, including variants where the
mixtures are constrained to behave like k nearest neighbor neighborhoods
as in HPTR to better understand how useful density aware attention is in
practice.”

Date/Time: March 2, 2026 – 4:00 PM.

Examination Comitee:
”
Assoc. Prof.  Aggelos Pikrakis
Prof. Dimitrios Gunopulos
Dr. Konstantinos Koutroumbas
“

Presentation link:
https://unipi.webex.com/unipi/j.php?MTID=m8f41d115c98f3ec6aa42bcfedafc0041