CRHT: A CONTINUOUS REGRESSION HYBRID TRANSFORMER FOR VESSEL TRAJECTORY PREDICTION WITH ONLINE CLUSTER SAMPLING

Alexander Schiøtz, Bertram Hage, Christian Rand, Felix Thomsen, Peder Heiselberg © 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Abstract

Accurate vessel trajectory prediction is critical for maritime safety and anomaly detection, yet existing models often struggle with geographic bias and navigational realism. We propose the Continuous Regression Hybrid Transformer (CRHT), a deep learning framework designed to forecast vessel motion using Automatic Identification System (AIS) data. To mitigate spatial data imbalance, we introduce an online K-means cluster sampling strategy that ensures diverse exposure to rare maneuvers during training. Our hybrid architecture integrates 1D convolutional layers for local kinematic feature extraction with a multi-head attention mechanism for global temporal context. CRHT demonstrates superior performance in short-term forecasting, achieving the lowest errors at the 1-hour horizon. The results demonstrate that while discrete models provide high navigational stability over long horizons, CRHT offers an optimal balance of precision and maneuver tracking for real-time maritime surveillance.

I Introduction

The Automatic Identification System (AIS) provides the spatio-temporal foundation for Maritime Domain Awareness (MDA). In critical waterways like the Danish Straits, accurate vessel trajectory prediction is vital for collision avoidance and anomaly detection. However, modeling AIS data is complicated by irregular sampling, sensor noise, and bias where models overfit to linear paths and fail during complex maneuvers.

Vessel trajectory prediction research has evolved from classical state-space models like the Kalman Filter [8], which struggle with nonlinear maneuvers, to deep learning architectures. Recurrent Neural Networks (RNNs) and LSTMs [15] improved temporal modeling but are limited by the information bottleneck of fixed-size context vectors. Recently, Transformer-based architectures [12, 14] have set new benchmarks by using self-attention to capture long-range dependencies. While discrete generative models like the TrAISformer [12] offer navigational realism through coordinate binning, they often suffer from quantization errors and high inference latency.

In this study we address the challenges of geographic data imbalance and regression precision by introducing the Continuous Regression Hybrid Transformer (CRHT). This work contributes a scalable MapReduce preprocessing pipeline, an online K-Means cluster sampling strategy to capture diverse maneuvers, and a hybrid CNN-Transformer architecture that integrates local kinematic extraction with global attention for robust trajectory forecasting.

II Data

The Danish Maritime Authority provides historic archives of vessel data from Denmark since 2006 [5]. 3 months of data in the period July 24th - October 24th 2025 was gathered. This dataset included 1,818,441,680 AIS messages from 35,349 unique vessels based on their unique Maritime Mobile Service Identity (MMSI) number.

In this study we focused on larger vessels, namely tankers and cargo ships and their basic geographical and kinetic features of such as latitude, longitude, Speed over Ground (SOG) in knots, and Course over Ground (COG) in degrees. Additional information was available, such as vessel type and navigational status. However, these features lack data completeness, especially for smaller vessels.

The dataset was transformed from irregularly sampled and noisy single messages into regularly sampled coherent vessel trajectories. First, we excluded erroneous messages outside our defined geographical boundary of $lat\in[54,59]$ , $lon\in[5,17]$ and removed messages not conforming to $SOG\in[0,30]$ and $COG\in[0,360]$ . We then split the data by MMSI and perform further processing in parallel [6], reducing computation time and space while avoiding lossy simplification such as the otherwise popular Ramer–Douglas–Peucker (RDP) algorithm [13]. Following [12], we defined a voyage as a sequence of messages with time gaps smaller than 2 hours, partitioned voyages longer than 20 hours, and discarded those shorter than 4 hours or containing fewer than 20 messages. Outliers where the calculated speed exceeded 40 knots were removed. Finally, we applied linear interpolation to resample the trajectories at 5-minute intervals. These steps reduced the dataset from 72,995 to 41,458 voyages spanning 1,203 unique vessels.

III Methodology

We frame vessel trajectory prediction task as a multivariate sequence-to-sequence regression problem. Given an observed sequence of historical states $X=\{x_{1},\dots,x_{T}\}$ where $x_{t}\in\mathbb{R}^{4}$ , we predict the future sequence $Y=\{y_{T+1},\dots,y_{T+L}\}$ based on the feature set $F=\{\text{Latitude, Longitude, SOG, COG}\}$ .

We developed a Continuous Regression Hybrid Transformer (CRHT) for vessel trajectory prediction. It performs continuous regression of coordinate displacements via three stages:

1. Local-Global Encoder: The input sequence $X$ is processed by a 1D Convolutional block ( $Conv1d\to ReLU\to Conv1d$ ) to extract local kinematic derivatives (velocity, acceleration) and smooth sensor noise. The result is augmented with sinusoidal positional encodings and passed to an $N$ -layer Transformer Encoder, which uses multi-head self-attention to capture long-range dependencies across the 64-step history.

2. Query-Based Decoder: Unlike recurrent architectures, we utilize a Transformer Decoder with $L$ learnable query embeddings $Q\in\mathbb{R}^{L\times d_{model}}$ . These queries interact with the encoder’s memory via multi-head cross-attention, allowing the model to attend to specific historical maneuvers (e.g. turn entries) to predict future steps $t+1\dots t+L$ in parallel.

3. Scaled Delta Learning: To ensure stability, inputs are MinMax normalized to $[0,1]$ , and the model regresses inter-step displacements $\Delta\hat{y}_{t}$ . Predictions are denormalized during the forward pass and optimized using Huber Loss against scaled ground-truth targets to prevent gradient underflow:

\mathcal{L}_{TPTrans}=\text{Huber}(\Delta\hat{y}_{pred},\Delta y_{true}\cdot\alpha)

(1)

where $\alpha=100.0$ is a scaling factor applied to physical degree differences.

Figure 1: Architecture of the CRHT model. The dual-stream design separates the historical context encoding (bottom blue stream) from the future trajectory decoding (top orange stream). The decoder utilizes Positional Encoding (PE) and learnable queries combined with cross-attention to the encoder’s memory to regress displacement deltas in parallel.

III-A Online K-Means Cluster Sampling

Transformer-based sequence models incur a quadratic memory cost $\mathcal{O}(N^{2})$ with respect to the input sequence length $N$ , which renders direct training on multi-hour vessel trajectories infeasible with limited resources. Consequently, long trajectories are truncated during training and fixed-length subsequences extracted to reduce memory cost. However, uniform random sampling introduces a strong bias toward geographically dense and low-speed regions, as commercial vessels repeatedly traverse established shipping lanes and spend extended periods maneuvering near ports. As a result, uniformly sampled subsequences overrepresent spatially redundant segments while underrepresenting rarer route deviations and sparsely trafficked areas.

To mitigate this imbalance, we propose an online cluster-based sampling strategy based on spatial K-Means clustering [10]. K-Means is a highly efficient clustering method with distributed implementations that makes it effective for huge AIS datasets [2]. A K-Means algorithm with $k=50$ clusters is fitted on the latitude–longitude coordinates of AIS messages in the dataset. Each AIS message is assigned a cluster index corresponding to its nearest centroid in Euclidean coordinate space (see Fig. 2).

Online (during training) sampling is performed by first selecting a trajectory, after which a cluster is drawn uniformly from the clusters overlapping that segment. A starting index is then sampled uniformly over part of the sequence within the selected cluster, subject to the constraint that a contiguous subsequence can be formed. This procedure enforces approximately uniform exposure across spatial clusters while preserving temporal continuity within each sampled subsequence (see Fig. 3(a) and 3(b)).

Refer to caption — Figure 2: Voronoi tessellation [7] of the coordinate space based on $k=50$ centroids (red dots). The blue lines represent the boundaries between clusters, ensuring that training samples are drawn from diverse geographical regions.

IV Experiments

As a non-learning baseline, we implemented a standard Kalman Filter assuming a Constant Velocity (CV) model based on the vessel’s last known heading and speed [8, 11].

We also implemented a CRHT using a Gated Recurrent Unit (GRU) decoder. However, it performed poorly compared to the Transformer decoder, which we attribute to the Information Bottleneck problem: the GRU is forced to compress the entire 64-step history into a single fixed-size context vector [1, 4]. Consequently, the Transformer Decoder was selected for the final model. Furthermore, we evaluated different model capacities during training. We found that a medium-sized architecture offered the optimal trade-off between performance and generalization.

For the final CRHT implementation, we set the latent dimension $d_{model}=512$ with 8 attention heads, 8 encoder layers, and 8 decoder layers.

IV-A Evaluation

The dataset was partitioned into training, validation, and test sets in an 80:10:10 ratio split by MMSI to avoid data leakage and prevent the model from overfitting to individual vessels’ movement patterns. We train using the AdamW optimizer [9] with a batch size of 512 and an initial learning rate of $3\times 10^{-4}$ on a single NVIDIA A100 (80GB) GPU [3]. Training runs for a maximum of 200 epochs with early stopping (patience 50), utilizing a 5-epoch warm-up followed by validation-based learning rate decay (factor 0.5) triggered by loss stagnation. During training we fix $T=64$ (past) and $L=12$ (future). All evaluated models benefitted from the online K-Means cluster sampling, providing a fair comparison of kinematic performance.

While [12] employs a ”Best-of-N” sampling strategy ( $N=16$ ), selecting the best prediction post-hoc, we reject this as unsuitable for real-world maritime surveillance (e.g., Real-time trajectory prediction, port prediction, anomaly detection, and collision avoidance), where the true future is unknown at inference time. Instead, we enforce strict deterministic evaluation to assess reliability: [12] uses greedy decoding ( $T=0$ ) to select the most probable bin; CRHT regresses the deterministic mean of displacement deltas; and the Kalman Filter projects linearly without noise.

To handle long horizons, we employ autoregressive generation: [12] predicts step-by-step ( $t\to t+1$ ), while CRHT operates block-autoregressively, forecasting full 1-hour horizons ( $H=12$ ) iteratively by appending predictions to the history window.

We evaluate performance in kilometers (km) using the Haversine distance function $d_{hav}(\cdot)$ . We report the Average Displacement Error (ADE), measuring the mean path deviation, and the Final Displacement Error (FDE), assessing destination accuracy at step $T+L$ :

ADE=\frac{1}{L}\sum_{t=1}^{L}d_{hav}(\hat{y}_{T+t},y_{T+t})

(2)

FDE is the geospatial distance between the predicted and ground truth positions at the final time step ( $T+L$ ). This metric evaluates the model’s destination forecasting accuracy.

FDE=d_{haversine}(\hat{y}_{T+L},y_{T+L})

(3)

The mean trip length was $4.17$ hours.

TABLE I: Mean FDE and ADE prediction errors in kilometers (km). Bold indicates the best performance in each category.

	1 Hour Horizon		2 Hour Horizon
Model	ADE	FDE	ADE	FDE
Kalman Filter	$1.58$	$3.60$	$3.70$	$8.86$
TrAISformer	$1.38$	$2.41$	$2.62$	5.42
CRHT	1.21	2.07	2.55	$5.85$

V Results & Discussion

Table I reports trajectory prediction ADE and FDE across 1-hour and 2-hour horizons. The Continuous Regression Hybrid Transformer (CRHT) consistently achieves the strongest overall performance profile across both horizons.

At a 1-hour prediction horizon, CRHT achieves the lowest mean ADE (1.21 km) and the lowest mean FDE (2.07 km) among all evaluated models. At the 2-hour horizon, CRHT maintains the lowest mean ADE (2.55 km). For FDE, CRHT achieves a mean error of 5.85 km, which is slightly higher than TrAISformer (5.42 km). Across horizons, CRHT demonstrates a consistent reduction in displacement errors relative to classical filtering and discrete generative baselines.

Fig. 4 showcases a difficult prediction scenario in which a vessel, following a long and nearly linear trajectory south of Sweden, executes a pronounced northward turn around Bornholm. In this setting, CRHT predicts a trajectory that follows the northward maneuver and remains feasible over the full forecast horizon, closely matching the true vessel path. By contrast, both the Kalman Filter and TrAISformer extrapolate the preceding motion pattern and project the trajectory into the island of Bornholm, reflecting an inability to represent the observed change in heading in this scenario.

In Fig. 5, CRHT produces a trajectory that largely aligns with the ground truth, with only a small spatial deviation. The Kalman Filter continues along a straight-line path and achieves the lowest FDE in this instance, but its trajectory does not capture the executed maneuver and deviates from the actual vessel path. This example illustrates that low point-wise error metrics may coincide with qualitatively inconsistent motion representations.

Fig. 6 depicts a highly ambiguous scenario in which none of the evaluated models accurately predict the realized vessel trajectory. CRHT and the Kalman Filter diverge early from the ground truth, while TrAISformer produces a trajectory that would be realistic if the vessel had taken an eastward turn instead of the executed westward maneuver, effectively ”hallucinating” an alternative path. This example illustrates the challenge of modeling trajectories under high uncertainty, where multiple plausible routes exist and a model may generate a navigationally valid path that does not correspond to the observed behavior.

VI Conclusion

We introduced and evaluated the Continuous Regression Hybrid Transformer (CRHT) for AIS-based vessel trajectory prediction. CRHT achieves the lowest mean displacement errors across short- and intermediate-term horizons while producing smooth, continuous trajectories that handle nonlinear maneuvers. The results demonstrate its ability to follow complex paths realistically, in contrast to classical filtering and discrete generative models that either fail on sharp turns or generate plausible but unexecuted trajectories. By combining predictive accuracy with navigational feasibility, CRHT provides a robust framework for maritime trajectory forecasting under both typical and uncertain conditions.

All code for this study is available here

References

[1] D. Bahdanau, K. Cho, and Y. Bengio (2015) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. Cited by: §IV.
[2] M. F. Balcan, S. Ehrlich, and Y. Liang (2013) Distributed $k$ -means and $k$ -median clustering on general topologies. Advances in neural information processing systems 26. Cited by: §III-A.
[3] D. C. Center (2025) DTU computing center resources. Technical University of Denmark. Note: https://doi.org/10.48714/DTU.HPC.0001 External Links: Document Cited by: §IV-A.
[4] K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio (2014) Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §IV.
[5] Danish Maritime Authority (DMA) (2025) AIS data. Note: https://www.dma.dk/safety-at-sea/navigational-information/ais-dataAccessed: November 21, 2025 Cited by: §II.
[6] J. Dean and S. Ghemawat (2008) MapReduce: simplified data processing on large clusters. Communications of the ACM 51 (1). Cited by: §II.
[7] Q. Du, V. Faber, and M. Gunzburger (1999) Centroidal voronoi tessellations: applications and algorithms. SIAM review 41 (4), pp. 637–676. Cited by: Figure 2, Figure 2.
[8] R. E. Kalman (1960) A new approach to linear filtering and prediction problems. Transactions of the ASME–Journal of Basic Engineering. Cited by: §I, §IV.
[9] I. Loshchilov and F. Hutter (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §IV-A.
[10] J. MacQueen (1967) Multivariate observations. In Proceedings ofthe 5th Berkeley Symposium on Mathematical Statisticsand Probability, Vol. 1, pp. 281–297. Cited by: §III-A.
[11] A. Mansour, V. Groza, and E. Petriu (2020) The use of kalman filter techniques for ship track estimation. WSEAS TRANSACTIONS ON SYSTEMS. External Links: Document Cited by: §IV.
[12] D. Nguyen and R. Fablet (2024) TrAISformer: a transformer network with sparse augmented data representation. IEEE Transactions on Intelligent Transportation Systems. Cited by: §I, §II, §IV-A, §IV-A.
[13] U. Ramer (1972) An iterative procedure for the polygonal approximation of plane curves. Computer graphics and image processing 1 (3). Cited by: §II.
[14] W. Wang, W. Xiong, et al. (2024) TPTrans: vessel trajectory prediction model based on transformer using ais data. ISPRS International Journal of Geo-Information. Cited by: §I.
[15] X. Zhang, X. Fu, Z. Xiao, H. Xu, and Z. Qin (2022) Vessel trajectory prediction in maritime transportation: current approaches and beyond. IEEE Transactions on Intelligent Transportation Systems 23 (11). Note: 10.1109/TITS.2022.3192574 Cited by: §I.