Milestone Project Hafnia is proud to present Track 6 of the AI City challenge!
Challenge Track 6 focuses on fine-grained object detection in real-world traffic imagery under geographic domain shift. Participants are asked to train models on data from one city and evaluate them on hidden benchmarking data that includes a distinct target city with different visual characteristics, scene layouts, viewpoints, and environmental conditions. The track is designed to study cross-city generalization, a setting that remains underexplored due to the difficulty of obtaining large-scale real-world data from multiple locations. To support this effort, the track is powered by Milestone Project Hafnia, which provides access to anonymized video data together with managed training infrastructure. The data used in this track are anonymized, and the full training and benchmarking datasets are kept hidden from participants and accessed through Hafnia platform workflows, enabling privacy-conscious and compliance-aware experimentation on real-world camera data.
All accepted participants get access to Hafnia Training-aaS for Track 6, which is not yet available to the public.
Data
This track is based on a subset of the large-scale real-world dataset curated through Milestone Project Hafnia. The training split, including train and validation data (80-20), contains approximately 13k frames and 150k annotated object instances, with a comparable hidden benchmarking set. The benchmarking set includes both source-city data and data from a distinct target city. The data are extracted from real-world traffic video streams and include diverse viewpoints, roadway types, camera perspectives, and imaging conditions. Images are primarily provided at high resolution, with visual characteristics representative of real-world static traffic cameras.
The benchmark includes 10 object classes: See on the official website.
Annotations are provided as axis-aligned 2D bounding boxes. Objects are annotated as long as they are at least slightly visible, including partially occluded or truncated objects. The minimum box size considered in the benchmark is 10 × 10 pixels. To reduce ambiguity caused by tiny distant objects that are not part of the benchmark, some ignored regions are blurred. Frames are sampled from video with a minimum spacing of 2 seconds, increasing visual diversity and reducing redundancy.
Detailed dataset statistics, metadata, annotation information, a sample dataset, and dataset-format documentation will be available through the Hafnia platform. The full training data will be accessible only through managed Hafnia training jobs, and the hidden benchmarking data will remain unavailable for direct download or inspection.
Task
Given an input image, participants must detect all target objects and assign each detection one of the predefined fine-grained classes.
The task is formulated as a single-image detection problem. No temporal cues are allowed at inference time. Although Hafnia datasets may include additional metadata in other contexts, tracking information and temporal information will not be available for the challenge dataset. The primary challenge is not only achieving strong detection accuracy, but doing so under cross-city domain shift, where the model must generalize from the source-city training distribution to hidden benchmarking data that includes a distinct target-city distribution.
Participants may use pretrained models. External datasets are not allowed for training within the Track 6 challenge workflow, but participants may bring models pretrained outside the challenge. Ensembles are permitted as long as they can be executed as a single inference pipeline within the platform constraints. Very large models may be limited by upload size, model size, runtime, memory, and available compute resources, as specified in the platform documentation.
Submission Format
Participants will run inference through the Milestone Project Hafnia platform. Instead of manually preparing challenge submissions from raw test data, teams will upload:
- Trained model weights
- Inference source code
- Docker file describing the inference environment
- Any required configuration or runtime parameters.
Benchmarking will run in two steps. First, the Hafnia platform will execute inference on the hidden benchmarking data and produce prediction files in the format required by the official AICity evaluation system. Participants will then download the generated prediction files and manually submit them to the AICity challenge page, where evaluation against the hidden ground truth and ranking will take place.
Automatic transfer from Hafnia to the AICity submission system will not be part of the Track 6 workflow.
The training and inference package formats are designed to be similar, so that participants can reuse most of their code structure. Starter templates, including a quickstart guide and an example trainer package for training detectors, will be provided through the Hafnia platform documentation.
Evaluation
The primary evaluation will be based on standard object detection metrics, including mean Average Precision (mAP), with exact evaluation details to be released when the benchmarking functionality becomes available.
The ranking will be based on performance on the hidden benchmarking set, which includes both source-city and target-city data, with emphasis on robust performance under cross-city domain shift. A public leaderboard is expected to be available during the challenge. For final ranking, inference time and model size may be used as tie-breaking criteria.
Further metric details, including IoU thresholds, averaging conventions, class-level performance, city-level performance, and the final aggregate score, will be published in the official evaluation protocol.
Data Access
This track is hosted through Milestone Project Hafnia Training-as-a-Service. Participants will not directly access, download, or inspect the full training or hidden benchmarking data. Instead, the challenge is organized around a managed workflow that supports compliant experimentation on real-world data.
The expected high-level participation flow is:
- Sign up to the challenge on this page.
- Wait for account approval and challenge access to be granted by the organizers.
- Explore the Hafnia platform and the documentation.
- Download the sample dataset and starter materials.
- Review the quickstart guide and example trainer package provided through Hafnia.
- Adapt your training pipeline to the Hafnia dataset format.
- Build a Docker file for the training environment.
- Upload the training job, including code, model definition, and Docker environment.
- Monitor your training jobs on the Hafnia’s experiments tracker.
- Retrieve the trained model artifacts after training finishes.
- Upload the inference job with the trained model, inference code, and Docker environment.
- Run inference on the hidden benchmarking data through Hafnia.
- Download the generated prediction files.
- Submit the prediction files to the AICity evaluation page, where evaluation against the hidden ground truth and ranking will take place.
Platform Resources and Constrains
Participants will be able to use the downloadable sample dataset locally for:
- Understanding the dataset structure
- Validating data loading pipelines
- Testing training and inference code
- Checking Docker compatibility
The full training data will only be accessible through managed Hafnia training jobs. The platform will provide:
- Sample data
- Dataset documentation and statistics
- A quickstart guide and example trainer package for training detectors
- Starter templates for training and dataset configurations
- Training logs and experiment monitoring
- Export of trained model weights
Resource access and usage will be constrained through platform limits, including:
- There will be a limit of maximum 5 members per organisation.
- One experiment running at a time per participant account
- A 30,000-credit allocation per accepted participant account
- A 2 GB upload limit for training materials, including trainer files, model files, Docker-related files, and associated materials
- Model size, runtime, memory, and available compute limitations
- GPU availability constraints, especially for higher-tier GPUs
Participants may choose among the available GPU tiers while credits remain available. Higher-tier GPUs will consume credits faster and may have lower availability, so jobs using larger GPUs may take longer to start.
These constraints are intended to ensure fair access to shared infrastructure and will be described in more detail in the Hafnia platform documentation.
Privacy and Compliance
A key motivation behind Milestone Project Hafnia is the creation of a large-scale legally sourced computer vision data library for static real-world cameras. This track showcases that capability through a cross-city benchmark built on hidden real-world traffic data.
The data used in this challenge are anonymized. Participants do not access the raw full-resolution training or hidden benchmarking corpus directly. Instead, training and benchmarking are performed through managed Hafnia platform workflows that prevent direct dataset extraction.
This hidden-data workflow supports privacy-conscious benchmarking and compliance-aware experimentation on real-world traffic imagery while preserving the utility of the visual data for computer vision research.
Important dates
More information on submission and so on will be released soon.
About Hafnia
References
[1] Milestone Systems, “Project Hafnia: A Game-Changer in AI Model Training”, 2025.
[2] brighter AI, “Privacy v Progress: How DNAT Protects Privacy in the Age of Machine Learning,” 2022.
[3] Milestone Systems Hafnia Python SDK / CLI documentation – GitHub