Real-time segmentation, from training to the car

Segmentation sat under most of the perception stack I worked on at ESR Labs. Localisation matched its curbs to a map; panoptic and depth networks reused its encoder. It also had the hardest constraint: it had to run in real time on the car’s embedded GPU, next to everything else. I worked on it from late 2017 to the last deployment in 2021, first as an engineer and, from 2020, as perception lead.

Make networks cheap to compare (2017)

My first project was a config-driven TensorFlow framework. Colleagues had written the core; I set up the repo and added the tasks, losses and export around it. A new network is a small plugin class plus a JSON file, so every architecture trains on the same data, loss and logging, and inference time and accuracy compare directly.

Driving scenes are lopsided: road and sky cover most pixels. I added an ignore label for the unlabelled area and weighted every pixel by the inverse frequency of its class in the batch, so signs, poles and pedestrians count.

Then a network of my own, designed for speed:

• inception-style branches with 3, 7 and 11 kernels, then dilated ones;
• every convolution spatially separable (k×1, then 1×k) to cut multiply-adds;
• pyramid pooling for global context;
• a dense-upsampling decoder (pixel shuffle) instead of transposed convolutions.

A shrunk micro version, LinkNet and MobileNet ran in the same harness as baselines. In December 2017 the framework got a one-step TensorRT export (freeze graph, UFF, engine in FP32, FP16 or INT8), so any model could be measured on the target GPU from one config file.

One label space for many datasets

Public and in-house data never agree on classes. I mapped Mapillary’s 66 classes to 27 and later to 16, the OEM’s 44-class labels to the same 14–16, and split the curb out as its own class in June 2018 because localisation needed it. A shared 16-class space let public and in-house data train one model.

Two reading bugs taught me to check label types first: Cityscapes disparity PNGs are 16-bit, and reading them as 8-bit silently corrupted the ground truth. Mapillary labels read as uint8 destroyed the class ids.

Real-time candidates (2018–2019)

I kept adding candidates to the harness: ShuffleSeg, Tiramisu, a rewrite of a colleague’s BiSeNet prototype, and in August 2019 Fast-SCNN, with an extra fusion stage for the wide 1920×750 camera input. Lovász-Softmax went only on the thin classes.

For post-processing I tuned a dense CRF per class with 200 TPE trials each. Thin classes like signs and lane markings ended up with small spatial kernels; buildings and cars with large ones. One setting can’t serve both.

SinReQ: train the weights onto the grid (2020)

INT8 maps every weight to one of a few levels. Weights trained in FP32 land anywhere, so rounding them costs accuracy, and calibration has to work hard to find scales that hurt least.

In May 2020 I added SinReQ (Elthakeb et al., 2019) to our training code. It is a sinusoidal regulariser, Σ sin²(π(w + shift)/step), which is zero exactly on the quantisation levels and pulls each weight towards its nearest one while the network trains.

• Target: 8 bits (step 1/(2⁸ − 0.5)), the INT8 grid, strength 1e-6, batch-norm variables excluded.
• An oscillating variant: the strength is scaled by sin²(t/20) over training step t, so the term relaxes periodically. If it dominates all the time, weights freeze in their nearest bin; relaxing it lets them still move to a better one.

The cost in accuracy was negligible: 0.5827 mIoU against 0.5828 in FP32. The gain came at deployment. With the weights already close to their bins, INT8 calibration became an easy step instead of a search, and it took far less effort to reach full accuracy.

Train for the precision you deploy.

INT8 without losing accuracy (2020)

I wrote the INT8 module of our compression tool: frozen TensorFlow graph to ONNX to a TensorRT engine, an entropy calibrator on 100 random images, a latency benchmark, and an mIoU comparison between TensorFlow and TensorRT outputs. It also had an op-by-op tester that converts each op type on its own and diffs the outputs, to find which ops break.

Calibration is what closes the gap; SinReQ is what makes calibration cheap.

PyTorch, and the car (2020–2021)

In July 2020 segmentation was the first project in our new PyTorch perception repo. Its base model, config loader and training loop became the template for the SOLO, panoptic, depth and keypoint projects that followed. I made 244 of the repo’s 519 commits. The best Mapillary config, Fast-SCNN on 16 classes, reached 59 mIoU.

The network that went into the car:

• a ResNet-18 trunk, 19 A2D2 classes, 14.9 M parameters;
• 192×640 input, cropped from the road band of the 1208×1920 frame;
• zoom-in augmentation (random crops of 60–95% of the frame), so object size at that low resolution doesn’t surprise it;
• softmax and a uint8 class map inside the exported graph, so the runtime gets classes, not logits.

TensorRT 8 failed on upsampling with a dynamic scale factor, so the decoder uses an explicit output size. The segmentation engines for Drive AGX Pegasus came to 30.6 MB in FP16 and 18.8–21.8 MB in INT8, depending on input size.

One engine, every framework

The C++ side matters as much as the network. Our in-house middleware already had an inference engine with TensorFlow, LibTorch and TensorRT backends behind one interface. My part in 2021:

• Device-aware tensors. Each tensor knows its device; the engine copies it only when it isn’t already on the inference GPU.
• A hybrid backend. TensorRT for the part it can build, TorchScript for the rest. It is what made panoptic segmentation deployable.
• One detector component. I merged the instance and free-space detectors into one segmentation base class with semantic, panoptic and multi-view variants (about 1,400 lines removed).
• Runtimes for the car. PyTorch 1.8.2 and TensorRT 8.0.1 for x86 and arm64, including an aarch64 PyTorch wheel I built for the target.
• Demo deployments with default configs, so anyone on the team could run a network on the target hardware.

A new network should be a config change, not a C++ change.

Later: INT8 inside training (2023)

One more experiment: an L1 penalty on each weight’s distance to the nearest INT8 level. It sits between post-training quantisation and full quantisation-aware training.

What stayed with me: measure on the target, keep one label space, and calibrate before redesigning.