Auto & Moto 7 min read

Hiding number plates with AI on auto.moto.pt

From the product owner's go-ahead to production in 33 hours: our own AI now hides number plates in auto.moto.pt photos.

Tender Touch team
Portugal
A pixelated number plate inside a green detection frame on a dark background.

In early October the product owner of our own marketplace, auto.moto.pt(opens in a new tab), asked for something simple to state: let the advert form hide the number plate in sellers' photos. This is how we went from that request to production, with our own model, and what the first days of real use show.

The problem

A used car advert needs good photos, and a good photo of a car from the front or the back shows the plate. Many sellers do not want it public: of the 1,450 photos we later labelled, 237 already had a plate the seller had hidden, with a sticker before the picture or, far more often, in a photo editor. Many others forget. Photos also catch the plates of cars parked nearby, owned by people who never agreed to appear in an advert.

Our first answer was a manual tool: on each photo in the form, a button opens an editor where the seller marks the plate and it is covered with large square blocks. It works, but it is one more step per photo, and the product owner was clear that the goal was fewer clicks, not more. The real question became: can the form find the plate by itself?

Why we did not use a ready-made model

We first tried an open plate detector, locally, on 30 cover photos from the home page. It found the plate on 17 of the 18 photos where one was visible. It also found "plates" on dashboards and door handles, and it treated dealer signs as plates. That rules out covering automatically: a seller does not want a mosaic on the speedometer. There was a second problem: the licence of the model weights was not stated.

So we chose YOLOX-tiny, a small open-source detector whose code and weights are under the Apache-2.0 licence, and trained it on photos from our own adverts. The photos and the model never left our infrastructure.

Building the dataset

Labelling started with the detector proposing boxes and every photo being checked one by one: move a box, delete a box on a dealer sign, add a missed plate. The first set had 512 photos from 195 adverts. Between them they held 154 real plates, in 132 photos. The rest were there on purpose: 141 interiors and 82 photos with dealer signs, so the model learns what not to cover.

About 25 photos did not fit a simple rule, and each group was decided with the product owner: decorative or spare plates, brand placeholders, plates the seller had already hidden, plates seen edge-on, and tiny unreadable plates far in the background. The rule became: cover real, readable plates, including other people's cars; leave alone anything that is already hidden or carries nothing to hide.

Training and validation

The first model was tested on 50 photos it had never seen. It found the main car's plate on 10 of 11, the same as the ready-made model, but with half the false boxes (0.10 per photo against 0.20) and only 2 false covers out of 8 dealer-sign photos, against 8 of 8. On one CPU core it takes 0.15 seconds per photo, and the model file is 20 MB.

We were also asked whether classical OpenCV would do the job without a neural network. We measured it on the same 50 photos, with the settings tuned on the training set: searching for plate-shaped rectangles found 3 of 11 main plates. A real plate is tilted, reflective, framed or partly hidden, and only 8 of the 15 test plates even produced a clean rectangle.

The weak point of the first model was small plates in the background: 1 of 4. The fix was more data, not more cleverness. We added 938 photos from 573 adverts, each one reviewed, which took the number of training photos with real plates from 109 to 409. Training ran overnight on the ordinary processors of our own server and took 3 hours 15 minutes. To keep the numbers honest we built a second test set of 86 photos from other sellers, with 56 plates, and kept the old 50 photos unchanged for a like-for-like comparison.

Bar chart. Background plates: 4 of 19 with the first model, 14 of 19 with the current one. Main car plate: 33 of 37, then 35 of 37. Motorcycle plates: 17 of 19 with the current model.
The first model and the current one, on test photos neither had seen. Motorcycles were measured only for the current one.

On the new test set the main car's plate went from 33 of 37 to 35 of 37. We chose a lower sensitivity setting, which kept those results. The price is a little more noise than the first model: 0.07 false boxes per photo, about one in fourteen photos, none of them on a readable plate. It also still covers 8 of the 11 plates that sellers had already hidden, which the third round of training was meant to fix. An independent second check re-ran the whole evaluation with its own script, and the numbers matched.

Covering the plate, not the car

The first version on the test site drew a rectangle aligned with the image around each plate and added a margin. On a car photographed at an angle, the plate is a slanted strip and a box aligned with the image around it is mostly bodywork, so the mosaic looked much bigger than the plate. The fix uses OpenCV after all: inside the detector's box, a classical edge search finds the plate's own outline, and only that shape is covered. When the outline looks doubtful, the whole box is covered, as before.

Rear of a red Alfa Romeo Spider, twice: on the left a wide mosaic that covers the plate and part of the lights, on the right a tighter mosaic over the plate only.
Left: the first version, a box with a margin up to the lights. Right: the current one, close to the plate. Photo (edited): Bryn Pinzgauer(opens in a new tab), CC BY 2.0(opens in a new tab).

The blocks are large, and their size is calculated from the plate itself. Before release, an independent second check ran 131 photos from recent adverts through the test site: front, rear, motorcycles, dusk. No letter of a plate was readable on any of them, and photos without a plate came back unchanged.

How it fits in the advert form

In the photo step there is one tick: "Ocultar a matrícula com inteligência artificial" ("Hide the number plate with AI"). It is off by default, a decision of the product owner. When it is on, every uploaded photo is checked by a small detection service inside our own cluster, and the plate is covered before the photo is stored, so for new uploads the uncovered original never reaches the disk. For photos uploaded earlier, the edit form has the same tick: a covered copy replaces each photo in the advert.

Nothing in this path may block an upload. If the detector is busy, slow or down, the photo is saved as it came, and the form says so and points to the manual button on that photo.

Two photos side by side: a black motorcycle in front of a parked car whose plate is covered by a mosaic, and a yellow sports motorcycle whose square rear plate is covered.
Current version: a car behind a motorcycle; a motorcycle plate. Photos (edited): Alexkom000(opens in a new tab), CC BY 4.0(opens in a new tab); Belarmino Ribeiro(opens in a new tab), CC BY 3.0(opens in a new tab).

Results in numbers

From the owner's go-ahead to train our own model to the feature in production took about 33 hours: the first model was ready the same afternoon, the improved one overnight, and the site went live on the night of 7 October.

In the first days, up to 10 October, sellers published about 80 new adverts through the form (feed imports and staff accounts excluded). Eighteen of them, about one in five, have at least one photo covered by the AI. That is a lower bound: an advert where the tick was on but no plate was found leaves no trace. Six older adverts also had it switched on in the edit form. In total, 73 photos in 24 adverts from 21 sellers were covered automatically.

What we learned

Data beats tuning. The biggest gain, on background plates, came from labelling more real plates, not from changing the model.

Negative examples are half of the work. Interiors and dealer signs are what made automatic covering acceptable at all.

A newer model is not always better. A third round of training, which treated already hidden plates as non-targets and added motorcycles, made fewer false boxes but found 46 of the 56 test plates, against 49 for the model in production. It stays out until it is better on what matters most.

Small plates in the background are still the weakest case, at 14 of 19. Square motorcycle plates do better, and the production model finds 17 of 19 on a separate set of motorcycle photos. The same tick is already built into the mobile app, which is still in testing.

If your product handles photos, documents or other data that needs care, this is the kind of work we do for clients in custom software development.

Working on something similar?
Tell us what you are building.
Talk to us

More from the journal

See all