# Background removal: the rembg sidecar

Setup notes for the Python image-processing stack behind the review queue's background-removal option.

**Status: evaluated, not adopted.** The engine choice is still open — see the "Alternatives ruled out" section. Everything below was run and measured on 2026-09-02 against `danielgatis/rembg:latest`, on a Windows dev box under Docker Desktop. **The NAS is a different machine and will be slower**; treat these as an upper bound on capability, not a promise.

## Why a sidecar and not the host's Python

The application runs in a container. Python installed on the NAS host is not reachable from inside it, and putting Python and ONNX into the application image would add roughly 300 MB to an image that is already 1.06 GB. A sidecar keeps that weight out of the deployable and lets the two be restarted independently.

## The one thing that must not be got wrong

**Always send `model=u2net` explicitly. Never rely on the default.**

The default model this image downloads is `bria-rmbg`. BRIA's RMBG models are licensed for **non-commercial use**; this is a shop. That is the same trap that ruled out `@imgly/background-removal-node`, and it is easy to walk into because it happens silently on the first request.

Specifying `u2net` is not only the licensing answer — it is better on every measured axis:

| | `bria-rmbg` (default) | `u2net` (Apache-2.0) |
|---|---|---|
| Licence | **Non-commercial** | **Apache-2.0** |
| Warm request, 2000×1500 | 14–20 s | **1.1–2.3 s** |
| Model on disk | 977 MB | 168 MB |

A ten-times speed difference decides the interaction design on its own: at 1–2 s a click can wait, at 15–20 s it cannot.

## Running it

```bash
docker volume create rembg-models

docker run -d --name rembg \
  -p 7000:7000 \
  -v rembg-models:/root/.rembg \
  danielgatis/rembg:latest s --no-ui -p 7000
```

- `--no-ui` disables the bundled Gradio interface. It is not needed for API use and the image's own help notes it reduces idle CPU — which matters on a box also running Gitea, QA and production.
- The volume is on `/root/.rembg`, **not** `/root/.u2net`. The older path appears in much of the documentation online and is wrong for this image: models land in `/root/.rembg/models/<name>/`. Mounting the wrong path silently re-downloads 168 MB on every container start.
- Startup takes about **40 seconds** before the port answers. Anything health-checking it needs to allow for that.

Add to a Portainer stack as a second service on the same network, and have the application address it by service name rather than by host port.

## The API

```
POST /api/remove
  file=@photo.jpg      (multipart)
  model=u2net          (REQUIRED — see above)
```

Returns a PNG. Verified on a 2000×1500 input:

- `format: png`, `channels: 4`, `hasAlpha: true`
- dimensions preserved exactly
- alpha spans `0`–`255`, so it is a genuine cut-out rather than a flattened image with an alpha channel bolted on
- output was **1.7×** the input JPEG's bytes

The first call for a given model downloads it (about 17 s for `u2net`), so the first request after a fresh volume is slow. Pre-warm it rather than letting a person meet that delay:

```bash
curl -s -F "file=@any.jpg" -F "model=u2net" -o /dev/null http://localhost:7000/api/remove
```

`rembg d` downloads models without serving, if a build step is preferred.

## Costs

| | |
|---|---|
| Image | **4.24 GB** |
| `u2net` model in the volume | 168 MB |
| Application image, unchanged | 1.06 GB |

4.24 GB is the honest number and the main argument against this route. It buys no per-image cost, no external service, no credential, and no data leaving the NAS.

## What has not been established

- **Quality on a real photograph.** Everything above used a generated rectangle on a flat ground, which is trivially easy. The timings are sound because they depend on pixel count, but nothing here says how `u2net` handles a chipped vase on a patterned rug, and that is what consignors will send.
- **Behaviour on the NAS.** Slower, by an unknown factor.
- **Concurrency.** `--threads` exists and was not exercised. One request at a time is the safe assumption.

## Alternatives ruled out

**`@imgly/background-removal-node`** — AGPL, with commercial terms on request. Also pins `sharp ~0.32` against the project's `^0.35`, and a second sharp in the tree has caused a silent breakage here before.

**`rembg-node`** (the npm package, not this image) — MIT, and pulls no sharp, which looked ideal. But it downloads its model from a **Google Drive link**, which failed outright when tested, and its `modelPath` is readonly so a pre-baked model cannot be supplied through the API.

**A hosted API** (remove.bg, Photoroom) — clear terms, no disk cost, best quality. Roughly $0.20 an image, another credential, another external dependency in the pipeline, and would want the budget ceiling that #227 gave submissions.

## If this is adopted

The application would `POST` to the sidecar rather than doing any inference itself, which keeps every Python and ONNX dependency out of the Node image. Given 1–2 s warm, a synchronous request from the review queue is reasonable — unlike drafting, which was deliberately moved off the request path because a stranger can trigger it and must never wait.

The four design decisions already taken are independent of the engine: the cut-out replaces the item's image while the original is kept and restorable, the result is a transparent PNG, and the control sits on each photo in the review queue.