Skip to content

tams-clip-ingester & transcode watcher

Not everything in the store arrives from a camera. These bring in a file that already exists, and derive new versions of material that is already there.

TAMS — components tams-recorder tams-player tams-flow-manager tams-flow-visualiser tams-serverREST APIMySQL meta · S3 alloc register fetch delete reqs browse MySQL — metadata MinIO / S3 — media objects metadata in MySQL · media blobs in object storage via signed URLs · retention by flow-manager

Two separate things ship together, because they share the same understanding of how a clip becomes flows and segments:

ShapeJob
tams-clip-ingesterone-shot commandPut an existing media file into TAMS as a clip
tams-transcode-watcherlong-running serviceWatch for new flows and derive another format from them

The ingester takes a media file and registers it in TAMS as a source with its flows, segmenting the essence and uploading it. It is a command that runs and exits, not a service — the natural shape for “put this thing in the store”, which is an operation with an end.

It is driven by a config file rather than a long command line, so an ingest is reproducible: the same config against the same file produces the same result, and the config can be kept alongside the material it describes. In practice a template config is filled in per clip, which keeps the connection and authentication settings in one place and varies only what actually differs.

Segments are placed by frame count with a conventional GOP structure, so the segment boundaries land where a decoder can start rather than wherever a byte-count happened to fall. That is what makes the ingested clip seekable to the frame afterwards, and it is the difference between material that can be edited and material that can only be played.

The transcode watcher solves a different problem: material arrives in one format, and something downstream needs another — a different frame rate, a different encode profile, sometimes on a different server entirely.

It watches a source TAMS for new video segments, by webhook where available and a periodic backfill poll regardless. That belt-and-braces arrangement is deliberate: webhook delivery is best-effort, and a missed notification would otherwise mean material that silently never gets converted. The poll makes missed events a delay rather than a hole.

For each template it is configured with, it matches flows, transcodes them, and writes the derived essence to a destination TAMS — which may be the same server or a different one. Templates carry the target frame rate, the encode profile and the destination, and are supplied as a mounted JSON file rather than environment variables, because a list of conversion rules is not a setting.

It is idempotent. Running it again over material it has already converted does not convert it twice — which is what makes the backfill poll safe to run constantly rather than something that has to be reasoned about.

Between them these close the two gaps around a live recorder: material that was never live, and material that is in the store but in the wrong shape for what somebody now wants to do with it. Both write into TAMS the same way a recorder does, so nothing downstream — playback, retention, the timeline view — needs to know how a flow got there.