Broadcast standards and transport

Two families, one vocabulary

The standards that describe compressed video come from two bodies that have worked together for a long time. ISO/IEC MPEG publishes the MPEG series, which covers not only video coding but also the packaging of audio and video into streams. The ITU-T publishes the H.26x series, which concentrates on video coding itself. Several well-known standards carry both names: the scheme commonly called H.264 AVC is also published as the advanced video coding part of the MPEG-4 family, and H.265 HEVC is jointly published as well. When reading a specification, the dual numbering is not a contradiction but a record of joint work.

A codec specification such as H.264 AVC defines how pictures are coded, but it does not by itself define how the coded data is carried. That second task belongs to the system layers, and it is where the concepts of elementary stream and transport stream appear.

Elementary streams and packetised streams

An elementary stream is the raw output of one encoder: a sequence of coded video pictures, or a sequence of coded audio frames, in the order the encoder produced them. It has no packet boundaries and no timing marks beyond those the codec itself defines. To carry an elementary stream over a channel, the system layer slices it into packets and adds the headers that a receiver needs.

A packetised stream is that sliced form of a single elementary stream, with each packet carrying a small header and a segment of coded data. A transport stream is a higher level again: a single continuous carriage that interleaves the packets of several packetised streams, so that a receiver can recover the video, the audio and the configuration data from one line. A storage-oriented arrangement follows the same idea but is aimed at a medium without errors, whereas a transport stream is designed for a channel where a packet may occasionally be lost.

The timing model

Broadcast is a timed activity, and a receiver must know when to present each picture and each audio frame. The system layer therefore carries timing references, usually called clock references, that let the receiver reconstruct a common timebase. One reference establishes the rate of the system clock; others mark the moment at which a coded unit should be decoded and the moment at which it should be presented. The difference between the two moments must be at least the time the decoder needs to do its work, so the encoder deliberately places them a little apart.

When several elementary streams share one carriage, their timing marks share the same clock reference, and this is what keeps lip-sync intact. If a receiver drifts, it corrects its clock against the incoming references rather than running without correction, and the picture and sound remain aligned. The design has proved durable: the same idea underlies the transport of compressed video on satellite, on cable and, in adapted form, over packet networks.

Service information

Alongside the picture and sound, a broadcast carriage carries data that describes what is being sent. This service information includes the identifiers of the programmes, the languages of the audio tracks, the parental ratings and the schedule of the multiplex. A receiver reads it to build its channel list and to know which packets belong to which programme. Without it, the packets would be an unlabelled rail of data; with it, a viewer can tune to a named channel and hear the correct language.

Because service information is ordinary data, it may be extended by each operator, and different regions have adopted different tables. This is one of the reasons a receiver built for one market may not fully understand the listings of another, and why the international specifications leave room for regional practice. For the student of communication standards, the system layer is the reminder that a coded picture is only half the story: the other half is the discipline of carrying it, timed and labelled, to a receiver that was not present when it was made.