Watch Together

Watch Together synchronizes playback across devices for developers building a native client or player: one host seeks and moves the party between items, any participant can pause or resume, and all participants stay aligned through a server-coordinated protocol over Server-Sent Events. The server fans events out across replicas via Valkey pub/sub. Clients estimate the server clock and schedule coordinated resumes. Some commands exclude the sender; barriers and host-position frames reach the host too.

This builds on the normal player flow. Each participant still streams the media itself; Watch Together only coordinates when everyone plays.

Feature gate

Watch Together requires FEATURE_WATCH_TOGETHER=true on the server. Check capabilities.features.watchTogether in GET /server using a valid user token before attempting to create or join a session. Anonymous discovery omits capabilities. The administrator can also disable the watch-parties widget, which blocks creation even if the feature flag is enabled.

You’ll need

A user bearer token, a per-device clientId you generate, and an SSE client that can set an Authorization header (the browser’s built-in EventSource cannot, so a browser client needs a fetch-based streaming reader instead).

Watch Together endpoint handlers require an authenticated user and have no dedicated party scope. PATs and service tokens also pass the /api/v1 method gate: POST requires the literal write or admin scope; a fine-grained grant alone does not replace it. Prefer write for a party-capable PAT. Playback separately requires media:write for each participant’s own play session. Service-token path/method restrictions and media library/age restrictions still apply. Use OAuth clients for authentication. The protocol is currently beta; use your instance’s /api-docs?format=yaml for its installed contract.

Start here: the smallest correct integration

Implement these four steps first; the rest of this page explains each part.

1. Create a stable, unique clientId for each device or tab. Connect to the SSE stream and retain its last event id for reconnects.
2. The host creates a session; a guest sends join. Both then follow the same server-issued playback commands.
3. On every buffer_check, cancel an earlier resume task, pause, seek to currentTime, and send ready only once the local player has reached that position and buffered enough to continue.
4. On all_ready, seek again if necessary and play no earlier than resumeAt, and only when the local player is still at the target and ready. A newer buffer_check always wins over an older all_ready.

The four rules that prevent desynchronization

• Never play just because local buffering finished; only the server’s sync, all_ready, or resume event authorizes playback.
• Treat buffer_check.seekInitiated: true as target-authoritative, including on the host.
• Re-anchor any drift estimator at all_ready.currentTime and discard its old samples.
• Cancel pending resume timers on a newer barrier, pause, stall, navigation, or session end.

Protocol reference

Transport: one channel for everything

All client-to-server messages go through one endpoint:

POST /api/v1/watch-together/{code}/events
Authorization: Bearer <token>
Content-Type: application/json

{ "clientId": "client_1704891234567_k8m3x9f2q", "type": "seek", "payload": { "currentTime": 142.5, "isPlaying": true } }

Event type	Status	Body
join	200	{ "party": { … } }
position	200	{ "resyncing": boolean }
everything else	202	{ "accepted": true }

The event itself reaches other clients on the SSE stream, not in this response. All server-to-client events arrive on:

GET /api/v1/watch-together/{code}/events?clientId=<clientId>[&lastEventId=<seq>]
Authorization: Bearer <token>

clientId is required and must belong to the authenticated user in this party. Create or join first; opening SSE does not enroll a device. sync, navigate, and version_change exclude their sender, while barriers, stalls, reactions, and host positions can return to it. Apply your own accepted command locally too. The server keeps a capped replay ring of the last 50 events per session (4-hour TTL). On reconnect, pass the last sequence you saw either as the standard Last-Event-ID header or as ?lastEventId=. The header wins if you send both.

Browser Bearer authentication

EventSource cannot attach an Authorization header. A browser client needs an SSE reader built on fetch; native clients can use their normal HTTP streaming API. Do not put a bearer token in the URL.

clientId vs userId

Field	Identifies	Example
userId	the person (account)	"user-uuid-abc123"
clientId	a specific device / tab	"client_1704891234567_k8m3x9f2q"

The host role belongs to one clientId. The same user on a phone and a TV are two separate participants, and only the host device can seek or navigate: amIHost = party.hostClientId === myClientId.

Session lifecycle

1. Host:  POST /api/v1/watch-together/parties        → 201 { party }
2. Guest: POST .../events {type:"join"}               → 200 {party}; barrier scheduled after ~1 s
3. All:   GET  .../events?clientId=…                  (SSE)
4. Any:   POST .../events {type:"sync"}  (play/pause) · Host: {type:"seek"}
5. All:    POST .../events {type:"position"} every ~10 s · Host: {type:"heartbeat"} every ~5 s
6. Seek/join: buffer_check → clients POST ready → server broadcasts all_ready
   Drift: each guest corrects locally; position reports do not pause the group
7. Stall: POST {type:"stall"} → all pause · POST {type:"stall-ready"} → resume
8. End:   host POST {type:"disband"} · guest POST {type:"leave"}
          (disband is how a session ends; there is no DELETE on the public API)
9. Host gone: no host SSE connection and no host heartbeat for ~10 min
          → server sends session_ended to everyone

When the server ends a party

Besides the host’s disband, the server ends a party on its own when the host is gone: the host has no open SSE stream and has sent no heartbeat for WATCH_TOGETHER_HOST_ABSENT_MINUTES. The default is 10 minutes; a server admin can change it under Admin > Settings or with the environment variable (1 to 1440 minutes). Every participant receives session_ended.

The server checks once a minute, so a party ends up to a minute after the limit. A host whose app crashed can still look connected for about 90 seconds, until its stream’s keepalives stop counting.

For a host client this means:

• Keep the SSE stream open while hosting, and send heartbeat every ~5 s while playing.
• A host that leaves the app for a short while (a TV’s Home button, switching apps on a phone) keeps the party, as long as it comes back within that time.
• After transfer-host, or when the server promotes a new host because the host left, the new host’s time starts from that moment.

Event catalog

The type field is a closed enum. The host-only types are navigate, version, visibility, transfer-host, disband, and activity; the handler returns 403 if a non-host client sends one. seek is also host-only: the service rejects a guest’s seek with 403. sync is open to every participant, so any participant can pause or resume the party.

type	Who	Payload	SSE event emitted
seek	host-only	{ currentTime, isPlaying }	buffer_check → all_ready
sync	any	{ currentTime, isPlaying, playbackRate? }	sync
position	any	{ position }	none; diagnostic drift evidence, no group barrier
navigate	host-only	{ itemId, mediaType, title, posterUrl?, fileId? }	navigate (frame omits mediaType; refresh party)
version	host-only	{ fileId }	version_change: switch to another file (version) of the current item
visibility	host-only	{ visibility: "listed" or "unlisted" }	visibility_changed: list or unlist the party
activity	host-only	{ activity, message? }	host_activity: tell guests what the host is doing (watching, searching, or browsing)
transfer-host	host-only	{ newHostUserId }	host_changed
disband	host-only	{}	session_ended
join	any	{}	participant_joined + buffer_check
leave	any (idempotent)	{}	participant_left
ready	any	{ checkId, currentTime? }	all_ready when last client signals ready
stall / stall-ready	any	{ stallId }	stall / resume
heartbeat	any	{ currentTime, isPlaying }	host_position when sent by the host, otherwise none
reaction / effect	any	{ emoji } / { effect }	reaction / screen_effect

Lumastra’s own clients also block guest seeks in the UI. Do the same in a third-party player: show the party’s position and disable the seek bar for guests. On ready, currentTime is read only from the host.

A host seek does not emit a standalone seek frame

POST {type:"seek"} runs the full buffer_check → ready → all_ready barrier so everyone re-buffers and resumes together; it never broadcasts a plain seek event. Handle host seeks entirely through that barrier and re-anchor your drift reference to the all_ready position (see Re-anchor after every barrier).

heartbeat vs position: a host heartbeat updates party position and broadcasts host_position to everyone, including the host. A guest heartbeat refreshes the party without changing its position or play state. position records drift evidence; even persistent large drift currently does not trigger a group barrier. Its resyncing: true means a barrier is already pending, not that your report started one. Use position for the periodic 10-second report; use heartbeat every 5 seconds. A host heartbeat must carry the host’s live isPlaying. The server persists it (host-only), so the play state a joining guest is told to adopt self-heals to the host’s reality every few seconds. Build the heartbeat from the live player state each time. Never send a cached flag.

An explicit sync, or a host seek that carries isPlaying, pins that play state for 3 seconds. A host heartbeat that contradicts it inside that window is ignored, because it was sampled before the press. The next heartbeat, 5 seconds later, still corrects a play state that a missed sync left wrong.

SSE frame reference (server → client)

Event type	Data fields
connected	party, lastEventId, serverTime (ms), requestReceivedAt, bufferCheck (see below)
sync	state: {currentTime, isPlaying, playbackRate, timestamp, fileId?}, author, clientId?
buffer_check	checkId, currentTime, initiator, isPlaying, seekInitiated
all_ready	checkId, currentTime, resumeAt (ms), isPlaying, serverTime (ms)
stall / resume	author, clientId? / no fields beyond type
host_position	position, timestamp (ms), isPlaying, playbackRate
session_ended	host (public author)
participant_joined	participant: {userId, displayName, images, joinedAt, clientId}
participant_left	author, clientId?
host_changed	newHost, newHostClientId?
navigate	itemId, title, fileId?, author, clientId?
version_change	fileId, author, clientId?
visibility_changed	visibility
host_activity	activity, message?
reaction / screen_effect	emoji, author / effect, author
resync	`reason: “gap"

Positions (currentTime, position) are seconds; resumeAt, serverTime, requestReceivedAt, timestamp, joinedAt, createdAt, and lastSync are Unix milliseconds. playbackRate is a multiplier (0.25–4, default 1 on sync). Public authors contain userId, nullable displayName, and images; do not read the old username, hostUserId, or newHostUserId fields from server frames. The latter remains the request field for host transfer. activity.message is at most 200 trimmed characters. Nonnegative positions and required booleans are validated.

The connected snapshot

Besides party, lastEventId, serverTime, and requestReceivedAt, the connected frame carries bufferCheck: the barrier that is pending right now, or null. It has checkId, currentTime, isPlaying, and seekInitiated. A client that connects or reconnects while a barrier is pending must treat it like a buffer_check: pause, seek to the target, buffer, and send ready with that checkId.

Playback is gated on isPlaying, never on your own buffer

isPlaying on sync, buffer_check, and all_ready means “should playback resume after this sync completes.” A guest must never start playback just because its buffer filled. Any participant can pause or resume the party with sync. The host reports its live state on host_position every ~5 s, so a guest may additionally honor it to self-correct.

A host_position can be old when it arrives

host_position.timestamp is the server time when the host’s heartbeat was processed. After a reconnect the replay ring delivers frames that are seconds old, and a stalled connection can hold one up. If you anchor such a frame at the moment it arrives, a playing host looks seconds behind and your guest seeks back. Anchor it at its send time instead: age = your server-clock estimate − timestamp, and extrapolate the host’s position by that age.

Trust only the age beyond the uncertainty of your clock estimate, which is half the round trip of the handshake you estimated it from, capped at 1 s. A slow first request (cold server, TLS, auth) makes that estimate run ahead, and then every live frame would look old and your guest would sit ahead of the host. The cap stops a very slow handshake from letting a replay that is many seconds old pass as live.

Explicit seeks are target-authoritative

buffer_check.seekInitiated distinguishes a deliberate host seek (true) from a join or drift re-sync (false). For an explicit seek, the server’s currentTime is authoritative for every client, including the host. Do not replace it with a locally reported position, or a rapid backward seek turns into a resume at the old position. For join/drift barriers (false), a host may retain a clearly newer local position as a guard against a stale server snapshot; keep that exception narrow.

A second host seek during the first one’s barrier

The barrier of a host seek pauses the host’s own player until all_ready. If the host seeks again in that window (two quick presses of “skip forward”), the isPlaying of the second seek must be the play state the party intends, not the paused state of your player. Otherwise the second seek is sent with isPlaying: false and the whole party ends up paused. Remember the play state of your own last seek until its barrier settles, and clear it on any explicit play/pause, a remote sync, or an all_ready that ends paused.

Error frames

event: error
data: {"type":"error","error":"Stream write error"}

On a transient error frame, close the stream and reconnect with your current last event id. There is one exception. When the session ends, the server sends session_ended and then an event: error frame with the message Session disbanded by host. Treat that as the end: close cleanly, leave the party UI, and do not reconnect.

JavaScript reference: authenticated transport and coordination

This module runs in a browser with fetch, streams, and crypto.randomUUID(). Pass your existing player as an adapter; stream negotiation remains the normal player flow described below. The adapter must implement these asynchronous operations:

• load(party): stop the previous local play session, start this item/file with the viewer’s token, attach delivery, and leave it paused. Reusing an unchanged selection is allowed. Resolve once metadata and seeking are available.
• position(): return seconds on the whole-file party clock.
• seek(seconds): map the party clock to the local player clock and resolve when done.
• ready(seconds): resolve only at that position with future media data buffered; reject after a bounded wait. Both seek and ready take an AbortSignal as their second argument and must stop promptly on cancellation. Never resolve merely because a timer elapsed.
• playing() returns actual playback state; active() is false while backgrounded, navigating, or otherwise unable to report reliable positions.
• pause(), play(), rate(multiplier), close(): control or end local playback. play() must surface autoplay refusal; close() stops local reporting and releases the local play session/grant. All adapter changes suppress matching media-event echoes.

This reference handles party commands, barriers, generation cancellation, SSE parsing, reconnects, and teardown. Optional calibrated drift, device commands, background OS integration, and the adapter’s playback/progress implementation remain application work. Use explicit UI/OS action handlers for the returned control methods; do not wire every raw pause or playing media event to a party command.

export function watchParty({ base, token, code, clientId, player,
  onParty = () => {}, onError = console.error }) {
  // base is your instance URL, including any deployment subpath, ending in '/'.
  const root = new URL('api/v1/', base);
  const path = `watch-together/${encodeURIComponent(code)}`;
  let party, cursor, snapshotSeq = 0, offset = 0;
  let started = false, stopped = false, online = false, loading = false, stalled = false, recovering = false;
  let generation = 0, barrier = null, resuming = false, resumePlan = null;
  let controller, heartbeat, positionTimer;
  const sleep = (ms, signal) => new Promise((resolve, reject) => {
    const abort = () => { clearTimeout(timer); reject(new DOMException('Cancelled', 'AbortError')); };
    const timer = setTimeout(() => { signal?.removeEventListener('abort', abort); resolve(); }, ms);
    if (signal?.aborted) abort();
    else signal?.addEventListener('abort', abort, { once: true });
  });
  const headers = () => ({ Authorization: `Bearer ${token()}` });
  const host = () => party?.hostClientId === clientId;
  let operation = new AbortController();
  const cancel = () => {
    operation.abort(); operation = new AbortController();
    generation++; resuming = false; player.pause();
  };

  async function request(suffix, body) {
    const response = await fetch(new URL(suffix, root), {
      method: body === undefined ? 'GET' : 'POST',
      headers: { ...headers(), ...(body === undefined ? {} :
        { 'Content-Type': 'application/json' }) },
      body: body === undefined ? undefined : JSON.stringify(body),
    });
    if (!response.ok) {
      const problem = await response.json().catch(() => ({}));
      throw Object.assign(new Error(problem.error || `HTTP ${response.status}`),
        { status: response.status, problem });
    }
    return response.json();
  }
  const send = (type, payload = {}) =>
    request(`${path}/events`, { clientId, type, payload });
  async function refresh() {
    party = (await request(path)).party;
    onParty(party);
  }
  async function load() {
    cancel(); resumePlan = null; loading = true;
    try { await player.load(party); player.rate(party.playbackRate); }
    finally { loading = false; }
  }
  function prepare(check) {
    cancel(); resumePlan = null; barrier = check.checkId;
    const mine = generation, signal = operation.signal;
    // Do not block the SSE reader while buffering: newer commands must cancel this.
    void (async () => {
      await player.seek(check.currentTime, signal);
      await player.ready(check.currentTime, signal);
      if (stopped || mine !== generation) return;
      await send('ready', { checkId: check.checkId,
        ...(host() ? { currentTime: player.position() } : {}) });
    })().catch((e) => { if (e.name !== 'AbortError') onError(e); });
  }
  function schedule(target, resumeAt, playing) {
    cancel(); resumePlan = { target, resumeAt }; resuming = true; const mine = generation, signal = operation.signal;
    void (async () => {
      await player.seek(target, signal);
      await player.ready(target, signal);
      await sleep(Math.max(0, resumeAt - offset - Date.now()), signal);
      if (stopped || mine !== generation || stalled) return;
      // The adapter verifies target/readiness again after the scheduled wait.
      await player.ready(target, signal);
      if (!stopped && mine === generation && !stalled && playing) {
        await player.play();
      }
    })().catch((e) => { if (e.name !== 'AbortError') onError(e); })
      .finally(() => {
        if (mine === generation) { resuming = false; if (!stalled) resumePlan = null; }
      });
  }
  async function receive(data, seq, sentAt) {
    if (data.type === 'connected') {
      const receivedAt = Date.now();
      offset = ((data.requestReceivedAt - sentAt) +
        (data.serverTime - receivedAt)) / 2;
      snapshotSeq = data.lastEventId;
      party = data.party; onParty(party);
      stalled = false; barrier = null; recovering = party.isPlaying;
      await load();
      if (data.bufferCheck) prepare(data.bufferCheck);
      // Re-enroll an existing client after the stream opens. When playing, the
      // server schedules a fresh barrier; this avoids playing stale replay state.
      else if (party.isPlaying) await send('join');
      return;
    }
    if (data.type === 'resync') {
      cursor = String(data.lastEventId); // replace even an ahead cursor
      cancel(); await refresh(); await load();
      onError(new Error('Replay unavailable; explicitly rejoin to recover playback'));
      await close();
      return;
    }
    // The bootstrap is newer than historical playback commands. Keep cursor
    // progress, but do not let replayed seeks/navigation replace that snapshot.
    // Stall/resume remain transient and must be processed in replay order.
    if (seq && Number(seq) <= snapshotSeq &&
        !['stall', 'resume', 'session_ended'].includes(data.type)) return;
    switch (data.type) {
      case 'buffer_check': prepare(data); break;
      case 'all_ready':
        if (barrier && barrier !== data.checkId) break;
        barrier = null; recovering = false;
        party.currentTime = data.currentTime; party.isPlaying = data.isPlaying;
        // A drift estimator would reset here, with its anchor time = resumeAt.
        schedule(data.currentTime, data.resumeAt, data.isPlaying);
        break;
      case 'sync':
        recovering = false;
        party.isPlaying = data.state.isPlaying;
        party.playbackRate = data.state.playbackRate;
        player.rate(party.playbackRate);
        if (barrier) break; // readiness continues even if intent changes to paused
        schedule(host() ? player.position() : data.state.currentTime,
          Date.now() + offset, party.isPlaying);
        break;
      case 'stall':
        stalled = true;
        if (barrier) player.pause(); // retain the barrier's readiness task
        else cancel(); // retain resumePlan while cancelling its timer
        break;
      case 'resume': {
        stalled = false;
        const plan = resumePlan;
        if (!barrier && !recovering) schedule(plan?.target ?? player.position(),
          Math.max(plan?.resumeAt ?? 0, Date.now() + offset), party.isPlaying);
        break;
      }
      case 'navigate':
        barrier = null; await refresh(); await load(); break;
      case 'version_change': {
        const target = player.position();
        barrier = null; await refresh(); await load();
        schedule(target, Date.now() + offset, party.isPlaying); break;
      }
      case 'host_changed': case 'participant_joined': case 'participant_left':
      case 'visibility_changed': await refresh(); break;
      case 'session_ended': await close(); break;
      // host_position is input for optional guest-local drift correction.
    }
  }

  async function stream(signal) {
    const params = new URLSearchParams({ clientId });
    if (cursor !== undefined) params.set('lastEventId', cursor);
    const sentAt = Date.now();
    const response = await fetch(new URL(`${path}/events?${params}`, root),
      { headers: headers(), signal });
    if (!response.ok) throw Object.assign(new Error(`SSE HTTP ${response.status}`),
      { status: response.status });
    if (!response.body) throw new Error('Missing SSE body');
    online = true;
    const reader = response.body.getReader(), decoder = new TextDecoder();
    let buffer = '';
    try {
      while (!stopped) {
        const { done, value } = await reader.read();
        if (done) break;
        buffer += decoder.decode(value, { stream: true });
        let end;
        // Lumastra emits LF-delimited frames; chunks may split anywhere.
        while ((end = buffer.indexOf('\n\n')) >= 0) {
          const raw = buffer.slice(0, end); buffer = buffer.slice(end + 2);
          let name = 'message', id, lines = [];
          for (const line of raw.split('\n')) {
            if (line.startsWith('id:')) id = line.slice(3).trim();
            if (line.startsWith('event:')) name = line.slice(6).trim();
            if (line.startsWith('data:')) lines.push(line.slice(5).trimStart());
          }
          // Cursor-only frames advance replay without dispatching a message.
          if (id !== undefined) cursor = id;
          if (!lines.length) continue; // includes ': ping' keepalives
          const data = JSON.parse(lines.join('\n'));
          if (name === 'error') {
            if (data.error === 'Session disbanded by host') await close();
            else throw new Error(data.error);
          } else await receive(data, id, sentAt);
          if (stopped) break;
        }
      }
    } finally { online = false; await reader.cancel().catch(() => {}); }
  }
  async function run() {
    while (!stopped) {
      controller = new AbortController();
      try { await stream(controller.signal); }
      catch (error) {
        if (stopped) break;
        onError(error);
        // Refresh authentication in your token provider before starting again.
        if ([401, 403, 404].includes(error.status)) { await close(); break; }
      }
      if (!stopped) { cancel(); await sleep(2000); }
    }
  }
  async function close() {
    if (stopped) return;
    stopped = true; cancel(); controller?.abort();
    clearInterval(heartbeat); clearInterval(positionTimer);
    await player.close();
  }
  const reportable = () => online && party && player.active() &&
    !loading && !barrier && !stalled && !recovering && !resuming;
  return {
    async start() {
      if (started || stopped) throw new Error('Create a new controller to restart');
      started = true;
      party = (await send('join')).party;
      heartbeat = setInterval(() => {
        if (reportable() && host()) void send('heartbeat', {
          currentTime: player.position(), isPlaying: player.playing(),
        }).catch(onError);
      }, 5000);
      positionTimer = setInterval(() => {
        if (reportable()) void send('position', { position: player.position() }).catch(onError);
      }, 10000);
      void run(); // start() enrolls; onParty observes stream bootstrap
    },
    async setPlaying(isPlaying) {
      await send('sync', { currentTime: player.position(), isPlaying,
        playbackRate: party.playbackRate });
      party.isPlaying = isPlaying; // sender will not receive its own sync
      if (barrier) return; // all_ready carries the revised intent
      schedule(player.position(), Date.now() + offset, isPlaying);
    },
    async seek(currentTime) {
      if (!host()) throw new Error('Only the host device can seek');
      // Keep intended play state while the barrier pauses the physical player.
      await send('seek', { currentTime, isPlaying: party.isPlaying });
    },
    async navigate(selection) {
      if (!host()) throw new Error('Only the host device can navigate');
      loading = true; cancel();
      try { await send('navigate', selection); barrier = null; await refresh(); await load(); }
      finally { loading = false; }
    },
    async changeVersion(fileId) {
      if (!host()) throw new Error('Only the host device can change version');
      const target = player.position();
      loading = true; cancel();
      try {
        await send('version', { fileId }); barrier = null;
        await refresh(); await load();
        schedule(target, Date.now() + offset, party.isPlaying);
      } finally { loading = false; }
    },
    stall: (stallId) => send('stall', { stallId }),
    recovered: (stallId) => send('stall-ready', { stallId }),
    async exit(endForEveryone = false) {
      try { await send(endForEveryone ? 'disband' : 'leave'); }
      finally { await close(); }
    },
    close, // local shutdown; use exit() for an intentional departure
  };
}

Provide token as a function returning the current access token. Keep clientId stable across reconnects, unique per active tab/device; persist the same value used at creation. Call start() once. A native client can implement the same adapter and state machine using its HTTP streaming API. A browser must handle an autoplay denial by showing a user gesture control; keep the party paused locally until playback is allowed.

The example deliberately waits for a fresh barrier after reconnecting to a playing party. Replay still updates the cursor, including sender-excluded frames. A gap has no complete transient stall snapshot: the reference closes local playback and reports that a fresh explicit rejoin is needed. Construct a new controller to rejoin without the stale cursor; do not guess that every participant is ready. Production clients should also honor Retry-After with backoff on 429/capacity errors and fence overlapping local control requests in action order.

Create a session (host)

POST /api/v1/watch-together/parties
Authorization: Bearer <token>
Content-Type: application/json

{ "itemId": "550e8400-e29b-41d4-a716-446655440000", "mediaType": "movie", "title": "Nosferatu (1922)", "currentTime": 0, "clientId": "living-room-player", "visibility": "unlisted" }

mediaType is one of movie, episode, series. A series id has no playable item of its own, so resolve it to an episode first and create the party for the concrete movie or episode that Play would start. Treat series as “resolve to an episode first”. Response is 201 with a bare { party: WatchTogetherParty }. party.code is the shareable session code (e.g. "MOVIE-7X3K") used in all subsequent requests, and carries hostClientId (the host device, see clientId vs userId above), visibility (listed or unlisted), participants[], and createdAt/lastSync. Omit visibility to create an unlisted party. A host can change it later with the visibility event.

Creation starts paused with rate 1 and currentTime defaulting to 0. fileId is optional; an ID that does not belong to the item is dropped, so inspect the returned party. Supplied posterUrl is ignored at creation; the server derives artwork.

If the user is already in another party, creating one first ends that party when the user is its host, or leaves it when the user is a guest there. GET /api/v1/watch-together returns the user’s current party (the one a Rejoin prompt should offer). A party the user left earlier ending later never clears that.

Join a session (guest)

POST /api/v1/watch-together/MOVIE-7X3K/events
{ "clientId": "client_…", "type": "join", "payload": {} }

Response is 200 { party }. The server pauses the stored party state, broadcasts participant_joined, and schedules a barrier after about 1 second so the new SSE stream can connect. buffer_check then reaches all current participants, including the joiner, so everyone pauses and synchronizes before the new guest starts playing. If the guest’s SSE stream connects after that broadcast, it picks the barrier up from connected.bufferCheck. If a barrier never completes, the server resolves it after about 5 seconds, with a further 1.5-second resume lead time. Startup and local buffering can take longer; this is not a five-second join deadline. An exact rejoin with the same user/client pair is idempotent and schedules a barrier if the party is playing. A client ID already owned by another user is refused.

A code is an invitation, not a media-access grant. Join checks the viewer’s library and age access and returns 404 on refusal. The current participant cap is 100; a full party also returns 404. GET {code} requires account membership; the SSE stream and every non-join command require the exact owned client ID. A listed party is discoverable, not automatically joined. The listing returns {items, page} and filters library/age access; ?includePrivate=true includes unlisted parties only for an admin principal.

Reconnect when the replay cursor is stale

The SSE replay ring keeps the newest 50 business frames. The unnumbered connected frame arrives before replay and includes the current party snapshot and lastEventId. If the cursor is older than the ring, the server sends resync with reason gap; a cursor ahead of the server counter gets reason ahead. Malformed cursors, or a cursor received while Valkey is unavailable, also produce gap. In each case replay is skipped. Re-read GET /api/v1/watch-together/{code} and continue from the current snapshot; do not apply an incomplete replay. Replace an ahead cursor with the server value (taking max(old, new) would repeat the error forever). Keep the replay cursor separate from the snapshot watermark: setting it to the bootstrap counter before processing replay would wrongly discard missed frames. Apply historical frames in sequence without letting old playback commands overwrite the newer bootstrap selection/barrier. Business frames normally have increasing id: values; replay-storage failure can produce live frames without an ID. Handle those without resetting the cursor. A sender-excluded replay entry can contain only id: and no data; advance the cursor without dispatching a command. Keepalive comments (: ping) are not business events.

Barrier details: buffer-gated resume

On a host seek or join/rejoin the server broadcasts buffer_check. Every client pauses, seeks to the target position, buffers, verifies it has actually reached the target, and only then sends ready. Once the expected live clients have reported ready (or the five-second server timeout expires), the server broadcasts all_ready with a resumeAt timestamp about 1.5 seconds ahead of the server clock so everyone un-pauses together.

Buffer-gated resume

Fit diagram to width

Scroll, swipe, or click and drag to explore. Use the arrows to move, or fit the diagram above.

Schedule the resume against the server clock, but treat the timestamp as an earliest play time, not permission to play an old player position. At that time, verify again that the player is near currentTime, is not seeking, and has future media data; poll briefly and cancel the older resume task if a newer buffer_check arrives. Do not fall back to an unconditional play() after a timeout.

If the host has paused when a guest joins, buffer_check.isPlaying is false; everyone still syncs to the target position via all_ready, but no one resumes.

Re-anchor after every barrier

If you run drift correction, you keep a local estimate of the host’s live position: the last host_position (or sync) value, extrapolated by elapsed time. When an all_ready barrier completes, overwrite that anchor with the barrier’s currentTime and reset your drift compensator. This matters because a host seek arrives only as the barrier: it carries no fresh host_position. Skip the re-anchor and your next drift tick extrapolates from the stale pre-seek anchor, sees a phantom drift, and hard-seeks the guest right back off the position everyone just synced to. Anchor playing time at resumeAt, rather than at receipt or at all_ready.serverTime; the scheduled 1.5-second pause must not count as playback.

Navigating between items (host)

On navigate the server sets the party to currentTime 0 and isPlaying false, with no barrier. The navigate frame goes to every participant except the sender. Guests switch to the new item and wait. The host does not start the next item locally either. The party waits at the start of the new item until someone presses Play, and that sync starts everyone. Refresh GET {code} to obtain the full media type and selection because the navigate frame omits them. The sender also loads the accepted item locally. Navigation is refused with 403 if any participant cannot access the target; host access alone is insufficient. An inaccessible target for the host returns 404. Invalid foreign fileId is dropped on navigation; version instead refuses an invalid file with 403.

A version change keeps the party clock and does not start a barrier. Each client reopens its own stream at that clock using the accepted physical fileId. Suspend reports during replacement, then restore the current intended play state.

When the host moves to another item (navigate) or ends playback, stop position/heartbeat reporting and drift correction the instant navigation begins, and never report a position read from a torn-down player: it reads currentTime: 0, and a stray report at that moment broadcasts host_position: 0 to everyone, hard-seeking a guest still on the old item back to the start. Guests should suppress drift hard-seeks while a navigate is in flight.

Pause, resume, and the position a guest sends

Any participant can pause or resume the party with sync. When a guest pauses a playing party, send the guest’s real currentTime. The server accepts it only if it is at most 1 s behind or up to 10 s ahead of the host’s last heartbeat. Otherwise it keeps the host’s position, so a guest whose player still sits at 0 cannot rewind the party. For isPlaying: true from a guest, the position is ignored. In every other case only the host sets the position.

The host applies a guest’s sync play state/rate without seeking backwards to the echoed server position, which may be based on an older heartbeat. Keep position changes authoritative through host seeks and their barriers.

Play and Pause from the lock screen, a headset, a remote, media keys or the OS “Now Playing” controls are user actions like the on-screen button: send sync for them. Pausing only your local player leaves the rest of the party playing.

Echo suppression

After you apply a remote play, pause, or seek, your own player changes state and your code sees it. Do not send that change back. Suppress only a change toward the state the remote command asked for. A user press the other way, inside the same window, is a real action and must be sent. A blanket time window after every remote command swallows a host’s Play pressed right after a join.

The host’s app goes to the background

Send nothing when the app goes to the background; do not leave. But the operating system or browser may pause the host’s player while it is in the background (iOS and mobile browsers suspend background media; Chromium can freeze a background tab). That is not a party pause, and no sync goes out for it. Make sure your heartbeat does not report it either:

• When you go to the background while playing, remember the position and the time.
• While the system keeps your player paused, do not send heartbeat. A “paused” heartbeat would mark the party paused while the guests keep playing.
• On return, if the party is still playing (GET /api/v1/watch-together), seek your own player to the current authoritative host-position estimate and play. A saved background anchor plus elapsed seconds × playbackRate is only a fallback when no seek, navigation, pause, or host transfer occurred while away. The guests never stopped, so do not move them with a party-wide seek.
• A pause the user made on the lock screen while in the background is a real pause: send sync for it (see above) and do not catch up afterwards.

Starting an item

Never auto-play on load inside a party. The session decides how a new item starts. This applies to guests after every reload, and to the host on first load and after navigate. Load the item paused, then follow sync and all_ready.

End of an item as a guest

When your own media ends, do not close the player and do not send leave. Stay in the session and wait for the host’s navigate or for session_ended. If the host has gone away, the server ends the party after about 10 minutes (see When the server ends a party), so a guest never waits forever.

Each participant owns a local playback session

Joining a party does not start playback, share the host’s grant, or bypass admission. Every device calls POST /api/v1/play-sessions with its own bearer token, mediaId: party.itemId, its own deviceId, optional fileId: party.fileId, and startPositionMs: Math.round(party.currentTime * 1000). Use entry: "explicit" for a party start. Pass the actual player capabilities and use the returned delivery URLs, playbackSessionId, grant, and reporting block. Keep the stream paused until the party authorizes playback. A full local start recipe is in Building a player.

Use each viewer’s own reporting.href, bearer authentication and intervalMs for progress; Watch Together’s heartbeat and position do not renew that local lease or record watched progress. Convert party seconds to progress milliseconds. Stream URLs carry the local playback grant; do not copy them to guests or put account tokens in URLs. On navigation, version replacement, leave, or terminal party shutdown, stop local reporting, end the old play session, cancel stream requests/timers, and release the old player before opening a replacement. See Playback state for statuses and cleanup.

Physical releases and episode windows

Watch Together carries a physical fileId. Select a complete physical release for the concrete movie/episode and use the same release on every participant. A version command changes that physical file only.

Multi-episode files are supported. Their party positions and progress remain on the whole physical file clock. delivery.seek.streamStartPositionMs selects an episode’s entry point; it does not create a zero-based episode clock or a stop boundary. When creating a party for a later episode in such a file, initialize currentTime from the negotiated whole-file entry position (or the host’s actual position). Do not replace it with episode-relative zero. For an HLS player whose local media time restarts at a seek base, map both directions using the negotiated stream base. Do not subtract the episode start from party reports, add it twice, or clamp playback at an episode window’s end.

Host handoff and departure

Read party.host.userId for the account and hostClientId for the controlling device. transfer-host takes another participant’s user UUID, not a client ID. The server chooses that user’s first matching participant; this API cannot target another device of your own account. On host_changed, use newHost and newHostClientId (or refresh the party) and switch seek/navigation controls and heartbeat ownership immediately.

A host device’s leave hands control to its oldest surviving same-account device, or to the oldest remaining participant if none exists. The last departure ends the party. disband ends it for everyone and is host-device-only on this event surface. Losing SSE is not an intentional leave; reconnect with the same identity. Background guests can be pruned after stale connection evidence and a grace period; if reconnect returns 403, explicitly join again before opening SSE. Never silently replace your client ID to bypass an ownership refusal.

Stall generations

For a real mid-playback rebuffer, generate a fresh stallId, send stall, then send stall-ready with the same ID when recovered. The recovery cannot clear a newer stall generation, and an overtaken stall cannot revive an already recovered one. The server resumes only after every active stall clears, or abandoned entries age out (about 30 seconds, with a failsafe timer). resume has no position or play-state payload: preserve the party’s intended isPlaying and any newer pause/barrier. Do not unconditionally play a paused party. The connected snapshot contains a pending buffer check, but no complete active-stall snapshot; gap recovery needs conservative paused handling and a fresh join when state is uncertain.

Reference state machine

State	Meaning	Entered on
IDLE	Not in a session	Initial, or after leave/session_ended
PAUSED	In session, not playing	sync or all_ready with isPlaying: false
PLAYING	In session, playing	sync, or all_ready once the local target is ready
BUFFERING	Waiting for every participant to be ready	buffer_check received
STALLED	Group paused for a slow participant	stall received

Optional: drift correction

Guests report position every ~10 s (type:"position"). Between hard re-syncs, close small drift by nudging playbackRate over a target window rather than seeking:

drift    = hostPosition − guestPosition    (positive = guest behind)
syncTime = min(targetSyncTime ≈ 300 s, hostRemaining)
guestRate = originalRate × (syncTime + drift) / syncTime

A 10-second lag over a 5-minute window produces a ~3.3% rate change: imperceptible, but enough to catch up. Reserve a hard seek for large drift or an explicit host seek, and stop adjusting in the last minute of the media. Choose and test your own hard-seek threshold (for example, 3 s) and cap rate correction to ±5% of the party rate. At a 1× baseline, a 5 s gap takes about 100 s to close at the maximum nudge. Reset the estimator on every all_ready (see Re-anchor after every barrier).

Advanced: calibrated drift compensation

The web player uses a calibrated variant, moving through three states per participant:

• MEASURING: the web integration gathers samples for 3 s (at least 3 samples).
• CALIBRATING: compute the target-sync-time rate from those samples.
• LOCKED: rate is applied. Recalibrates on an exponential backoff (web integration: 30 s → 60 s max) rather than continuously, and returns to MEASURING on a drift past the hard-seek threshold or a host seek.

Simpler clients can skip this and use the fixed-rate dead-zone above. The calibrated variant changes rate less often; either needs a fresh anchor after every barrier.

What a third-party player must send

• A stable per-device clientId on the SSE URL and in every event.
• Host: a heartbeat every ~5 s with { currentTime, isPlaying } read from the live player, and a position every ~10 s. Stop both on navigate and on teardown.
• ready only when the player is actually at the target and buffered, with the checkId of the barrier.
• stall and stall-ready around real rebuffering, not startup buffering.
• Never broadcast state you did not cause (see Echo suppression).
• An explicit leave when this device exits, or disband when the host ends the party for everyone. Send nothing when the app goes to the background or the user switches apps, and no heartbeat while the system keeps a host paused there (see The host’s app goes to the background).

Optional: reactions and screen effects

Reactions (type:"reaction"): 😂 😮 😢 😍 🔥 👏 😱 🤯. Screen effects (type:"effect"): shake confetti snow hearts flash zoom spin rage matrix vhs. Both are available to any participant.

Device remote control

Party membership and device command delivery are separate protocols. Use party.join through the device command API to invite a supported device; that device must authenticate, join with its own clientId, start its own playback, and connect SSE. See Remote control and the API reference for command delivery and device-session IDs.

Errors and refusal behavior

HTTP failures use application/problem+json with code, error, and status. Do not infer success from a delivered request alone; check the HTTP response.

Status	What to do
401	Refresh/re-authorize the account token; do not retry with a media grant.
403	Check scope_required, membership/client ownership, and host role; navigation may also be blocked by another viewer’s access.
404	Feature disabled, missing/expired party, or inaccessible media/join refusal; do not expose private title details.
422 validation_failed	Fix the payload, required fields, or enum; do not strip policy or identity fields and retry.
429	Back off using Retry-After; join/connect and concurrent SSE limits apply.
503 stream_limit	SSE capacity is exhausted; retry later with backoff.
503 / 500	Dependency/startup failure; bounded retry, then show a recoverable error. Some mutation failures currently collapse to 403/404 or 500.

Playback negotiation can separately refuse admission or file compatibility even after joining. Keep the local player paused and explain that failure; never bypass it with a host’s grant, another identity, or another release selected without agreement.

Endpoint reference

Method	Path	Who	Description
GET	/api/v1/watch-together	user	Current user’s active party (or null)
GET	/api/v1/watch-together/parties	user	List parties the host chose to make visible; admins can request unlisted parties
POST	/api/v1/watch-together/parties	user	Create a paused party (201 {party})
GET	/api/v1/watch-together/{code}	user	Get a party by code
POST	/api/v1/watch-together/{code}/events	user	Send an event
GET	/api/v1/watch-together/{code}/events?clientId=	user	SSE event stream

All Watch Together endpoints require a valid bearer token. The SSE endpoint additionally returns 403 if the caller is not a participant, and 404 if the feature is off or the session does not exist.

Implementation checklist

• Generate a stable per-device clientId; pass it on the SSE URL and in every POST /events body.
• Check capabilities.features.watchTogether before showing the feature.
• Estimate clock offset using both handshake timestamps and the local send/receive times; retain half the network round trip as uncertainty (capped at 1 s).
• Implement buffer_check → ready → all_ready; treat seekInitiated: true as target-authoritative, including on the host.
• Cancel a pending all_ready resume on a newer buffer_check; never use an unconditional timeout resume.
• Correct drift locally; periodic position reports never pause the group. Use playbackRate (±5% at most); hard seek only for large drift according to your tested client threshold.
• Re-anchor your host-position estimate on every all_ready.
• Stop position/heartbeat reporting the moment you start navigating away.
• Reconnect with Last-Event-ID (or ?lastEventId=) on SSE disconnect, but never after session_ended.
• On resync, discard the missing replay range and refresh the party snapshot.
• Enter the barrier in connected.bufferCheck and send ready.
• Never auto-play on load; stay in the session when your own media ends.
• Age host_position by its timestamp, trusting only the age beyond your clock estimate’s uncertainty (half the round trip, at most 1 s).
• Send sync for lock-screen, headset and remote Play/Pause.
• A second host seek inside a barrier carries the intended play state, not the paused player’s.
• Host in the background: never report a pause the system imposed; catch up locally on return.

Next

• Local development: running the stack to test this yourself.
• Architecture decisions: why the Developers surface is shaped the way it is.