Momentum Zoom Fellowship project

Under the hood

How Momentum is built.

How the system fits together: where the code came from, how live lecture audio turns into a timeline and glossary, and how the AI keeps answering when a provider goes down.

If you want the engineering detail rather than the tour, the technical deep-dive has the sequence diagrams, the message protocol, and the reasoning behind each decision.

One web pageThe whole app is a web page that Zoom shows inside the meeting
One backendA single server handles messages, transcripts, and AI
Three AI modelsIf one model fails, the next one automatically takes over

The big picture

The pieces and how they connect.

Everything in Momentum lives in one of three places: the Zoom meeting itself, our server, or an outside service we call for help. The professor and the students never talk to each other directly. Every message makes a quick round trip through the server, which keeps everyone's screen in sync.

Inside the Zoom meeting

Host Dashboard What the professor sees: launch polls, start games, watch the live analysis
Student View What each student sees: timeline, glossary, transcript, bookmarks

Same app, same web address. It checks your Zoom role and shows the right screen.

Our server (one machine in the cloud)

Web page files Serves the app itself to Zoom
Message relay Passes polls, answers, and scores between host and students
Transcript listener Receives the live lecture transcript from Zoom, line by line
AI engine Turns transcript into polls, topics, glossary terms, and recaps
Database Remembers transcripts, bookmarks, quizzes, and translations

Outside services

Zoom Cloud Sign-in, live transcription, and meeting events
ASU CREATE AI Gateway to Claude 4.5 Sonnet and GPT-5
AWS Bedrock Backup AI (Llama 3 70B) that needs no API key

The starting point

It began with Zoom's Arlo app.

A Zoom App runs in an unusual place: a small browser embedded inside the Zoom client, with its own rules for security, sign-in, and talking to the meeting. Momentum started from Arlo, Zoom's open-source reference app, to get those basics working, then rebuilt around the classroom features it needed.

Borrowed from Arlo

  • The SDK loading trick. Use the global window.zoomSdk from Zoom's own script tag. The npm package alone can't talk to the Zoom client and silently times out.
  • One meeting ID for everyone. Ask Zoom for the meeting's UUID the way Arlo does, so the host and every student agree on which room they're in.
  • Starting live transcription. Call the transcription API with options the way Arlo does, because the shortcut method accepts no settings at all.
  • Messaging over WebSockets. Arlo relays real-time messages through its own server rather than Zoom's built-in app messaging. Momentum followed suit after the built-in route proved unreliable.

Built new for Momentum

  • The classroom features. Polls, the trivia arena, the live topic timeline, the glossary, bookmarks, and post-class recovery packs.
  • The transcript pipeline. Storing every line of lecture with an order number, surviving reconnects, and handling Zoom's two-IDs-for-one-meeting quirk.
  • The tiered AI engine. Three models in a failover chain, with prompt guards, JSON repair, and per-task tuning.
  • Translation with a shared cache. Six languages, translated once per class instead of once per student.

Real-time sync

How a poll reaches every student.

The server runs a simple relay. Every meeting gets a room, and anything one person sends goes to everyone else in that room.

  1. 1 Professor Launches a poll

    One click sends the question to the server with a sequence number attached.

  2. 2 Server Relays it to the room

    Every student connected to that meeting's room gets the poll instantly.

  3. 3 Students Answer on their screens

    Each answer travels back through the same relay, never directly to the host.

  4. 4 Professor Watches results live

    Answers roll into a bar chart in real time, and the professor adjusts the class.

Why the sequence numbers matter

Classroom wifi is messy. Messages can arrive late, twice, or out of order. Every message carries a counter, and each screen simply ignores anything older than what it has already shown. That one rule keeps everyone's view settling to the same state instead of flickering. The relay also protects itself: it drops connections that don't name a meeting, caps message sizes, rate-limits chatty clients, and pings every connection every 30 seconds to sweep out "ghost" students whose laptops fell asleep.

The transcript pipeline

How lecture audio becomes study material.

Momentum doesn't do its own speech recognition. It uses Zoom's Real-Time Media Streams (RTMS), a service where Zoom transcribes the lecture as it happens and streams the text to our server. From there, AI turns raw transcript into topics, definitions, and bookmarks.

Professor speaks Just teaches the class, nothing special
Zoom transcribes live RTMS turns speech into text as it happens
Server saves each line Numbered in order, with the speaker's name
AI reads the last few minutes Spots topic changes and new technical terms
Students see it live Timeline cards, glossary entries, and transcript, in 6 languages

One meeting with two different IDs

Zoom refers to the same meeting by two different ID strings: one given to the app inside the meeting, another used by the transcription service. So transcripts were being saved under one name while the app kept asking for them under the other, and got back nothing, with no error to point at the cause. The fix: right before transcription starts, the app tells the server "here's my ID," and when the transcription service announces itself moments later, the server links the two IDs together. If a lookup still misses, the server checks active transcription sessions as a safety net. The app never has to know two IDs ever existed.

Only calling the AI when something new is said

The live timeline runs on a simple loop: every 10 seconds, check whether anything new was said. If not, do nothing: no AI call, no cost. If yes, send just the fresh transcript to the AI and ask one question: "did the topic change?" Greetings, audio checks, and administrative chatter get filtered out first. Missing a topic is a small problem. A glossary full of "the professor said good morning" is a much bigger one.

The AI engine

Three AI models with automatic fallback.

If one AI provider goes down or slows to a crawl, the class shouldn't stall. Every AI request in Momentum goes through one function that tries three models in order and returns the first response it gets. That covers polls, quizzes, topic updates, the glossary, translations, and recovery packs.

First choice Claude 4.5 Sonnet Via ASU's CREATE AI gateway. Handles almost everything.
Backup GPT-5 Same gateway, different model. Steps in automatically.
Last resort Llama 3 70B On AWS Bedrock. Needs no API key: the server's own cloud identity is the credential, so it still answers when every token has expired.

Hard time limits on every call

Each attempt gets a hard time limit (30 seconds for the gateway, 15 for Bedrock). A hung provider fails fast and the chain moves on, instead of leaving a professor staring at a loading wheel.

Getting clean structured data back

The app needs structured answers (a poll is a question plus four options), but models love wrapping answers in chatty prose. A small repair step gives every response four chances to yield clean data, and lecture text is scrubbed before it enters a prompt so nothing can break the instructions.

Translating each line once

Six languages, but each transcript line is translated exactly once and cached in the database. If thirty students switch to Spanish in the same second, one AI call does the work and twenty-nine requests wait for its result. If translation fails, students see the English original, never a blank.

Where it runs

The server setup.

Zoom needs a public, secure web address it can load the app from and send transcripts to. Momentum runs on one cloud server, with a tunnel in front of it supplying that address.

Zoom The Zoom client loads the app; Zoom Cloud streams transcripts and events
Public tunnel Gives the server a stable public HTTPS address

The cloud server

Node server, one process API, relay, transcripts, and web pages, all in one
Auto-restart Comes back on failure and survives reboots
Cloud identity for AI The machine's identity is the credential, so no AI keys are stored on it

The full technical write-up

The complete technical deep-dive, with sequence diagrams, the message protocol, and every design decision, lives in the open source repository.

Read the deep-dive Try the demo