Integrating FLV.js and HLS.js into a Unified Video Player for PC Live Streaming

The project originated from a need to build a PC‑based live‑streaming site that could play FLV live streams and, on user request, replay video using HLS; the initial solution used both flv.js and hls.js together with video.js, but the combined library size was considered excessive.

FLV.js analysis – FLV.js downloads FLV files, transmuxes them into ISO BMFF (MP4) fragments, and feeds them to the HTML5 Video element via Media Source Extensions (MSE). Its core components are NativePlayer and FLVPlayer , with the latter built on MSEController and Transmuxer . The MSEController connects InitSegment and MediaSegment to the SourceBuffer, while the Transmuxer consists of a TransmuxingWorker (multithreaded) and a TransmuxingController that handles download, decode, transcode, and segment delivery.

The TransmuxingController contains three sub‑modules: IOController , demuxer , and remuxer . IOController selects appropriate loaders, stores incoming data, and forwards it to the demuxer. The demuxer parses FLV data into video and audio tracks, which the remuxer then converts into InitSegment and MediaSegment for MSE consumption.

HLS.js analysis – HLS.js first fetches the index.m3u8 playlist, parses levels, then downloads TS fragments, transmuxes them into MP4 fragments, and delivers them via MSE. Its architecture is flatter than FLV.js, with a central buffer‑controller.js handling events such as BUFFER_RESET , BUFFER_CODECS , BUFFER_APPENDING , etc. Example constructor:

constructor (hls: any) {
    super(hls,
      Events.MEDIA_ATTACHING,
      Events.MEDIA_DETACHING,
      Events.MANIFEST_PARSED,
      Events.BUFFER_RESET,
      Events.BUFFER_APPENDING,
      Events.BUFFER_CODECS,
      Events.BUFFER_EOS,
      Events.BUFFER_FLUSHING,
      Events.LEVEL_PTS_UPDATED,
      Events.LEVEL_UPDATED);
    this.config = hls.config;
}

HLS.js uses a series of controllers (playlist‑loader, level‑controller, stream‑controller) and loaders (FragmentLoader, XHRLoader) to manage manifest loading, fragment fetching, demuxing, remuxing, and buffering.

Combining the two libraries – Both share the same overall pipeline: download → demux → remux → MSE. The main differences are the extra playlist parsing step in HLS and distinct control flows. The integration plan keeps FLV.js as the base and adds HLS.js’s loader, demuxer, and remuxer modules, creating a unified player that supports HTTP‑FLV live streams and single‑bitrate HLS VOD.

Implementation steps : Integrate HLS loaders (FragmentLoader, XHRLoader, M3U8Parser, LevelController, StreamController) into FLV.js’s IOController. Introduce separate codec objects: FLVCodec for FLV and TSCodec for TS, managed by a unified _mediaCodec in TransmuxingController. Handle seek for HLS by mapping timestamps to the appropriate fragment. Add logging and error propagation to the existing FLV.js error handling.

Additional enhancements include TypeScript type checking, integration of jamken’s MP4 remuxer to fix audio‑video sync, and support for Supplemental Enhancement Information (SEI) events ( GET_SEI_INFO ) to enable real‑time interactive features in live shows.

Real‑time interaction experiment – Instead of using a socket server, SEI data embedded in the video stream triggers interactive panels synchronized with the host’s presentation, reducing latency and cost.

Conclusion – The unified player was deployed on the Huajiao PC site, open‑sourced as HJPlayer , and is intended to help developers facing similar dual‑protocol streaming requirements.