pmeerw's blog /

Some sites have online videos playable with an embedded JavaScript player. It would be nice to store those files for offline viewing...

Firefox can produce a .har file (HTTP Archive file), a JSON-formatted log that records all network traffic. The particular site I was interested in used something called THEOplayer. It fetches chunks of content periodically. The chunks seem part of an MPEG transport stream (ts). Each chunk is requested with a signature on the request (which I did not care to compute). However, all requests for all chunks are listed upfront.

This is what a single request looks like:

ttps://chunk-oci-us-phoenix-1-vop1.fastly.mux.com/v1/chunk/EO...1Z/0.ts?skid=default&signature=Nm..MA==&zone=2

And the list of chunks is given here in the HAR file:

          "content": {
            "mimeType": "application/vnd.apple.mpegurl",
            "size": 35377,
            "text": "#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-TARGETDURATION:6\n#EXT-X-PLAYLIST-TYPE:VOD\n#EXTINF:5,\nhttps://chunk...

Probably there's tools that already do the job, but here's my attempt

Let's get to work (using copilot): prompt #1:

create python script that opens a .har file (firefox browser network log), file the list of video content urls (content with mimetype "application/vnd.apple.mpegurl", text element), extract the list of urls with /chunk/ and download all chunks and concatinate into .ts file

prompt #2:

there is a file sample.har to test

Resulting Python code:

#!/usr/bin/env python3
"""Extract HLS chunk URLs from a Firefox .har file, download and concatenate into a .ts file."""
import argparse
import json
import sys
from concurrent.futures import ThreadPoolExecutor
from urllib.request import Request, urlopen
from urllib.parse import urljoin

MIME = "application/vnd.apple.mpegurl"


def find_chunk_urls(har_path):
    with open(har_path, encoding="utf-8") as f:
        har = json.load(f)
    urls = []
    for entry in har["log"]["entries"]:
        content = entry.get("response", {}).get("content", {})
        if MIME not in (content.get("mimeType") or "").lower():
            continue
        text = content.get("text")
        if not text:
            continue
        if content.get("encoding") == "base64":
            import base64
            text = base64.b64decode(text).decode("utf-8", "replace")
        base = entry["request"]["url"]
        for line in text.splitlines():
            line = line.strip()
            if line and not line.startswith("#") and "/chunk/" in line:
                urls.append(urljoin(base, line))
    # de-duplicate, keep order
    return list(dict.fromkeys(urls))


def fetch(url, retries=3):
    for i in range(retries):
        try:
            req = Request(url, headers={"User-Agent": "Mozilla/5.0"})
            with urlopen(req, timeout=30) as r:
                return r.read()
        except Exception as e:
            if i == retries - 1:
                raise
            print(f"retry {url}: {e}", file=sys.stderr)


def main():
    p = argparse.ArgumentParser()
    p.add_argument("har")
    p.add_argument("-o", "--output", default="output.ts")
    p.add_argument("-j", "--jobs", type=int, default=8)
    a = p.parse_args()

    urls = find_chunk_urls(a.har)
    if not urls:
        sys.exit("No /chunk/ URLs found")
    print(f"{len(urls)} chunks")
    with open(a.output, "wb") as out, ThreadPoolExecutor(a.jobs) as ex:
        for i, data in enumerate(ex.map(fetch, urls), 1):
            out.write(data)
            print(f"\r{i}/{len(urls)}", end="", flush=True)
    print(f"\nWrote {a.output}")


if __name__ == "__main__":
    main()

And it works. (End of story)

The resulting MPEG file can be played/viewed using ffplay, other tools have some sync issues.

posted at: 08:32 | path: / | permanent link

Fri, 28 Aug 2026

Fixed in clang-23

The x64 vectorizer issue has been fixed in clang-23.

posted at: 19:35 | path: /programming | permanent link

Wed, 12 Aug 2026

Vectorizer breaks encryption since clang-20

Below code should output 00000000fc52127e31b9685a7fc3bdda302494879630be65bffa2dc38b680ac9 with -O2; clang-20, clang-21, clang-22 produce different output on x64 (see Issue 215690).

// x.cpp
#include <cstdint>
#include <cstdio>
#include <cstdlib>

__attribute__((noinline)) void bar(uint8_t *b, size_t size)
{
        const uint32_t k = 0xc93e5407;
        for (uint32_t i = 0; i < size; i++)
        {
                b[i] ^= ((uint8_t*)&k)[i % sizeof(uint32_t)];
        }
}

int main() {
        uint8_t y[] = "\x07\x54\x3e\xc9\xfb\x06\x2c\xb7\x36\xed\x56\x93\x78\x97\x83\x13\x37\x70\xaa\x4e\x91\x64\x80\xac\xb8\xae\x13\x0a\x8c\x3c\x34";
        const size_t n = sizeof(y);
        bar(y, n);
        for (size_t i = 0; i < n; i++) printf("%02x", y[i]); puts("");
}

Just compile with clang++-22 -O2 x.cpp on x64; clang 19.1.7 (20ubuntu4) is OK, clang 22.1.2 (1ubuntu1) is NG.

-opt-bisect-limit indicates that loop-vectorize breaks things (see Using -opt-bisect-limit): compile with clang++-22 -O2 -mllvm -opt-bisect-limit=87 -c x.cpp to see/bisect which optimization step breaks the output.

The __attribute__((noinline)) is helpful to actually reproduce the issue in a single file; otherwise different compilation units do the thing.

posted at: 01:03 | path: /programming | permanent link

Mon, 27 Jul 2026

Linux Proton, games finally running

Struggled to get Cascadia and Expedition 33 running on Steam under Linux.

I noticed the following output when running (a very recent) wine:

GL_INVALID_ENUM in glTexBufferRange(internalFormat GL_RGBA8_SNORM

and the fact that the OpenGL renderer was used and it might not implement certain texture formats (I'm using a AMD Ryzen 5600G with its internal GPU).

After installing mesa-vulkan-drivers (which was apparently missing for whatever reason), things switch to the Vulkan renderer and started working. Hurray!

fo:  Game: SandFall-Win64-Shipping.exe
info:  DXVK: v3.0.2
info:  Build: x86_64 gcc 16.1.0
info:  Vulkan: Found vkGetInstanceProcAddr in winevulkan.dll @ 0x6ffff9007840
...
info:  Found device: AMD Radeon Graphics (RADV RENOIR) (radv 26.1.5)
info:  Found device: llvmpipe (LLVM 21.1.8, 256 bits) (llvmpipe 26.1.5)

posted at: 23:46 | path: /configuration | permanent link

Fri, 23 Oct 2026

Store video from .har file