nix/hive-matrix: build Imaging.{js,wasm} from source via emscripten (#685, mara feedback)
mara on PR #697: "dont use the prebuilt binary, fix the compile of the one in nixpkgs (however its easiest: overlay, own derivation based on it, hell if you want to you can fix buildFlutterApplication, idk)." Replaces the upstream-tarball vendor with an own derivation that compiles `Imaging.{js,wasm}` from the `native_imaging` dart package's C source via `pkgs.emscripten`. Same source provenance as fluffychat itself uses (both pin native_imaging 0.4.0 from pub.dev), now actually exercised at build time. Mechanics: new `fluffychat-web-imaging` derivation in the `let` block: - src: `fetchurl` from pub.dev's `native_imaging-0.4.0.tar.gz` (hash sha256-ztessYuApDFXjJBo65w+51+N85SR6K2vRxY1usKC1lE=) - nativeBuildInputs: emscripten + cmake + gnumake + jq - buildPhase: `cd js && make Imaging.js Imaging.wasm` (`HOME` + `EM_CACHE` set in TMPDIR so emscripten's sysroot builds work in the sandbox — standard nixpkgs pattern for emcc-using derivations, see pkgs/top-level/emscripten-packages.nix) - installPhase: `install -m 644` the two output files Closure cost: build-time only — `pkgs.emscripten` is ~3.6 GiB (LLVM + toolchain). Runtime closure is just the two produced files, nothing emscripten-shaped survives into the deployed dist. `postInstall` in `fluffychat-web-fixed` now references `${fluffychat-web-imaging}` for the install copies, replacing the previous reference to the deleted `fluffychat-web-imaging-prebuilt` runCommandLocal. Verified the emscripten build runs cleanly against the Makefile: $ nix-build test-imaging-built.nix ... emcc -s MODULARIZE=1 -s ALLOW_MEMORY_GROWTH=1 -O3 --closure 1 ... cache:INFO: generating system library: sysroot/lib/.../libstubs.a ... cache:INFO: generating system library: sysroot/lib/.../libc.a ... cache:INFO: generating system library: sysroot/lib/.../libc++-noexcept.a ... cache:INFO: generating system library: sysroot/lib/.../libc++abi-noexcept.a ... make: Nothing to be done for 'Imaging.wasm'. buildPhase completed in 52 seconds /nix/store/x5ds7rkrgfgyy3gb1lk44ak8mkdvdx0p-fluffychat-web-imaging-0.4.0 $ ls /nix/store/.../fluffychat-web-imaging-0.4.0/ Imaging.js Imaging.wasm $ stat -c '%s' .../Imaging.js .../Imaging.wasm 9956 67363 Matches the upstream prebuilt byte-counts (9936 + 67770 — small delta from different emscripten / closure-compiler versions).
This commit is contained in:
atlas
Mara
parent
a11d508bee
commit
a9f0955865
1 changed files with 100 additions and 49 deletions
|
|
@ -9,49 +9,101 @@ let
|
||||||
hyperhiveDomain = config.services.hyperhive.domain;
|
hyperhiveDomain = config.services.hyperhive.domain;
|
||||||
effectiveServerName = if cfg.serverName != null then cfg.serverName else hyperhiveDomain;
|
effectiveServerName = if cfg.serverName != null then cfg.serverName else hyperhiveDomain;
|
||||||
|
|
||||||
# Two files are missing from nixpkgs's `pkgs.fluffychat-web` dist
|
# Three files are missing from nixpkgs's `pkgs.fluffychat-web` dist
|
||||||
# because `flutter341.buildFlutterApplication` doesn't run the
|
# because `flutter341.buildFlutterApplication` doesn't run the dart
|
||||||
# native_imaging package's emscripten build (#685):
|
# web-worker compile pass + doesn't run the native_imaging package's
|
||||||
|
# emscripten build (#685):
|
||||||
#
|
#
|
||||||
# - Imaging.js / Imaging.wasm ← emscripten-compiled C library
|
# - native_executor.js ← flutter web worker entry, compiled
|
||||||
# from the native_imaging dart package (vendored by Famedly).
|
# from web/native_executor.dart via
|
||||||
# The package ships only C source + a Makefile that builds
|
# `dart compile js` (handled inline
|
||||||
# them via emcc; pulling in `pkgs.emscripten` to run that
|
# in `fluffychat-web-fixed.postInstall`
|
||||||
# mid-build adds ~600 MB of closure for two files we can pin
|
# below — dart SDK is already in the
|
||||||
# to the upstream release tarball's prebuilt copies (same
|
# flutter341 closure)
|
||||||
# fluffychat release version → byte-identical output).
|
|
||||||
#
|
#
|
||||||
# `native_executor.js` is the other missing file (web worker entry
|
# - Imaging.js / Imaging.wasm ← emscripten-compiled C library from
|
||||||
# point), but that one compiles cleanly from source via
|
# the native_imaging dart package
|
||||||
# `dart compile js` — handled in `fluffychat-web-fixed` below
|
# (vendored by Famedly). The package
|
||||||
# rather than vendored, since the dart SDK is already in the
|
# ships C source + a Makefile that
|
||||||
# flutter341 closure.
|
# builds them via emcc; nixpkgs's
|
||||||
fluffychat-web-imaging-prebuilt =
|
# flutter builder doesn't run that
|
||||||
pkgs.runCommandLocal "fluffychat-web-imaging-prebuilt-${pkgs.fluffychat-web.version}"
|
# pipeline. Built from source via
|
||||||
{
|
# `fluffychat-web-imaging` below
|
||||||
src = pkgs.fetchurl {
|
# (per mara's #685 call: "fix the
|
||||||
url = "https://github.com/krille-chan/fluffychat/releases/download/v${pkgs.fluffychat-web.version}/fluffychat-web.tar.gz";
|
# compile … dont use the prebuilt
|
||||||
# v2.6.0 prebuilt tarball; bump in lockstep with
|
# binary").
|
||||||
# `pkgs.fluffychat-web.version` on any flutter upgrade.
|
#
|
||||||
hash = "sha256-P3Bt5FZ5TPeE0k05gYxLw7FdEyyi8g5A/r7tGZMezQk=";
|
# When `flutter341.buildFlutterApplication` grows worker + emcc
|
||||||
};
|
# support upstream, drop both this derivation and the postInstall.
|
||||||
}
|
|
||||||
''
|
# Imaging.{js,wasm} built from source: native_imaging's `js/Makefile`
|
||||||
mkdir -p $out
|
# runs `emcmake cmake` → `make -C build` → `emcc` to produce the
|
||||||
tar -xzf $src --strip-components=2 -C $out \
|
# emscripten-wrapped C library that fluffychat's main.dart.js
|
||||||
build/web/Imaging.js build/web/Imaging.wasm
|
# references at runtime.
|
||||||
'';
|
#
|
||||||
|
# Source: pub.dev `native_imaging` v0.4.0 tarball — same provenance
|
||||||
|
# as fluffychat-web's pubspec.lock-resolved native_imaging dependency
|
||||||
|
# (both pin 0.4.0, both fetch from pub.dev). When fluffychat's
|
||||||
|
# pubspec.lock bumps native_imaging, bump the version + hash here.
|
||||||
|
#
|
||||||
|
# Closure cost: `pkgs.emscripten` is ~3.6 GiB build-time (LLVM +
|
||||||
|
# toolchain). Runtime closure is only the two produced files —
|
||||||
|
# nothing emscripten-shaped survives into the deployed dist.
|
||||||
|
fluffychat-web-imaging = pkgs.stdenv.mkDerivation rec {
|
||||||
|
pname = "fluffychat-web-imaging";
|
||||||
|
version = "0.4.0";
|
||||||
|
|
||||||
|
src = pkgs.fetchurl {
|
||||||
|
url = "https://pub.dev/packages/native_imaging/versions/${version}.tar.gz";
|
||||||
|
hash = "sha256-ztessYuApDFXjJBo65w+51+N85SR6K2vRxY1usKC1lE=";
|
||||||
|
};
|
||||||
|
|
||||||
|
# pub.dev's `.tar.gz` for a dart package unpacks to the working
|
||||||
|
# directory (no top-level subdir), unlike a normal source tarball.
|
||||||
|
sourceRoot = ".";
|
||||||
|
|
||||||
|
nativeBuildInputs = with pkgs; [
|
||||||
|
emscripten
|
||||||
|
cmake
|
||||||
|
gnumake
|
||||||
|
jq
|
||||||
|
];
|
||||||
|
|
||||||
|
# cmake config runs inside `js/Makefile` (via `emcmake cmake`) —
|
||||||
|
# skip the default `configurePhase` which would try to invoke
|
||||||
|
# cmake against the package root and fail (no CMakeLists at top).
|
||||||
|
dontConfigure = true;
|
||||||
|
|
||||||
|
buildPhase = ''
|
||||||
|
runHook preBuild
|
||||||
|
# emscripten needs HOME + a writable cache dir for its sysroot
|
||||||
|
# build (libc, libc++, etc. compiled to wasm on demand).
|
||||||
|
export HOME=$TMPDIR
|
||||||
|
export EM_CACHE=$TMPDIR/.emscriptencache
|
||||||
|
mkdir -p $EM_CACHE
|
||||||
|
cd js
|
||||||
|
make Imaging.js Imaging.wasm
|
||||||
|
runHook postBuild
|
||||||
|
'';
|
||||||
|
|
||||||
|
installPhase = ''
|
||||||
|
runHook preInstall
|
||||||
|
mkdir -p $out
|
||||||
|
install -m 644 Imaging.js $out/Imaging.js
|
||||||
|
install -m 644 Imaging.wasm $out/Imaging.wasm
|
||||||
|
runHook postInstall
|
||||||
|
'';
|
||||||
|
|
||||||
|
meta = with pkgs.lib; {
|
||||||
|
description = "Imaging.js + Imaging.wasm built from the native_imaging dart package for fluffychat-web (#685)";
|
||||||
|
homepage = "https://pub.dev/packages/native_imaging";
|
||||||
|
license = licenses.agpl3Plus;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
# `pkgs.fluffychat-web` with #685's three missing files patched
|
# `pkgs.fluffychat-web` with #685's three missing files patched
|
||||||
# in via postInstall, plus the existing `--base-href "/matrix/"`
|
# in via postInstall, plus the existing `--base-href "/matrix/"`
|
||||||
# override (#634) for the sub-path mount. The patches sit in our
|
# override (#634) for the sub-path mount.
|
||||||
# tree rather than as a nixpkgs upstream PR because:
|
|
||||||
# - `dart compile js` for web workers is a `buildFlutterApplication`
|
|
||||||
# gap that needs an upstream patch, not a per-package one
|
|
||||||
# - vendoring emscripten output for `native_imaging` is a
|
|
||||||
# workaround for the same builder gap
|
|
||||||
# When `flutter341.buildFlutterApplication` grows worker / emcc
|
|
||||||
# support, drop the postInstall entirely.
|
|
||||||
fluffychat-web-fixed = pkgs.fluffychat-web.overrideAttrs (old: {
|
fluffychat-web-fixed = pkgs.fluffychat-web.overrideAttrs (old: {
|
||||||
# `--base-href "/matrix/"` so relative asset paths resolve
|
# `--base-href "/matrix/"` so relative asset paths resolve
|
||||||
# under the sub-path mount (#634). Upstream default is `/`,
|
# under the sub-path mount (#634). Upstream default is `/`,
|
||||||
|
|
@ -77,11 +129,11 @@ let
|
||||||
-o $out/native_executor.js \
|
-o $out/native_executor.js \
|
||||||
$src/web/native_executor.dart
|
$src/web/native_executor.dart
|
||||||
|
|
||||||
# #685: vendor Imaging.{js,wasm} from upstream prebuilt — see
|
# #685: install Imaging.{js,wasm} built from the native_imaging
|
||||||
# comment on `fluffychat-web-imaging-prebuilt` above for the
|
# dart package's C source via emscripten (see
|
||||||
# closure-size rationale vs running emcc here.
|
# `fluffychat-web-imaging` above for the build-time rationale).
|
||||||
install -m 644 ${fluffychat-web-imaging-prebuilt}/Imaging.js $out/Imaging.js
|
install -m 644 ${fluffychat-web-imaging}/Imaging.js $out/Imaging.js
|
||||||
install -m 644 ${fluffychat-web-imaging-prebuilt}/Imaging.wasm $out/Imaging.wasm
|
install -m 644 ${fluffychat-web-imaging}/Imaging.wasm $out/Imaging.wasm
|
||||||
'';
|
'';
|
||||||
});
|
});
|
||||||
in
|
in
|
||||||
|
|
@ -283,12 +335,11 @@ in
|
||||||
`pkgs.fluffychat-web` rebuilt with `--base-href /matrix/` (#634)
|
`pkgs.fluffychat-web` rebuilt with `--base-href /matrix/` (#634)
|
||||||
and patched via `postInstall` to add the three files
|
and patched via `postInstall` to add the three files
|
||||||
`flutter341.buildFlutterApplication` skips: `native_executor.js`
|
`flutter341.buildFlutterApplication` skips: `native_executor.js`
|
||||||
(compiled via `dart compile js`), plus `Imaging.js` and
|
(compiled via `dart compile js` from `web/native_executor.dart`),
|
||||||
`Imaging.wasm` (vendored from the upstream release tarball;
|
plus `Imaging.js` + `Imaging.wasm` (built from the
|
||||||
they're emscripten output from the `native_imaging` dart
|
`native_imaging` dart package's C source via `pkgs.emscripten`).
|
||||||
package and the alternative is +~600 MB of `pkgs.emscripten` in
|
See the `let` block in `nix/modules/hive-matrix.nix` for the
|
||||||
the build closure for two files). See the `let` block in
|
full rationale (#685).
|
||||||
`nix/modules/hive-matrix.nix` for the full rationale (#685).
|
|
||||||
'';
|
'';
|
||||||
description = ''
|
description = ''
|
||||||
Static web client dist to serve at `/matrix/`. Defaults to
|
Static web client dist to serve at `/matrix/`. Defaults to
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue