dsteem Modernization Plan -Phase 5

blaze.apps (71)verified developerin Steem Dev • 15 hours ago (edited)

Day 5: I ripped out the entire 2018-era build stack — browserify, tsify, babelify, uglify-js, dts-generator, watchify, derequire — and replaced it with one config file: tsup (esbuild under the hood). Along with core-js@2, regenerator-runtime, whatwg-fetch, node-fetch, and the rest of the dead polyfills, that's 606 npm packages removed from node_modules. The browser bundle drops from 782KB to 351KB (-55%). The Makefile (which assumed a BSD sed toolchain) is gone, replaced by plain npm run build that works on Windows, Mac, and Linux. This is the biggest scope-by-line-count phase in the migration.

Day 1 — Phase 0: dsteem Modernization Plan
Day 2 — Phase 1: CI Plumbing on GitHub Actions
Day 3 — Phase 2: Retiring tslint, Adopting ESLint 9
Day 4 — Phase 3: TypeScript 3.1 → 5.6 + Strict Mode
Day 5 — Phase 4: The @noble Crypto Swap
Today — Phase 5: tsup, Dual ESM/CJS, and 606 Packages Deleted (you are here)
Tomorrow — Phase 6: Mocha 11, c8 coverage, Playwright browser tests
Day 8 — Phase 7: typedoc 0.13 → 0.28
Next week — Phase 8: Cleanup, README refresh, and the v0.12.0 release

What Died Today

The 2018-era browserify + babel + uglify pipeline was a marvel of duct tape. Here's what got uninstalled in one npm uninstall command:

browserify              ← bundler (replaced by esbuild)
tsify                   ← browserify TS transform
babelify                ← browserify babel transform
@babel/core             ← transpiler core
@babel/preset-env       ← target-based polyfill injection
derequire               ← strip require() calls for browser
dts-generator           ← bundle .d.ts files (replaced by tsup --dts)
uglify-js               ← minifier (replaced by esbuild's minifier)
watchify                ← browserify watch mode
encoding                ← optional dep for node-fetch
core-js                 ← v2 polyfill suite (deprecated since 2019)
regenerator-runtime     ← async/await polyfill (Node 8 era)
whatwg-fetch            ← fetch polyfill for IE/Edge Legacy (both dead)
node-fetch              ← fetch polyfill for Node <18 (we're on 22+)
karma                   ← browser test runner (replaced by Playwright)
karma-browserify        ← karma+browserify glue
karma-mocha             ← karma+mocha glue
karma-mocha-reporter    ← karma test output
karma-sauce-launcher    ← Sauce Labs integration

The output:

removed 606 packages in 3s

606 packages. Six hundred and six. That's not a typo. The transitive dependency graph of browserify + karma + babel alone pulled in three orders of magnitude more code than the actual dsteem library. Removing them shrinks node_modules by an order of magnitude.

Enter `tsup`

tsup is a zero-config wrapper around esbuild that handles the three things this project needs:

Bundle source TypeScript into single-file outputs (ESM + CJS + IIFE)
Generate .d.ts declarations (via TypeScript's built-in --declaration)
Apply source maps + minification consistently

The full config file:

// tsup.config.ts
import {defineConfig} from 'tsup'
import {polyfillNode} from 'esbuild-plugin-polyfill-node'
import {readFileSync} from 'fs'

const pkg = JSON.parse(readFileSync('./package.json', 'utf8'))
const banner = `/*! dsteem v${pkg.version} | BSD-3-Clause | https://github.com/jnordberg/dsteem */`

export default defineConfig([
    // ──── Node build — dual ESM + CJS + .d.ts ─────────────
    {
        entry: {index: 'src/index-node.ts'},
        outDir: 'dist',
        format: ['esm', 'cjs'],
        target: 'node22',
        platform: 'node',
        dts: true,
        sourcemap: true,
        clean: true,
        treeshake: true,
        noExternal: ['verror', 'bs58', 'bytebuffer'],
        define: {__PACKAGE_VERSION__: JSON.stringify(pkg.version)},
        banner: ({format}) => format === 'esm'
            ? {js: `${banner}\nimport {createRequire as __dsteemCreateRequire} from 'module';\nconst require = __dsteemCreateRequire(import.meta.url);`}
            : {js: banner},
        outExtension({format}) {
            return {js: format === 'cjs' ? '.cjs' : '.mjs'}
        },
        esbuildOptions(options) {
            options.alias = {bytebuffer: 'bytebuffer/dist/bytebuffer.js'}
        }
    },
    // ──── Browser IIFE bundle ──────────────────────────────
    {
        entry: {'dsteem.browser': 'src/index-browser.ts'},
        outDir: 'dist',
        format: ['iife'],
        globalName: 'dsteem',
        target: 'es2020',
        platform: 'browser',
        sourcemap: true,
        minify: true,
        define: {
            __PACKAGE_VERSION__: JSON.stringify(pkg.version),
            global: 'globalThis'
        },
        banner: {js: banner},
        esbuildPlugins: [polyfillNode({polyfills: {crypto: true}})],
        noExternal: [/.*/]
    }
])

That's the whole build system. Compare to the previous Makefile + multiple package.json browserify configs + dts-generator config + uglifyjs flags — replaced by 50 lines of TypeScript.

The Three Interop Headaches (and How I Solved Them)

A clean dual ESM/CJS build sounds simple. It is not. Here are the three real problems I hit and what fixed each one.

1. `verror`'s named export breaks ESM

First build, first smoke test:

file:///A:/dsteem/dist/index.mjs:8
import { VError } from 'verror';
         ^^^^^^
SyntaxError: Named export 'VError' not found.
The requested module 'verror' is a CommonJS module...

verror is CommonJS-only and doesn't have proper named exports. The ESM consumer can't pluck VError off it directly.

Fix: bundle verror (and bs58 and bytebuffer) into the ESM output via noExternal. esbuild handles the CJS→ESM interop internally, and the consumer just sees normal ESM imports.

2. `bytebuffer`'s Node entry uses dynamic `require('buffer')`

After fixing #1, the round-trip smoke test exploded with:

Error: Dynamic require of "buffer" is not supported
    at file:///A:/dsteem/dist/index.mjs:18:9
    at node_modules/bytebuffer/dist/bytebuffer-node.js

bytebuffer@5 ships two entry points: bytebuffer.js (browser-safe) and bytebuffer-node.js. The Node entry does Buffer = require('buffer').Buffer at runtime to alias the global Buffer. esbuild bundles that require() as a synchronous shim — and ESM doesn't allow synchronous require().

Fix: alias bytebuffer to the browser entry, which doesn't need the dynamic require. It works on Node too because Buffer is a global in both ESM and CJS:

esbuildOptions(options) {
    options.alias = {bytebuffer: 'bytebuffer/dist/bytebuffer.js'}
}

3. `safe-buffer` (transitive) still does `require('buffer')` in the ESM output

Even after bytebuffer was aliased, safe-buffer (pulled in by verror) hit the same dynamic-require wall. This one's pervasive — safe-buffer is a transitive dep of dozens of common packages.

Fix: inject a createRequire shim banner at the top of the ESM output. Node's createRequire(import.meta.url) returns a working require function inside ESM code, so the bundled shim resolves to the real Node built-in module:

banner: ({format}) => format === 'esm'
    ? {js: `${banner}\nimport {createRequire as __dsteemCreateRequire} from 'module';\nconst require = __dsteemCreateRequire(import.meta.url);`}
    : {js: banner}

After this, the ESM build round-trips cleanly:

$ node --input-type=module -e "import {PrivateKey, cryptoUtils} from './dist/index.mjs'; ..."
Verify: true Recover matches: true

The New `package.json` `exports` Map

Replacing the old main + browser + typings triplet with a proper exports map gives modern bundlers and TypeScript everything they need to pick the right entry:

{
  "name": "dsteem",
  "version": "0.12.0",
  "main": "./dist/index.cjs",
  "module": "./dist/index.mjs",
  "types": "./dist/index.d.ts",
  "browser": "./dist/dsteem.browser.global.js",
  "exports": {
    ".": {
      "types": "./dist/index.d.ts",
      "browser": "./dist/dsteem.browser.global.js",
      "import": "./dist/index.mjs",
      "require": "./dist/index.cjs"
    },
    "./package.json": "./package.json"
  },
  "files": ["dist"],
  "engines": {
    "node": ">=22.0.0"
  }
}

Resolution outcomes:

Consumer	Resolves to
`import {Client} from 'dsteem'` (ESM, Node)	`dist/index.mjs`
`require('dsteem')` (CJS, Node)	`dist/index.cjs`
`import {Client} from 'dsteem'` (webpack/vite, browser target)	`dist/dsteem.browser.global.js`
TypeScript IntelliSense	`dist/index.d.ts`
Top-level `<script src="...dsteem.browser.global.js">`	global `window.dsteem`

All from one declaration. No more brittle browser: { "./lib/index-node": "./dist/dsteem.js" } redirect hacks.

Gutting the Polyfills

The legacy src/index-browser.ts was an 18-line polyfill shrine:

import 'regenerator-runtime/runtime'

if (global['navigator'] && /Edge/.test(global['navigator'].userAgent)) {
  delete global['fetch']  // Edge Legacy workaround
}

import 'core-js/es6/map'
import 'core-js/es6/number'
import 'core-js/es6/promise'
import 'core-js/es6/symbol'
import 'core-js/fn/array/from'
import 'core-js/modules/es7.symbol.async-iterator'
import 'whatwg-fetch'

export * from './index'

Every one of those is dead code in 2026:

regenerator-runtime: async/await is native in every browser since 2017 and every Node since 7.6
Edge Legacy: replaced by Chromium Edge in 2020. The user agent is gone from the field.
core-js/es6/*: Map, Number, Promise, Symbol, Array.from — native since IE Edge / Chrome 38 / Firefox 13 era
core-js/modules/es7.symbol.async-iterator: native since Chrome 63 / Firefox 57 / Node 10
whatwg-fetch: native since Chrome 42 / Firefox 39 / Safari 10.1 / Edge 14

The new file:

// Modern browsers all ship native fetch, Promise, Symbol, Map, async iterators,
// and async/await. The legacy polyfills the v0.11.x bundle pulled in are no
// longer required. Node built-ins (Buffer, etc.) are polyfilled at bundle time
// by esbuild-plugin-polyfill-node.

export * from './index'

Same story for src/index-node.ts: dropped the require('node-fetch') polyfill (Node 18+ has global fetch) and the core-js/modules/es7.symbol.async-iterator import. It's now also just export * from './index'.

The Version Stub, Rewritten

The legacy src/version.ts was a stub that the Makefile overwrote at build time:

// overwritten by buildscript
export default require('./../package.json').version as string

…and the Makefile had a 5-line VERSION_TEMPLATE that injected the real version into lib/version.js. Brittle, platform-specific (used BSD sed -i ''), and required a separate build step.

The new approach uses tsup's define:

// src/version.ts
declare const __PACKAGE_VERSION__: string

let version: string
try {
    version = __PACKAGE_VERSION__
} catch {
    version = require('./../package.json').version
}

export default version

// tsup.config.ts
define: {__PACKAGE_VERSION__: JSON.stringify(pkg.version)},

At build time, esbuild substitutes the literal version string in. At dev time (ts-node), the __PACKAGE_VERSION__ symbol isn't defined and the catch block reads package.json directly. Works everywhere, no Makefile, no build-script intervention.

The Phase 5 Build Result

$ npm run build

CLI Building entry: {"index":"src/index-node.ts"}
CLI Building entry: {"dsteem.browser":"src/index-browser.ts"}
CLI Target: node22
CLI Target: es2020
ESM Build start
CJS Build start
IIFE Build start
DTS Build start
CJS dist\index.cjs      224.50 KB
ESM dist\index.mjs      223.30 KB
IIFE dist\dsteem.browser.global.js   350.51 KB
DTS dist\index.d.ts     91.47 KB
DTS dist\index.d.mts    91.47 KB
✓ Build success

Artifact	Size	Down from
`dist/index.mjs`	223 KB	(didn't exist before)
`dist/index.cjs`	224 KB	(didn't exist before)
`dist/index.d.ts`	91 KB	106 KB (old dts-generator output)
`dist/dsteem.browser.global.js`	351 KB	782 KB (old browserify+uglify output)

The browser bundle is 55% smaller than the v0.11.3 ship. Most of the win comes from dropping core-js v2 (~150KB), regenerator-runtime (~50KB), whatwg-fetch (~30KB), and the native-secp256k1 fallback shim (~100KB). The remaining 350KB is mostly bytebuffer + the polyfilled Buffer from esbuild-plugin-polyfill-node.

The Smoke Tests

End-to-end checks on the built artifacts:

$ node -e "const m = require('./dist/index.cjs'); console.log('VERSION:', m.VERSION, 'exports:', Object.keys(m).length)"
VERSION: 0.12.0 exports: 21

$ node --input-type=module -e "import {PrivateKey, cryptoUtils} from './dist/index.mjs'; const k = PrivateKey.fromSeed('test'); const msg = cryptoUtils.sha256('hello'); const sig = k.sign(msg); console.log('Verify:', k.createPublic().verify(msg, sig));"
Verify: true

$ npm pack --dry-run
npm notice 359.0kB dist/dsteem.browser.global.js
npm notice 1.9MB dist/dsteem.browser.global.js.map
npm notice 229.4kB dist/index.cjs
npm notice 558.6kB dist/index.cjs.map
npm notice 93.7kB dist/index.d.mts
npm notice 93.7kB dist/index.d.ts
npm notice 228.4kB dist/index.mjs
npm notice 558.5kB dist/index.mjs.map
npm notice package size: 792.7 kB
npm notice total files: 11

✅ CJS smoke: VERSION reads correctly, 21 named exports (same as v0.11.x surface)
✅ ESM smoke: round-trip sign+verify works
✅ Tarball: 792.7 KB unpacked, ships only dist/ + LICENSE + README + package.json

And the test suite still passes:

$ npm test
  49 passing (424ms)

What Else Was Deleted

Makefile     ← gone (BSD-sed-dependent, replaced by npm scripts)
test/_node.js          ← gone (only existed to polyfill global.fetch)
test/_browser.js       ← gone (karma scaffolding)
test/_karma.js         ← gone
test/_karma-sauce.js   ← gone

The Makefile was the single biggest cross-platform pain point — sed -i '' is BSD syntax that GNU sed (Linux/Windows) doesn't accept, so the original Makefile only worked on macOS. Replacing it with portable npm run * scripts means the build now works on Mac, Linux, and Windows without modification.

The Meta-Lesson

The 2018-era JS build stack was an empire of complexity. Three concerns — bundling, transpiling, minifying — got fragmented across browserify + babelify + uglifyjs, each with their own configs and integration glue (tsify, derequire, watchify). Plus a separate tool (dts-generator) for declaration bundling. Plus a Makefile to orchestrate them all.

esbuild — and tsup on top of it — consolidates the same three concerns into one tool with one config. The cognitive load drop is enormous: I have one file to reason about, one set of options to learn, one performance profile to measure. Build times went from "make a coffee" to 2 seconds.

The lesson generalizes: when you find yourself maintaining glue code between three tools that overlap 70% of their concerns, the right move is usually to find a single tool that does all three. The cost of one richer tool is almost always less than the integration tax of three narrow ones.

What's Next: Phase 6

Tomorrow I finish the test infrastructure modernization: mocha 5 → 11, nyc → c8 with a 70% line-coverage gate, Playwright for browser smoke tests against the actual bundled dist/dsteem.browser.global.js, and finally — finally — fix that 6-year-old broken inspect test by switching to [util.inspect.custom]. Plus network-test gates so the live api.steemit.com + api.moecki.online calls don't make CI flaky.

How You Can Help

🔍 Watch the repo: blazeapps007/dsteem
🐛 Spot regressions: API compatibility with v0.11.x is non-negotiable
💬 Comment: questions about specific phase choices help shape the rest of the series

This work was developed with Claude AI assistance. All technical decisions reflect Steem ecosystem needs and the hard requirement of zero breaking changes .

Support Secure Steem Development

If you value proactive engineering, UX polish, and performance optimizations for the STEEM ecosystem, please consider supporting my witness: blaze.apps

🗳️ Vote Here:
Vote for blaze.apps Witness

Disclaimer: This post describes work in progress on the blazeapps007/dsteem fork. The release will be tagged v0.12.0 after Phase 8 ships next week.

#steem #steemdev #dsteem #opensource #witnesscategory

15 hours ago in Steem Dev by blaze.apps (71)verified developer

$14.86