Project Architecture

UCD.js is a monorepo that provides tools and APIs for working with Unicode Character Database (UCD) files.

Repository Structure

• packages/ - core libraries published to npm
• apps/ - applications (API worker, store, web, docs)
• tooling/ - internal development tools (eslint-plugin, tsconfig, tsdown-config, moonbeam)
• vscode/ - VS Code extension

Core Systems

• Storage and filesystem: @ucdjs/ucd-store, @ucdjs/fs-backend, @ucdjs/lockfile
• Pipelines: @ucdjs/pipeline-* (internal extraction/build path)
• API and Store: apps/api, apps/store
• CLI and Apps: @ucdjs/cli, apps/web, apps/docs
• Schemas and generation: @ucdjs/schemas, @ucdjs/codegen

High-Level Flow

UCD source files can be accessed either through the storage layer (fs-backend + ucd-store) or directly through the API. The API does not depend on the storage layer; it can serve files and metadata by calling upstream sources. Pipelines run internally to extract data and produce outputs/data packages. Consumers typically use the API (and data packages when available). The CLI can explore local files, call the API directly, and orchestrate local workflows.

UCD source files
  ↙           ↘
Storage + Store    API (direct upstream)
      ↓               ↓
   Local access    API consumers
      ↓
 Pipelines (internal build path)
      ↓
Outputs / Data Packages → Consumers (NPM packages)

Storage and Filesystem

The storage layer is built around @ucdjs/ucd-store and @ucdjs/fs-backend. fs-backend provides Node.js, HTTP, and custom backend patterns. ucd-store handles file access and metadata across those backends. Lockfile and snapshot utilities track integrity and state for stored data.

Use cases:

• Local access and cached workflows
• Store app access and store-aligned tooling
• CLI operations that work against local stores

Pipelines

Pipelines are an internal extraction/build path. They define how data is processed and outputs are produced. Pipelines are split into dedicated packages for core types, loading, execution, graph ordering, presets, and the Pipeline Server UI.

Key packages:

• @ucdjs/pipeline-core
• @ucdjs/pipeline-loader
• @ucdjs/pipeline-executor
• @ucdjs/pipeline-graph
• @ucdjs/pipeline-presets
• @ucdjs/pipeline-server

Most consumers do not run pipelines directly.

API and Store

The API (apps/api) is a Cloudflare Worker using Hono with OpenAPI. It serves versioned routes for files, versions, and schemas, and can fetch data directly from upstream sources.

The Store (apps/store) provides direct access to UCD files and metadata via the storage layer. Both are public access paths and should remain stable.

File Path Boundary

API and Store do not expose the same file path shape.

• The API file surface maps directly to the upstream Unicode.org layout. A request for /api/v1/files/17.0.0/... resolves against https://unicode.org/Public/17.0.0/....
• Store is the HTTP file-store boundary used by @ucdjs/fs-backend, and it exposes store-style paths as /{version}/{filepath}.
• For versions that have a ucd/ directory, Store inserts that segment when resolving upstream paths.
• For older versions that do not have a ucd/ directory, Store resolves the same store path directly under the version root.

The point of Store is to own this normalization so the storage-facing path shape stays stable even though the upstream Unicode.org layout is not.

CLI and Apps

@ucdjs/cli is the primary local interface. It can explore files, run pipelines, and call the API directly. apps/web and apps/store are end-user interfaces, while apps/docs is the documentation site.

Example (API access via CLI):

./packages/cli/bin/ucd.js files get 17.0.0/ArabicShaping.txt

Schemas and Generation

@ucdjs/schemas provides Zod schemas for Unicode data files, and @ucdjs/codegen generates TypeScript field definitions from raw Unicode data using AI. The OpenAPI spec is generated from Zod schemas and drives @ucdjs/client.

Tooling and Development Utilities

Internal tooling lives under tooling/ and includes shared configs and tsdown helpers. Moonbeam (@ucdjs/moonbeam) is used when running scripts that need workspace source resolution without building packages first.

Documentation

• https://docs.ucdjs.dev
• apps/docs (source)