UCD.js Docs

Pipeline Presets

Architecture notes for @ucdjs/pipeline-presets

@ucdjs/pipeline-presets provides reusable parsers, routes, resolvers, transforms, and ready-made pipelines for Unicode data processing.

Role

  • Encapsulates common UCD parsing strategies and route definitions.
  • Lets authors build useful pipelines quickly without recreating the standard building blocks.
  • Bridges raw Unicode text formats and the generic pipeline DSL.

Mental Model

The package is a toolbox with three levels:

  • low-level parsers and resolvers
  • reusable route definitions and transforms
  • complete preset pipelines like createFullPipeline()

It exists so most pipeline authors can compose, not invent.

Method Flows

createFullPipeline()

definePipeline()allRoutescreatePropertyJsonResolver()createFullPipeline()definePipeline()allRoutescreatePropertyJsonResolver()createFullPipeline()AuthorcreateFullPipeline({ versions, strict, concurrency })createPropertyJsonResolver()include common preset routesdefinePipeline({ include: byExt('.txt'), fallback, routes })PipelineDefinitionfull UCD pipelineAuthor

Standard route processing

propertyJsonResolver()/groupedResolver()standardParser()/unicodeDataParser()preset routepropertyJsonResolver()/groupedResolver()standardParser()/unicodeDataParser()preset routeExecutorexecute matched fileparse raw UCD fileParsedRow streamresolve rows into property JSONnormalized output entriesExecutor

Transform composition

pipeline-executorroute definitioncreateRowFilter()pipeline-executorroute definitioncreateRowFilter()Authorconfigure row filtering transformPipelineTransformDefinitionattach transforms to routerun parser output through transformstransformed rows for resolverAuthor

Design Notes

  • Presets should encode common Unicode patterns, not every possible pipeline style.
  • Ready-made pipelines like createFullPipeline() are intentionally opinionated.
  • The package keeps raw-format knowledge close to the parser/resolver layer instead of scattering it across apps.
  • Authors can always drop down to pipeline-core when presets are too constraining.

Testing Use

  • parser correctness for common UCD file formats
  • resolver output normalization
  • ready-made pipeline assembly
  • transform behavior in preset routes
  • compatibility with executor and playground example pipelines

On this page