Filters

Filters are predicates used to decide whether a source or route should process a file or row. In practice, you will most often use them in:

• source.includes
• source.excludes
• route.filter

All built-in helpers live in @ucdjs/pipeline-core.

Common file filters

byName(...)

Match a specific file name:

filter: byName("UnicodeData.txt")

byDir(...)

Match a logical directory:

filter: byDir("ucd")

byExt(...)

Match by file extension:

filter: byExt(".txt")

byGlob(...)

Match a path with a glob:

filter: byGlob("**/*.txt")

byPath(...)

Match an exact path or a regular expression:

filter: byPath("data/colors.txt")

filter: byPath(/^emoji\/.*\.txt$/)

bySource(...)

Match files from one or more source IDs:

filter: bySource("unicode-org")

filter: bySource(["unicode-org", "cldr"])

Row-aware filter helper

byProp(...)

Match a parsed row property name with a string or regular expression:

filter: byProp("Script")

filter: byProp(/^Emoji/)

This is most useful in row-aware filtering contexts and composition helpers.

Filter combinators

and(...)

All filters must match:

filter: and(
  bySource("unicode-org"),
  byExt(".txt"),
)

or(...)

Any filter may match:

filter: or(
  byName("Blocks.txt"),
  byName("Scripts.txt"),
)

not(...)

Invert a filter:

filter: not(byGlob("**/generated/**"))

always() and never()

Utility filters for explicit allow-all and match-nothing behavior:

includes: always()

excludes: never()

Source-level vs route-level filtering

Use source filters when you want to broadly shape the files a source contributes.

Use route filters when you want a specific route to match a specific subset of files from the already-resolved source set.

Custom filters

You can always write a custom filter function:

filter: (ctx) => {
  return ctx.file.name.endsWith(".txt") && ctx.file.path.startsWith("data/");
}

Custom filters work well, but built-in helpers have one extra benefit: the system can derive a readable filter description for inspection UIs and diagnostics.