flod

High-level interface for flod. Provides the most commonly used functions of the package.

More...

Modules

adapter
module flod.adapter

Adapters connecting stages with incompatible interfaces.

buffer
module flod.buffer

Various buffer implementations.

file
module flod.file

File I/O pipes.

meta
module flod.meta

Various metaprogramming helpers.

metadata
module flod.metadata

Pass metadata alongside the data stream.

pipeline
module flod.pipeline

Pipeline composition.

range
module flod.range

This module contains pipes used to convert between ranges and pipelines.

traits
module flod.traits

Templates for declaring and examining static interfaces of pipeline stages.

utils
module flod.utils

Types and functions for managing creation and copying of objects.

Public Imports

flod.range
public import flod.range : copy, pass, byLine, byChunk;
flod.file
public import flod.file : read, write;

Members

Functions

array
auto array(S schema)

A sink that reads the entire stream and stores its contents in a GC-allocated array.

discard
void discard(S schema)

A sink that discards all data written to it.

take
auto take(S schema, ulong n)

Lazily takes only up to n elements of a stream.

Detailed Description

Package description

flod is a library for processing streams of data using composable building blocks.

In flod, a pipeline is a chain of one or more stages, which communicate with each other using the specified interfaces.

A stage can be either a source, a sink, or a filter, which is both a sink and a source. A source-only stage is one that only produces data, e.g. reads a file or generates random bytes. A sink-only stage is one that only consumes data from previous stages. For example, a sink may use the data to build an array or replay audio samples. A filter stage receives data on its sink end, and produces transformed data on its source end. Examples of such stages are media decoders or cipher implementations.

There are four methods of passing data from a source to a sink:

methoddescriptionbuffer is owned by
pullsink calls source.pull()sink
pushsource calls sink.push()source
peeksink calls source.peek() and source.consume()source
allocsource calls sink.alloc() and sink.commit()sink

Note that a filter can use different methods for its sink and source ends. For example, it may call peek and consume to access the input data, and output transformed data using push.

Meta

License

BSL-1.0.