Skip to main content
Version: Next

from

Operator

from source data from pools, files, or URIs

Synopsis

from <pool>[@<tag>] [range <start>] [to <end>] [ => <leg> ]
file <path> [format <format>]
get <uri> [format <format>]
from (
pool <pool>[@<tag>] [range <start>] [to <end>] [ => <leg> ]
file <path> [format <format>] [ => <leg> ]
get <uri> [format <format>] [ => <leg> ]
...
)

Description

The from operator identifies one or more data sources and transmits their data to its output. A data source can be

  • the name of a data pool in a Zed lake;
  • a path to a file; or
  • an HTTP, HTTPS, or S3 URI. Paths and URIs may be followed by an optional format specifier.

In the first three forms, a single source is connected to a single output. In the fourth form, multiple sources are accessed in parallel and may be joined, combined, or merged.

A data path can be split with the fork operator as in

from PoolOne | fork (
=> op1 | op2 | ...
=> op1 | op2 | ...
) | merge ts | ...

Or multiple pools can be accessed and, for example, joined:

from (
pool PoolOne => op1 | op2 | ...
pool PoolTwo => op1 | op2 | ...
) | join on key=key | ...

Similarly, data can be routed to different paths with replication using switch:

from ... | switch color (
case "red" => op1 | op2 | ...
case "blue" => op1 | op2 | ...
default => op1 | op2 | ...
) | ...

The output of a fork consists of multiple legs that must be merged. If the downstream operator expects a single input, then the output legs are merged with an automatically inserted combine operator.

Examples

Copy input to two paths and merge

echo '1 2' | zq -z 'fork (=>pass =>pass) | sort this' -

=>

1
1
2
2