Skip to content

Expressions

An Expr is one node of a query expression tree: a description of a computation, not a value. Expressions are built from col and lit, combined with operators and methods, and evaluated only when the enclosing query collects. Every method returns a new Expr — nothing mutates.

Building blocks

col(name)            # a reference to a column
lit(value)           # an int, float, bool, or str literal
time_lit(ns)         # a Time literal from nanoseconds since midnight
corr(a, b)           # Pearson correlation aggregate over a group
if_else(cond, a, b)  # lane-wise choice on a boolean condition
when(cond).then(a)...otherwise(b)  # multi-branch lane-wise choice
coalesce(exprs)      # first non-null lane across a list of expressions

One convention decides what a bare Python string means, and it is worth learning once. Where a function accepts expressions — select, with_columns, group_by, agg, over — a string is a column name. Where a function accepts a value — filter, lit, if_else, corr, and every operator operand — a string is a string literal. When in doubt, write col("name"); it is never ambiguous.

import keyten as kt

df = kt.DataFrame([
    kt.Series.string("sym", ["ev", "od", "ev", "od"]),
    kt.Series.int("qty", [1, 2, 3, 4]),
])

# "sym" below is a column name; "ev" is a string literal.
out = df.lazy().filter(kt.col("sym") == "ev").select("sym").collect()
assert out.shape == (2, 1)

Operators

Arithmetic (+, -, *, /, %), comparisons (==, !=, <, <=, >, >=), and logic (&, |, ~) all build expressions; the other operand may be an Expr or a plain literal, on either side. Comparisons return an Expr, not a bool — which is why bool(expr) raises by design, and Python's and/or/not cannot be used. Write &, |, and ~ (or not_() for the latter), and parenthesize around comparisons ((a > 1) & (b < 2)), because Python binds & tighter than >. - also works unary (-col("qty") negates) and abs() takes the absolute value; both propagate null like the rest of arithmetic.

Math

log()  log1p()  exp()  sqrt()  sign()
round()  floor()  ceil()
pow(exp)          # Expr ** exp; also spelled **
clip(lo, hi)

log, log1p, exp, and sqrt promote Int to Float and follow IEEE domain semantics rather than raising — log(0.0) == -inf, and log(-1.0) and sqrt(-1.0) are NaN. sign returns -1/0/1 and keeps the column's kind, Int staying Int and Float staying Float (sign(NaN) == NaN, sign(+-0.0) == 0.0). round (half away from zero), floor, and ceil only touch Float — an Int column passes through them unchanged. pow, also spelled **, is always Float even over an Int column, since a negative or fractional exponent can't stay Int; the exponent must be an int or float literal, and a modulo argument to ** raises TypeError (unlike Python's three-argument pow). clip(lo, hi) clamps into [lo, hi] and keeps the column's kind (Int column with Int bounds stays Int; anything Float promotes); lo > hi raises at build time, and NaN passes through unclamped, same as the rest of arithmetic.

import math
import keyten as kt

df = kt.DataFrame([kt.Series.float("px", [100.0, 101.0, 99.0])])
out = df.lazy().with_columns(
    (kt.col("px").log() - kt.col("px").log().shift(1)).alias("ret")
).collect().to_dict()
assert out["ret"][0] is None
assert abs(out["ret"][1] - math.log(101.0 / 100.0)) < 1e-12

import keyten as kt

df = kt.DataFrame([kt.Series.int("qty", [1, 5, 9])])
out = df.lazy().with_columns([
    (kt.col("qty") ** 2).alias("sq"),
    kt.col("qty").clip(2, 8).alias("cl"),
]).collect().to_dict()
assert out["sq"] == [1.0, 25.0, 81.0]
assert out["cl"] == [2, 5, 8]

Temporal literals

Anywhere an expression accepts a literal — comparisons, lit, fill_null, is_in — a plain datetime.date, datetime.datetime, or datetime.time value works directly, compared against date, timestamp, and time columns respectively. Timezone-aware values raise TypeError: Keyten's temporals are naive, so there is no offset to compare against and the engine refuses to guess.

import keyten as kt
from datetime import date

df = kt.DataFrame([kt.Series.date("d", [date(2026, 1, 5), date(2026, 1, 6)])])
out = df.lazy().filter(kt.col("d") == date(2026, 1, 6)).collect()
assert out.shape == (1, 1)

import keyten as kt

df = kt.DataFrame([kt.Series.int("qty", [1, 2, 3, 4])])
out = df.lazy().filter((kt.col("qty") > 1) & (kt.col("qty") < 4)).collect()
assert out.shape == (2, 1)

Datetime components

year()  month()  day()
hour()  minute()  second()
weekday()      # ISO: Monday=1..Sunday=7
week()         # ISO-8601 week number, 1-53
quarter()      # 1-4
ordinal_day()  # day of year, 1-366

Each pulls one field out of a Date, Timestamp, or Time value, and each is legal only on the kinds that actually carry that field: Date has no time of day, so hour, minute, and second on a Date column raise a kind error; Time has no calendar, so year, month, day, weekday, week, quarter, and ordinal_day on a Time column raise the same way. Timestamp carries both and accepts everything. All return Int.

import keyten as kt
from datetime import datetime

df = kt.DataFrame([kt.Series.timestamp("ts", [
    datetime(2026, 1, 5, 8, 0),
    datetime(2026, 1, 5, 10, 30),
    datetime(2026, 1, 5, 16, 30),
])])
open_hours = df.lazy().filter(
    (kt.col("ts").hour() >= 9) & (kt.col("ts").hour() < 16)
).collect()
assert open_hours.shape == (1, 1)

Truncating time

truncate(every)   # floor to a multiple of a duration

every is <positive-int><unit> with unit one of ns/us/ms/s/m/h/d (e.g. "5m", "1h") — any other form raises ValueError at build time. It floors: pre-epoch values truncate toward -inf, not toward zero. Timestamp accepts any unit; Date requires a whole multiple of a day; Time requires less than 24h. LazyFrame.group_by_bar(time_col, every) fuses this with group_by in one call — see LazyFrame and GroupBy.

import keyten as kt
from datetime import datetime

df = kt.DataFrame([kt.Series.timestamp("ts", [
    datetime(2026, 1, 5, 9, 30, 5),
    datetime(2026, 1, 5, 9, 30, 45),
    datetime(2026, 1, 5, 9, 31, 10),
])])
bars = df.lazy().with_columns(kt.col("ts").truncate("1m").alias("bar")).collect().to_dict()["bar"]
assert bars[0] == bars[1] != bars[2]

Naming and casting

alias(name)   # name the output column
cast(kind)    # "int", "float", "bool", "str", "date", "timestamp", "time"

An expression without a natural name — arithmetic, corr, if_else — must be given one with alias before it is used in select, with_columns, or agg. cast validates its dtype string immediately (TypeError at build time, not at collect); "i64", "f64", and "string" are accepted aliases, case-insensitive.

Aggregations

sum()  min()  max()  mean()  count()  first()  last()
std()  var()  median()  quantile(q)  n_unique()
corr(a, b)          # module-level: a PAIR aggregate

Aggregations are legal at the root of a group_by(...).agg(...) entry, or in a select where every entry is aggregate-rooted (a whole-frame aggregation). count counts valid values; std is the sample deviation and var the sample variance (ddof = 1; var() == std() ** 2, same underlying sums, no intermediate sqrt) — both null under two valid values; median and mean are always Float; n_unique counts distinct valid values exactly. quantile(q) takes q in [0, 1] (ValueError outside that range, checked at build time) and linearly interpolates between order statistics — quantile(0.5) reproduces median() bit-for-bit, since the median case is handled with the same exact arithmetic rather than a generic lerp that would round differently. Both var and quantile are null on an empty or all-null group. For corr, a row contributes only when both inputs are valid. Float sums and means accumulate in parallel, so their final bits can differ run to run — the engine's one documented carve-out.

Aggregations can be combined arithmetically inside the same expression — ((col("px") * col("qty")).sum() / col("qty").sum()) is one VWAP expression, not two steps — as long as every col reached is underneath an aggregate; a row-level column reached outside one raises SchemaError naming it, and nesting one aggregate inside another (col("x").sum().sum()) is rejected the same way. See agg() for the full rule and the split-step alternative.

import keyten as kt

df = kt.DataFrame([
    kt.Series.string("sym", ["a", "a", "b", "b"]),
    kt.Series.float("px", [1.0, 2.0, 10.0, 20.0]),
    kt.Series.int("qty", [5, None, 7, 8]),
])
out = (
    df.lazy()
    .group_by("sym")
    .agg([
        kt.col("px").mean().alias("avg"),
        kt.col("qty").count().alias("n"),
        kt.corr(kt.col("px"), kt.col("qty")).alias("c"),
    ])
    .sort("sym")
    .collect()
)
assert out.to_dict()["n"] == [1, 2]

import keyten as kt

df = kt.DataFrame([kt.Series.float("px", [10.0, 11.0, 12.0, 22.0])])
out = df.lazy().select([
    kt.col("px").std().alias("sd"),
    kt.col("px").var().alias("vr"),
    kt.col("px").median().alias("med"),
    kt.col("px").quantile(0.5).alias("q50"),
    kt.col("px").quantile(0.25).alias("q25"),
]).collect().to_dict()
assert abs(out["vr"][0] - out["sd"][0] ** 2) < 1e-9
assert out["med"] == out["q50"]

Null and NaN

is_in(values)     # membership; null stays null
is_nan()          # Float NaN test; null stays null
is_null()         # true where the lane is null
is_not_null()     # true where the lane is valid
fill_null(value)  # replace null lanes
nan_to_null()     # Float NaN lanes become null lanes

Null propagates: a null operand yields a null result through arithmetic and comparisons, is_in and is_nan keep null null rather than answering false, and aggregations skip null inputs. is_null and is_not_null are the one exception to that propagation — they always answer True or False, never null, which is what makes them the tool for finding or filtering nulls in the first place. NaN is a value (a Float lane that is not null) until nan_to_null says otherwise.

Conditional values

when(cond).then(value)   # start a branch; chain more with .when(...).then(...)
.otherwise(default)      # required — finalizes into an Expr
coalesce(exprs)          # first non-null lane across a list of expressions

when(cond) returns a When builder and .then(value) advances it to WhenThen. Chaining another .when(cond2).then(value2) adds branches evaluated in order, first match wins — a later branch's condition is never even considered for a lane an earlier branch already claimed. .otherwise(default) is required to turn the chain into a usable Expr; without it (e.g. passing the bare WhenThen to select) raises TypeError, since there is no value defined for a lane that matched nothing. It is sugar over nested if_else: a two-branch when is exactly if_else(cond, value, default).

coalesce(exprs) takes a list of expressions (columns or literals) and returns, lane by lane, the first one that is non-null — the multi-column fill_null. An empty list raises ValueError.

import keyten as kt

df = kt.DataFrame([kt.Series.float("px", [5.0, 15.0, 25.0])])
out = df.lazy().with_columns(
    kt.when(kt.col("px") < 10.0).then(100.0).when(kt.col("px") < 20.0).then(200.0).otherwise(300.0).alias("bucket")
).collect().to_dict()
assert out["bucket"] == [100.0, 200.0, 300.0]

nulls = kt.DataFrame([
    kt.Series.float("a", [1.0, None, 3.0, None]),
    kt.Series.float("b", [10.0, 20.0, None, None]),
])
c = nulls.lazy().with_columns(
    kt.coalesce([kt.col("a"), kt.col("b"), 99.0]).alias("c")
).collect().to_dict()
assert c["c"] == [1.0, 20.0, 3.0, 99.0]

Strings

str_contains(pat)  # plain substring test, not a regex
str_extract(pat)   # first regex match (capture group if present); null on no match
str_len()          # character count (code points, not bytes)

str_extract validates its pattern eagerly: an unsupported construct raises ValueError with the reason at build time, never a late error at collect.

Windows and over

cum_sum()                            # running sum
shift(n=1)                           # value from n rows earlier
diff()                               # difference from the previous row
rolling_mean(window_size, min_samples=None)
forward_fill()                       # carry the last valid value forward
rank(method="average", *, descending=False)  # rank within the column
row_number()                         # 1-based position within the column
over(partition_by)                   # evaluate per partition

These read the whole column in row order — they cannot run per-block, so they form their own execution stage. rolling_mean's min_samples defaults to window_size: a full window is required unless you lower it. over partitions by its keys (strings are column names) and evaluates the inner expression independently per partition: an aggregate inner broadcasts one value across its partition; a window inner restarts at each partition boundary.

rank orders an Int or Float column and assigns each valid lane its position; null lanes stay null and consume no rank (they neither shift nor receive one). method picks the tie rule: "average" (the default) gives tied lanes the mean of the ranks they span, always producing a Float output; "min" and "max" give every tied lane the lowest/highest rank in the span; "dense" is like "min" but never leaves a gap for the ties it collapsed; "ordinal" breaks ties by row order, giving every valid lane a distinct rank. "min", "max", "dense", and "ordinal" are always Int. descending reverses the value order the ranks are drawn from; NaN sorts as the largest valid value, same as everywhere else in the engine (sort, the rolling family). row_number() is simpler: a 1-based position in row order, ignoring values entirely — never null, always Int. Under .over(partition_by), both restart at 1 for each partition rather than running across the whole frame.

import keyten as kt

df = kt.DataFrame([
    kt.Series.string("sym", ["a", "a", "a", "b", "b", "b"]),
    kt.Series.float("px", [10.0, 30.0, 20.0, 5.0, 5.0, 1.0]),
])
out = df.lazy().with_columns([
    kt.col("px").rank(method="ordinal").over("sym").alias("rk"),
    kt.col("px").row_number().over("sym").alias("rn"),
]).collect().to_dict()
assert out["rk"] == [1, 3, 2, 2, 3, 1]  # sym b's [5, 5, 1] tie breaks on row order
assert out["rn"] == [1, 2, 3, 1, 2, 3]  # restarts at each symbol boundary

import keyten as kt

df = kt.DataFrame([
    kt.Series.string("sym", ["a", "a", "a", "b", "b"]),
    kt.Series.int("px", [10, 12, 11, 100, 101]),
])
out = df.lazy().with_columns([
    kt.col("px").diff().over("sym").alias("chg"),
    kt.col("px").max().over("sym").alias("hi"),
]).collect()
assert out.to_dict()["chg"] == [None, 2, -1, None, 1]
assert out.to_dict()["hi"] == [12, 12, 12, 101, 101]

Rolling and exponentially weighted windows

rolling_sum(window_size, min_samples=window_size)
rolling_mean(window_size, min_samples=window_size)
rolling_min(window_size, min_samples=window_size)
rolling_max(window_size, min_samples=window_size)
rolling_std(window_size, min_samples=window_size)
rolling_var(window_size, min_samples=window_size)
cum_min()  cum_max()   # next to cum_sum, running min/max
ewm_mean(*, span=None, alpha=None)
ewm_std(*, span=None, alpha=None)
ewm_var(*, span=None, alpha=None)
rolling_sum_by(time_col, window)
rolling_mean_by(time_col, window)
rolling_min_by(time_col, window)
rolling_max_by(time_col, window)
rolling_std_by(time_col, window)
rolling_var_by(time_col, window)

rolling_{sum,mean,min,max,std,var} slide a trailing window of window_size rows: for row i the window is rows i - window_size + 1 ..= i, clipped at the start of the column. A lane is null when fewer than min_samples values inside its window are valid — nulls in the window are skipped, not counted as zero. min_samples defaults to window_size (a full window required); pass a lower value to get output before the window fills. rolling_sum/min/max keep the input's kind (Int stays Int); rolling_std/var always promote to Float and use sample variance (ddof = 1, dividing by n - 1, matching std()) — because variance needs at least two points, rolling_std/rolling_var silently raise their own effective floor to max(min_samples, 2), so a lane is null under two valid values even if min_samples asks for fewer. All six reject non-numeric columns (Bool included).

cum_min and cum_max sit next to cum_sum: a running minimum/maximum over valid values seen so far. A null lane stays null in the output and leaves the running accumulator untouched — the same convention cum_sum and every window op use.

import keyten as kt

df = kt.DataFrame([kt.Series.float("x", [1.0, 2.0, None, 4.0])])
out = df.lazy().with_columns([
    kt.col("x").rolling_sum(2).alias("s"),
    kt.col("x").rolling_min(2, min_samples=1).alias("lo"),
    kt.col("x").cum_max().alias("hi"),
]).collect().to_dict()
assert out["s"] == [None, 3.0, None, None]  # default min_samples=2: needs a full window
assert out["lo"] == [1.0, 1.0, 2.0, 4.0]
assert out["hi"] == [1.0, 2.0, None, 4.0]   # null lane stays null; accumulator held at 2.0

ewm_mean, ewm_std, and ewm_var are exponentially weighted: span and alpha are keyword-only and mutually exclusive — pass exactly one, or building the expression raises ValueError. span converts as alpha = 2 / (span + 1); alpha itself must be in (0, 1]. Weighting is adjust=True: when evaluated at row i, row j's observation carries weight (1 - alpha) ** (i - j), so early rows are not artificially underweighted the way a naive recursive EWMA would understate them. A null row still decays every other row's accumulated weight (nulls are not skipped over as if they never happened) but itself emits null. ewm_std/ewm_var use the unbiased (reliability-weight-corrected) estimator, matching the ddof=1 correction rolling_var and std() use, just exponentially weighted instead of a flat window; both are null until a second valid value has been seen. Int columns promote to Float for all three.

import keyten as kt

df = kt.DataFrame([kt.Series.float("x", [1.0, None, 3.0])])
out = df.lazy().with_columns([
    kt.col("x").ewm_mean(alpha=0.5).alias("by_alpha"),
    kt.col("x").ewm_mean(span=3).alias("by_span"),
]).collect().to_dict()
assert out["by_alpha"][0] == 1.0
assert out["by_alpha"][1] is None          # null in, null out — but weight still decayed
assert abs(out["by_alpha"][2] - 2.6) < 1e-12

rolling_{sum,mean,min,max,std,var}_by(time_col, window) are the duration-based counterpart: instead of a fixed row count, the window for row i is every row j at or before row i with time[j] in (time[i] - window, time[i]] — trailing, closed on the right, open on the left, so a row never double-counts a tie at exactly window in the past. "At or before row i" matters when time_col has ties: two rows sharing the same timestamp see different windows, since the later of the pair sees the earlier one but not vice versa. time_col must be a Timestamp column, sorted ascending with no nulls; an unsorted or null timestamp raises at collect (never silently produces a wrong answer). window is a duration string in the same <positive-int><unit> form as truncate ("1m", "30s"). There is no min_samples parameter — a lane is null only when its window holds zero valid values (or, for _std/_var, fewer than two), the same floor as the row-count rolling family.

import keyten as kt
from datetime import datetime

stamps = [datetime(2026, 1, 5, 9, 30, 0), datetime(2026, 1, 5, 9, 30, 30),
          datetime(2026, 1, 5, 9, 31, 0), datetime(2026, 1, 5, 9, 33, 20)]
df = kt.DataFrame([
    kt.Series.timestamp("ts", stamps),
    kt.Series.float("px", [1.0, 2.0, 4.0, 8.0]),
])
out = df.lazy().with_columns(kt.col("px").rolling_sum_by("ts", "1m").alias("s")).collect()
assert out.column("s").to_list() == [1.0, 3.0, 6.0, 8.0]

CAVEAT: rolling_sum/std/var and ewm_std/var all accumulate with a running total that adds on window-entry and subtracts on window-exit (or decays every row, for EWM); over long windows of large-magnitude values this can lose precision to float reassociation compared to summing the window fresh each time, the same tradeoff cum_sum and the aggregate sum()/mean() make. rolling_std/var and ewm_std/var clamp variance to >= 0 before taking the square root to absorb the resulting cancellation noise rather than propagate a spurious NaN. A NaN value (as opposed to null) anywhere inside a rolling_sum/rolling_std/rolling_var window poisons the running accumulator and stays in every lane's output until that row finally exits the window — the same behavior rolling_mean already has; call nan_to_null() first if you don't want NaN to spread.

Recoding values

recode(mapping, default=None)

mapping is a dict or an iterable of (from, to) literal pairs. Unmatched values become default, or pass through unchanged when default is None; null stays null.

import keyten as kt

df = kt.DataFrame([kt.Series.string("ex", ["N", "Q", "Z"])])
out = df.lazy().select(
    kt.col("ex").recode({"N": "NYSE", "Q": "NASDAQ"}).alias("name")
).collect()
assert out.to_dict()["name"] == ["NYSE", "NASDAQ", "Z"]

Where to look next

Expressions run inside LazyFrame and GroupBy queries over Series and DataFrame data; how they execute is the subject of execution and parallelism.