Writing queries

This tutorial builds up the lazy query model from nothing: eager data, a plan, transformations, aggregation, and how to see what the engine will actually run. Everything here executes as shown.

Eager data, lazy queries

Data lives eagerly in Series and DataFrame. Queries are lazy: .lazy() starts a plan, each method adds to it, and nothing touches data until collect().

import keyten as kt

trades = kt.DataFrame([
    kt.Series.string("sym", ["ev", "od", "ev", "od", "ev"]),
    kt.Series.int("qty", [100, 250, 300, 150, 75]),
    kt.Series.float("px", [10.0, 20.5, 10.2, 19.9, 10.1]),
])
print(trades)

The laziness is not a formality. Because the whole query is known before anything runs, the optimizer pushes filters into the scan, prunes columns nobody reads, and fuses adjacent steps — you write the clear version, the engine runs the fast one.

Filter, derive, select

q = (
    trades.lazy()
    .filter((kt.col("qty") >= 100) & (kt.col("sym") == "ev"))
    .with_columns((kt.col("qty") * kt.col("px")).alias("notional"))
    .select(["sym", "qty", "notional"])
)
out = q.collect()
assert out.column("notional").to_list() == [1000.0, 3060.0]

Note the two roles a string plays: inside select it names a column; as an operand of == it is a string literal. col("...") is always unambiguous.

Aggregate

group_by takes keys, agg takes aggregations; the result is a new frame with one row per group. Group output order is unspecified — sort if you care.

summary = (
    trades.lazy()
    .group_by("sym")
    .agg([
        kt.col("qty").sum().alias("total_qty"),
        kt.col("px").mean().alias("avg_px"),
        kt.col("qty").count().alias("n"),
    ])
    .sort("sym")
    .collect()
)
assert summary.column("total_qty").to_list() == [475, 400]

A select where every entry is an aggregation aggregates the whole frame in one row:

totals = trades.lazy().select([
    kt.col("qty").sum().alias("qty"),
    kt.col("px").max().alias("hi"),
]).collect()
assert totals.shape == (1, 2)

See the plan

explain() renders the optimized plan without running it. Reading it is the fastest way to understand what a query costs — and to confirm a filter really reached the scan.

plan = (
    trades.lazy()
    .filter(kt.col("qty") > 1)
    .with_columns((kt.col("qty") * 10).alias("q10"))
    .group_by("sym")
    .agg(kt.col("q10").sum())
    .explain()
)
assert "SCAN" in plan
print(plan)

The scan line lists only the columns the rest of the plan needs, and the filter appears on the scan itself — predicate pushdown and projection pruning, visible.

Mistakes are sticky, not fatal

An unknown column or a type mismatch is recorded in the plan where it happens; the chain keeps building, and the error surfaces — as the original error — when you collect.

q = trades.lazy().filter(kt.col("typo") > 1).select("sym")
try:
    q.collect()
except kt.SchemaError as e:
    print("caught:", e)

Where to go next

Data in and out covers files; joins and time series cover the two big multi-frame workflows. The full method-by-method contract lives in the Python API reference.