Skip to content

Mutating an eager DataFrame

Series values and published blocks are immutable, but a DataFrame handle can replace its table in place. Keyten provides four eager mutation operations: append, upsert, update, and delete.

These methods change the DataFrame object immediately. They do not build a lazy plan and they do not rewrite a native table on disk unless you explicitly persist the result.

Append rows

append requires the same column names, order, and kinds on both frames.

import keyten as kt

accounts = kt.DataFrame([
    kt.Series.int("account_id", [1, 2]),
    kt.Series.float("balance", [100.0, 50.0]),
    kt.Series.bool("active", [True, True]),
])

before = accounts.lazy()
accounts.append(kt.DataFrame([
    kt.Series.int("account_id", [3]),
    kt.Series.float("balance", [25.0]),
    kt.Series.bool("active", [False]),
]))

assert accounts.shape == (3, 3)
assert before.collect().shape == (2, 3)

A LazyFrame keeps a snapshot of the table it was built from. Mutating accounts later does not change before.

Upsert by key

upsert(other, on=...) replaces rows whose key already exists and appends new keys. The incoming frame must have the exact same schema.

batch = kt.DataFrame([
    kt.Series.int("account_id", [2, 4]),
    kt.Series.float("balance", [80.0, 10.0]),
    kt.Series.bool("active", [True, True]),
])
accounts.upsert(batch, on="account_id")

state = accounts.to_dict()
assert state["account_id"] == [1, 2, 3, 4]
assert state["balance"] == [100.0, 80.0, 25.0, 10.0]

For a composite key, pass a list such as on=["venue", "order_id"]. Null keys in the incoming frame are rejected. If the incoming batch repeats a key, its last row wins. A matching key replaces every pre-existing row carrying that key.

Update matching rows

update takes a boolean expression and a mapping of columns to new values. Every set expression reads the frame as it existed before the update started, so assignments in the same call never observe each other.

accounts.update(
    where=~kt.col("active"),
    set={
        "balance": kt.col("balance") + 5.0,
        "active": True,
    },
)

assert accounts.column("balance").to_list() == [100.0, 80.0, 30.0, 10.0]
assert accounts.column("active").to_list() == [True, True, True, True]

The replacement must match the target column's kind. Unknown columns and invalid expressions raise SchemaError without partially applying the update.

Delete matching rows

accounts.delete(where=kt.col("account_id") == 1)
assert accounts.column("account_id").to_list() == [2, 3, 4]

Persist the result

In-memory mutation and native-table append are separate operations:

accounts.write_native("accounts.k10dir")
assert kt.scan_native("accounts.k10dir").collect().shape == (3, 3)
  • Use write_native to atomically replace a native table with the current frame.
  • Use append_native to append a same-schema frame directly to an existing native table.
  • Use append, upsert, update, or delete when you want to change an in-memory frame first.

See Series and DataFrame for the complete eager API and data in and out for persistence guarantees.