DataFrame and Arrow Data Bridge¶

wolfxl.data moves typed tabular data between Excel files and Python data systems through one native Arrow path. Read, scan, and write share a single option contract and a single set of type rules, so a value that survives a write is the value a later read returns.

Every optional dependency stays optional: importing wolfxl or wolfxl.data imports neither pandas, Polars, PyArrow, nor DuckDB, and no adapter imports one until it is called.

Entry points¶

Function	Direction	Result
`scan_arrow(path, sheet)`	file to Arrow	one-shot Arrow C stream, batch at a time
`read_arrow(path, sheet)`	file to Arrow	one `pyarrow.RecordBatch`
`scan_polars(path, sheet)`	file to Polars	`polars.LazyFrame`, still lazy
`read_polars(path, sheet)`	file to Polars	`polars.DataFrame`
`write_arrow(data, path)`	Arrow to file	one worksheet
`write_arrow_sheets(sheets, path)`	Arrow to file	ordered multi-sheet workbook
`register_pandas_engine()`	file to pandas	enables `read_excel(..., engine="wolfxl")`
`import_pandas` / `import_polars` / `import_arrow` / `import_records` / `import_sql_cursor`	Python object to worksheet	bounded `DataWriteReport`
`export_pandas(worksheet)`	worksheet to pandas	`pandas.DataFrame`

scan_arrow, read_arrow, scan_polars, and read_polars accept the same validated options: header, batch_rows, columns, start_row, max_rows, temporal, and strict. Invalid options are rejected before any file is opened.

Reading: type and null policy¶

Column types are frozen from a bounded inference window of the first 1,024 data rows in the requested range, then applied to every later row.

Worksheet content	Arrow type
logical `TRUE`/`FALSE`	`Boolean`
numbers, and serials under `temporal="serial"`	`Float64`
text, and any column mixing kinds inside the window	`Utf8`
date-formatted serials under `temporal="native"`	`Date32`
time-formatted serials under `temporal="native"`	`Time64[us]`
date-and-time serials under `temporal="native"`	`Timestamp[us]`, no time zone

Nulls. Every field is nullable. Blank and missing cells become null. A row that is entirely blank inside the requested range is still emitted, as nulls.
Formulas. Only the cached value of a formula cell contributes. Schema metadata records wolfxl.formula_mode = cached. WolfXL never recalculates during a scan.
Mixed columns. A column whose kinds disagree inside the inference window widens to Utf8, and its values are rendered as text. A value that first disagrees after the window is a late incompatible value: with strict=True (the default) it raises a data error naming the sheet, cell, and field; with strict=False it becomes null and the schema records wolfxl.loss_mode = null_on_incompatible.
Excel error cells (#REF!, #VALUE!, and siblings) have no Arrow scalar representation. They raise under strict=True and become null under strict=False.
Temporal metadata. Native temporal fields carry wolfxl.logical_type, wolfxl.number_format, and wolfxl.date_system (1900 or 1904). Sheet-level metadata records the sheet name, first row and column, header mode, temporal mode, and date system.
Header and projection. header=True consumes the first read row as field names; header=False generates positional names. columns selects fields by name or 1-based Excel column index and suppresses decoding of unselected cells. start_row is applied before header removal, and max_rows bounds emitted data rows.
Lifetime. A scanner is one-shot. The first export or iteration takes ownership; a second export raises OSError.

Writing: type and null policy¶

Arrow type	Worksheet result
`Null`	blank cell
`Boolean`	logical `TRUE`/`FALSE`
`Int8`-`Int64`, `UInt8`-`UInt64`	number
`Float32`, `Float64`	number
`Utf8`, `LargeUtf8`	text
`Date32`	serial with a date style
`Time64[us]`	serial with a time style
`Timestamp[us]`, no time zone	serial with a date-and-time style
`Decimal128`, `Decimal256`	exact text at the declared scale
`Dictionary` with integer keys and `Utf8`/`LargeUtf8` values	decoded text

Nulls. A null in any column, including a null dictionary key, becomes a blank cell rather than an empty string or a zero.
Decimal. Every Arrow-valid Decimal128/Decimal256 scale is accepted and written as exact text, preserving every digit and the declared scale. WolfXL does not convert a decimal to a double, because a double cannot carry the exact value; a caller who wants arithmetic in Excel must cast the column to a floating-point type first and accept that rounding.
Dictionary. Only dictionaries with Arrow integer keys (i8-i64, u8-u64) and Utf8/LargeUtf8 values are decoded to their text values. Every other key or value combination is rejected as an unsupported type.
Integers. An integer whose magnitude exceeds 2^53 cannot be represented exactly by Excel's number model and is rejected, naming the column and row. Cast the column to Utf8 to preserve every digit.
Non-finite numbers. NaN and infinities are rejected, naming the column and row. Excel has no cell value for either.
Header. header=True writes field names into the first worksheet row and that row counts against the row limit. header=False writes data only.
Index. Arrow has no index concept, so the Arrow write path never invents one. import_pandas(..., index=True) is the only path that materializes a pandas index, and it writes index levels as leading columns whose unnamed levels use an empty label.
Batching. Batches are consumed one at a time and never collected first, so an unbounded reader is streamed. Every batch of one sheet must share one schema; a mismatch names the offending batch index. Each producer must yield at least one batch.

Limits¶

1,048,576 rows and 16,384 columns per worksheet, enforced per sheet, with the header row counted when header=True.
The limit is checked before rows are appended, so an oversized stream fails without producing a partial file.
import_polars and import_arrow additionally accept a caller max_rows bound and reject a larger source instead of truncating it.

Unsupported write types¶

Binary, large binary, fixed-size binary, list, large list, struct, map, union, interval, duration, Date64, Time32, and any extension type are rejected with column "<name>" has unsupported Arrow type <type>; cast it to a supported scalar type before writing.

Temporal types outside the supported set are rejected specifically: only Time64[us] and Timestamp[us] without a time zone are accepted. A timezone-aware timestamp is rejected rather than silently converted, because Excel has no time-zone-aware cell type and any conversion would silently pick a zone on the caller's behalf.

Rejection always happens before the destination path is replaced. A failed write leaves both the source data and any existing target file unchanged.

Ordered multi-sheet writes¶

from wolfxl.data import write_arrow_sheets

write_arrow_sheets(
    [("Ledger", ledger_table), ("Labels", labels_table)],
    "pack.xlsx",
)

sheets is an ordered sequence of (sheet_name, data) pairs or an ordered mapping; worksheet order in the published workbook is the given order. Each data is any object exposing __arrow_c_stream__ or __arrow_c_array__.

The wrapper validates every sheet name (type, length, Excel's forbidden characters) and every producer's Arrow protocol, and rejects a name that repeats an earlier one under Excel's case-insensitive worksheet-name comparison, before the native writer is called at all. The native writer then preflights every sheet before it creates output and publishes the workbook atomically. Either the whole workbook appears, or nothing does.

pandas¶

import pandas as pd
import wolfxl.data

wolfxl.data.register_pandas_engine()
frame = pd.read_excel("book.xlsx", engine="wolfxl")

Registration is explicit and idempotent. Importing WolfXL never changes how pandas.read_excel behaves, repeated registration is a no-op, and a foreign engine already registered under the name wolfxl is reported rather than replaced. pandas is imported only by the registration call; a missing pandas raises ImportError naming the install command.

Engine boundaries:

The engine reads .xlsx files from a filesystem path. A file object or in-memory buffer is rejected with an explicit message; write it to a file, or use an engine that accepts buffers.
PyArrow is required at read time, and is imported only then. Worksheet values reach pandas one decoded Arrow column at a time; no cell is read through a Python worksheet object.
The first worksheet row is always consumed as Arrow field names, so pandas header=0 (the default) behaves as expected, and header=None sees that row as text.
nrows is forwarded as a native row bound, so a bounded read does not decode the whole sheet.
engine_kwargs accepts only temporal, strict, and batch_rows, forwarded to the scan. Any other key is rejected by name.
Read-side null, temporal, mixed-column, and error-cell policy is the policy documented above: pandas receives None for nulls and the same strict-mode failures.

export_pandas(worksheet) remains the in-memory counterpart for a worksheet that is already open in a Workbook; the engine is the path for a file on disk.

Polars¶

from wolfxl.data import read_polars, scan_polars

lazy = scan_polars("book.xlsx", "Data")       # stays lazy until .collect()
frame = read_polars("book.xlsx", "Data")      # materialized on request

scan_polars registers the scan through the public polars.io.plugins.register_io_source hook, forwards Polars column projection to scan_arrow(columns=...), honours a pushed-down predicate, and returns a LazyFrame that decodes nothing until the consumer collects. read_polars is the eager sibling: Polars ingests the native stream through the Arrow PyCapsule interface in one call.

Neither function imports pandas or PyArrow, and neither converts a value through a Python scalar. Polars is imported only when one of them is called; a missing Polars raises ImportError naming the install command, and a Polars release without register_io_source is reported instead of being patched around.

A Polars frame is also a valid write producer: it exposes __arrow_c_stream__, so write_arrow(frame, path) and write_arrow_sheets([("Sheet1", frame)], path) accept it directly under the write policy above.