Skip to main content

Submit a scan for planning

POST 
/v1/:prefix/namespaces/:namespace/tables/:table/plan

Submits a scan for server-side planning.

Point-in-time scans are planned by passing snapshot-id to identify the table snapshot to scan. Incremental scans are planned by passing both start-snapshot-id and end-snapshot-id. Requests that include both point in time config properties and incremental config properties are invalid. If the request does not include either incremental or point-in-time config properties, scan planning should produce a point-in-time scan of the latest snapshot in the table's main branch.

Responses must include a valid status listed below. A "cancelled" status is considered invalid for this endpoint.

  • When "completed" the planning operation has produced plan tasks and file scan tasks that must be returned in the response (not fetched later by calling fetchPlanningResult)

  • When "submitted" the response must include a plan-id used to poll fetchPlanningResult to fetch the planning result when it is ready

  • When "failed" the response must be a valid error response The response for a "completed" planning operation includes two types of tasks (file scan tasks and plan tasks) and both may be included in the response. Tasks must not be included for any other response status.

Responses that include a plan-id indicate that the service is holding state or performing work for the client.

  • Clients should use the plan-id to fetch results from fetchPlanningResult when the response status is "submitted"

  • Clients should inform the service if planning results are no longer needed by calling cancelPlanning. Cancellation is not necessary after fetchScanTasks has been used to fetch scan tasks for each plan task.

Request​

Path Parameters

    prefix stringrequired

    The catalog name in Gravitino, as returned in defaults.prefix by GET /v1/config

    namespace stringrequired

    A namespace identifier as a single string. Multipart namespace parts must be separated by the namespace separator as indicated via the /config override namespace-separator, which defaults to the unit separator 0x1F byte (url encoded %1F). To be compatible with older clients, servers must use both the advertised separator and 0x1F as valid separators when decoding namespaces. The namespace-separator should be provided in a url encoded form.

    Examples:

    Example: accounting

    table stringrequired

    A table name

    Example: sales

Header Parameters

    Idempotency-Key uuid

    Possible values: >= 36 characters and <= 36 characters

    Optional client-provided idempotency key for safe request retries.

    When present, the server ensures no additional effects for requests that carry the same Idempotency-Key. If a prior request with this key has been finalized, the server returns an equivalent final response without re-running the operation. The response body may reflect a newer state of the catalog than existed at the time of the commit.

    Finalization rules:

    • Finalize & replay: 200, 201, 204, and deterministic terminal 4xx (including 409 such as AlreadyExists, NamespaceNotEmpty, etc.)
    • Do not finalize (not stored/replayed): 5xx

    Key Requirements:

    • Key format: UUIDv7 in string form (RFC 9562).
    • The idempotency key must be globally unique (no reuse across different operations).
    • Catalogs SHOULD NOT expire keys before the end of the advertised token lifetime.
    • If Idempotency-Key is used, clients MUST reuse the same key when retrying the same logical operation and MUST generate a new key for a different operation.
    X-Iceberg-Access-Delegation string

    Possible values: [vended-credentials, remote-signing]

    Optional signal to the server that the client supports delegated access via a comma-separated list of access mechanisms. The server may choose to supply access via any or none of the requested mechanisms.

    Specific properties and handling for vended-credentials and remote-signing are documented in the LoadTableResult schema section of this spec document.

    Example: vended-credentials,remote-signing

Body

    snapshot-id int64

    Identifier for the snapshot to scan in a point-in-time scan

    select FieldName (string)[]

    List of selected schema fields

    filter

    object

    Expression used to filter the table data

    oneOf

    type ExpressionType (string)required
    min-rows-requested int64

    The minimum number of rows requested for the scan. This is used as a hint to the server to not have to return more rows than necessary. It is not required for the server to return that many rows since the scan may not produce that many rows. The server can also return more rows than requested.

    case-sensitive boolean

    Default value: true

    Enables case sensitive field matching for filter and select

    use-snapshot-schema boolean

    Whether to use the schema at the time the snapshot was written. When time travelling, the snapshot schema should be used (true). When scanning a branch, the table schema should be used (false).

    start-snapshot-id int64

    Starting snapshot ID for an incremental scan (exclusive)

    end-snapshot-id int64

    Ending snapshot ID for an incremental scan (inclusive). Required when start-snapshot-id is specified.

    stats-fields FieldName (string)[]

    List of fields for which the service should send column stats.

Responses​

Result of submitting a table scan to plan

Schema

    oneOf

    delete-files

    object[]

    Delete files referenced by file scan tasks

  • Array [

    • oneOf

    content

    string

    required

    Possible values: [data, position-deletes, equality-deletes]

    content stringrequired
    first-row-id int64

    The first row ID assigned to the first row in the data file

    column-sizes

    object

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    value-counts

    object

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    null-value-counts

    object

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    nan-value-counts

    object

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    lower-bounds

    object

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values

    object[]

    List of primitive type values, matched to 'keys' by index

  • Array [

    • oneOf

    boolean

  • ]

    • upper-bounds

    object

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values

    object[]

    List of primitive type values, matched to 'keys' by index

  • Array [

    • oneOf

    boolean

  • ]

    • content-offset int64

    Offset within the delete file of delete content

    content-size-in-bytes int64

    Length, in bytes, of the delete content; required if content-offset is present

    file-path stringrequired
    file-format FileFormat (string)required

    Possible values: [avro, orc, parquet, puffin]

    spec-id integerrequired

    partition

    object[]

    required

    A list of partition field values ordered based on the fields of the partition spec specified by the spec-id

  • Array [

    • oneOf

    boolean

  • ]

    • file-size-in-bytes int64required

    Total file size in bytes

    record-count int64required

    Number of records in the file

    key-metadata BinaryTypeValue (string)

    Encryption key metadata blob

    split-offsets int64[]

    List of splittable offsets

    sort-order-id integer

  • ]

    • file-scan-tasks

    object[]

  • Array [

    • data-file

    object

    required

    content

    string

    required

    Possible values: [data, position-deletes, equality-deletes]

    circular(DataFile)

    first-row-id int64

    The first row ID assigned to the first row in the data file

    column-sizes

    object

    Map of column id to total count, including null and NaN

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    value-counts

    object

    Map of column id to null value count

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    null-value-counts

    object

    Map of column id to null value count

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    nan-value-counts

    object

    Map of column id to number of NaN values in the column

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values int64[]

    List of Long values, matched to 'keys' by index

    lower-bounds

    object

    Map of column id to lower bound primitive type values

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values

    object[]

    List of primitive type values, matched to 'keys' by index

  • Array [

    • oneOf

    boolean

  • ]

    • upper-bounds

    object

    Map of column id to upper bound primitive type values

    keys IntegerTypeValue (integer)[]

    List of integer column ids for each corresponding value

    values

    object[]

    List of primitive type values, matched to 'keys' by index

  • Array [

    • oneOf

    boolean

  • ]

    • file-path stringrequired
    file-format FileFormat (string)required

    Possible values: [avro, orc, parquet, puffin]

    spec-id integerrequired

    partition

    object[]

    required

    A list of partition field values ordered based on the fields of the partition spec specified by the spec-id

  • Array [

    • oneOf

    boolean

  • ]

    • file-size-in-bytes int64required

    Total file size in bytes

    record-count int64required

    Number of records in the file

    key-metadata BinaryTypeValue (string)

    Encryption key metadata blob

    split-offsets int64[]

    List of splittable offsets

    sort-order-id integer
    delete-file-references integer[]

    A list of indices in the delete files array (0-based)

    residual-filter

    object

    An optional filter to be applied to rows in this file scan task. If the residual is not present, the client must produce the residual or use the original filter.

    oneOf

    type ExpressionType (string)required

  • ]

    • plan-tasks PlanTask (string)[]
    status PlanStatus (string)required

    Possible values: [completed, submitted, cancelled, failed]

    Status of a server-side planning operation

    storage-credentials

    object[]

    Storage credentials for accessing the files returned in the scan result. If the server returns storage credentials as part of the completed scan planning response, the expectation is for the client to use these credentials to read the files returned in the FileScanTasks as part of the scan result.

  • Array [

    • prefix stringrequired

    Indicates a storage location prefix where the credential is relevant. Clients should choose the most specific prefix (by selecting the longest prefix) if several credentials of the same type are available.

    config

    object

    required

    property name* string

  • ]

    • plan-id stringrequired

    ID used to track a planning request

Loading...
Last updated on