openapi: 3.1.0 info: title: Grava API version: 1.0.0 license: name: Proprietary identifier: LicenseRef-Proprietary description: >- Create living datasets from declarative web sources, run them, and read checked rows. Accepted Runs cost one credit per distinct checked page. Failed or held output costs no credits. servers: - url: https://app.grava.dev/api description: Production security: - apiKey: [] tags: - name: Datasets description: Create datasets and read their checked rows. - name: Runs description: Enqueue and inspect source executions. - name: Usage description: Read credit balance and ledger activity. paths: /v1/datasets: post: operationId: createDataset tags: [Datasets] summary: Create a dataset description: >- Atomically creates a draft dataset, its single Bot, and recipe version 1. Source fields are declarative selectors; arbitrary code is not accepted or executed. requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/CreateDatasetRequest" responses: "201": description: Dataset created. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: $ref: "#/components/schemas/DatasetResponse" "401": { $ref: "#/components/responses/Unauthorized" } "409": { $ref: "#/components/responses/Conflict" } "422": { $ref: "#/components/responses/ValidationError" } "500": { $ref: "#/components/responses/InternalError" } get: operationId: listDatasets tags: [Datasets] summary: List datasets description: Lists the authenticated owner's datasets, newest first. parameters: - $ref: "#/components/parameters/DatasetCursor" - $ref: "#/components/parameters/DatasetLimit" responses: "200": description: A page of datasets. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: type: object required: [data, nextCursor] properties: data: type: array items: $ref: "#/components/schemas/Dataset" nextCursor: oneOf: - type: string - type: "null" "400": { $ref: "#/components/responses/BadRequest" } "401": { $ref: "#/components/responses/Unauthorized" } "500": { $ref: "#/components/responses/InternalError" } /v1/datasets/{datasetId}: parameters: - $ref: "#/components/parameters/DatasetId" get: operationId: getDataset tags: [Datasets] summary: Get a dataset responses: "200": description: Dataset found. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: $ref: "#/components/schemas/DatasetResponse" "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "500": { $ref: "#/components/responses/InternalError" } patch: operationId: updateDataset tags: [Datasets] summary: Update a dataset description: >- Updates dataset metadata and schedule. Changing columns does not reinterpret existing rows and does not change the source recipe. requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/UpdateDatasetRequest" responses: "200": description: Dataset updated. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: $ref: "#/components/schemas/DatasetResponse" "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "409": { $ref: "#/components/responses/Conflict" } "422": { $ref: "#/components/responses/ValidationError" } "500": { $ref: "#/components/responses/InternalError" } /v1/datasets/{datasetId}/rows: parameters: - $ref: "#/components/parameters/DatasetId" get: operationId: listDatasetRows tags: [Datasets] summary: List dataset rows description: >- Lists current rows ordered by updatedAt descending, then id descending. This is a current-state view. Use the changes endpoint for durable history. Repeat the status parameter to include multiple statuses. parameters: - name: since in: query description: Include rows updated at or after this instant. schema: type: string format: date-time - name: status in: query description: Row status to include. May be repeated. style: form explode: true schema: type: array uniqueItems: true items: $ref: "#/components/schemas/DatasetRowStatus" - $ref: "#/components/parameters/RowCursor" - $ref: "#/components/parameters/RowLimit" responses: "200": description: A page of rows. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: type: object required: [data, nextCursor] properties: data: type: array items: $ref: "#/components/schemas/DatasetRow" nextCursor: oneOf: - type: string - type: "null" "400": { $ref: "#/components/responses/BadRequest" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "500": { $ref: "#/components/responses/InternalError" } /v1/datasets/{datasetId}/rows/{rowId}: parameters: - $ref: "#/components/parameters/DatasetId" - name: rowId in: path required: true schema: { type: string, format: uuid } get: operationId: getDatasetRow tags: [Datasets] summary: Inspect a row and its evidence responses: "200": description: Current accepted row with resolved source observations. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: { $ref: "#/components/schemas/RowDetail" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } patch: operationId: editDatasetRow tags: [Datasets] summary: Edit cell values or reset manual overrides description: Requires the current revision ID and a unique request ID. A retry with the same request and payload returns the original revision. Non-identity collected fields support manual overrides that persist across Runs. Reset an override explicitly to use the latest source value. Identity fields are read-only. requestBody: required: true content: application/json: schema: { $ref: "#/components/schemas/EditRowRequest" } responses: "200": description: Accepted edit revision. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: { $ref: "#/components/schemas/RowRevision" } "400": { $ref: "#/components/responses/BadRequest" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "409": { $ref: "#/components/responses/Conflict" } "422": { $ref: "#/components/responses/ValidationError" } /v1/datasets/{datasetId}/changes: parameters: - $ref: "#/components/parameters/DatasetId" get: operationId: listDatasetChanges tags: [Datasets] summary: Read ordered row and cell history description: Read changes in ascending sequence. Keep the returned through value for subsequent pages; pass nextAfter as after. After draining the page, use through as after for the next poll and omit through. Optional rowId and fieldId select a row or cell timeline. parameters: - name: after in: query schema: { type: integer, minimum: 0 } - name: through in: query schema: { type: integer, minimum: 0 } - name: limit in: query schema: { type: integer, minimum: 1, maximum: 100, default: 50 } - name: rowId in: query schema: { type: string, format: uuid } - name: fieldId in: query schema: { type: string, format: uuid } responses: "200": description: A fixed window of durable revisions with before values and historical schemas. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: { $ref: "#/components/schemas/ChangePage" } "400": { $ref: "#/components/responses/BadRequest" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "422": { $ref: "#/components/responses/ValidationError" } /v1/datasets/{datasetId}/runs: parameters: - $ref: "#/components/parameters/DatasetId" get: operationId: listDatasetRuns tags: [Runs] summary: List recent dataset runs parameters: - name: limit in: query schema: { type: integer, minimum: 1, maximum: 100, default: 20 } responses: "200": description: Recent Runs, newest first. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: type: object required: [data] properties: data: type: array items: { $ref: "#/components/schemas/Run" } "400": { $ref: "#/components/responses/BadRequest" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } post: operationId: createDatasetRun tags: [Runs] summary: Enqueue a dataset run description: >- Durably enqueues a manual run. No credit is charged until all checks pass. A failed check writes no rows and costs nothing. responses: "202": description: Run accepted and durably queued. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: type: object required: [runId, creditsRemaining] properties: runId: type: string format: uuid creditsRemaining: type: integer example: runId: 8dca4e89-2667-46aa-9913-148984e3ed1d creditsRemaining: 1000 "401": { $ref: "#/components/responses/Unauthorized" } "402": { $ref: "#/components/responses/InsufficientCredits" } "404": { $ref: "#/components/responses/NotFound" } "409": { $ref: "#/components/responses/Conflict" } "503": { $ref: "#/components/responses/QueueUnavailable" } /v1/runs/{runId}: parameters: - name: runId in: path required: true description: Run UUID. schema: type: string format: uuid get: operationId: getRun tags: [Runs] summary: Get a run responses: "200": description: Run found. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: type: object required: [run] properties: run: $ref: "#/components/schemas/Run" "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "500": { $ref: "#/components/responses/InternalError" } /v1/runs/{runId}/artifacts: parameters: - name: runId in: path required: true schema: { type: string, format: uuid } get: operationId: listRunArtifacts summary: List the provider artifacts recorded for a Run description: >- Index of bodies recorded at the research provider seam (planning output, cleaned page text, extraction output) so a Run can be explained and replayed. Bodies are never included here; fetch each one from the single-artifact endpoint. Artifacts larger than 64 KiB keep only their hash and size. Artifacts are retained for 30 days; Rows, evidence and history do not depend on them. Runs recorded before artifact capture return an empty list. tags: [Runs] responses: "200": description: Bounded artifact index for the Run. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: type: object additionalProperties: false required: [runId, recipe, truncated, artifacts] properties: runId: { type: string, format: uuid } recipe: description: >- The pinned research Recipe, so a replay can rebuild the Run. Null for deterministic Runs. oneOf: - { $ref: "#/components/schemas/ResearchRecipe" } - { type: "null" } truncated: type: boolean description: True when more than 200 artifacts exist. artifacts: type: array maxItems: 200 items: { $ref: "#/components/schemas/RunArtifact" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } /v1/runs/{runId}/artifacts/{artifactId}: parameters: - name: runId in: path required: true schema: { type: string, format: uuid } - name: artifactId in: path required: true schema: { type: string, format: uuid } get: operationId: getRunArtifact summary: Read one recorded artifact with its body description: >- The artifact and its body. The body is null when it exceeded the 64 KiB cap; the hash and size still describe what was produced. tags: [Runs] responses: "200": description: One artifact. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: type: object additionalProperties: false required: [artifact] properties: artifact: allOf: - { $ref: "#/components/schemas/RunArtifact" } - type: object required: [body] properties: body: description: Null when truncated. "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } /v1/runs/{runId}/diagnostics: parameters: - name: runId in: path required: true schema: { type: string, format: uuid } get: operationId: getRunDiagnostics summary: Explain what happened in a research Run description: >- One owner-scoped answer composed from what was recorded: the request and plan, sources, cells, reconciliation, completion, and cost and timing. Each area separates observations (recorded facts), findings (derived by fixed rules), and unknowns (what was not recorded). Nothing is inferred from absence; Runs before recording existed report unknowns rather than guesses. Read-only. tags: [Runs] responses: "200": description: Bounded diagnostics for the latest recorded attempt. headers: X-Credits-Remaining: { $ref: "#/components/headers/CreditsRemaining" } content: application/json: schema: type: object additionalProperties: false required: [diagnostics] properties: diagnostics: { $ref: "#/components/schemas/RunDiagnostics" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } /v1/runs/{runId}/trace: parameters: - name: runId in: path required: true schema: { type: string, format: uuid } get: operationId: getRunTrace tags: [Runs] summary: Get research step timings description: Returns owner-scoped timing metadata from the latest 1000 timing events. Start and finish events collapse into one step. Phase totals sum overlapping work and are not wall time. Older Runs may have no timing events; queueMs and wallMs are null when unavailable or truncated. No prompts, page bodies or raw provider errors are returned. responses: "200": description: Bounded Run trace. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: type: object required: [trace] properties: trace: { $ref: "#/components/schemas/RunTrace" } "401": { $ref: "#/components/responses/Unauthorized" } "404": { $ref: "#/components/responses/NotFound" } "500": { $ref: "#/components/responses/InternalError" } /v1/usage: get: operationId: getUsage tags: [Usage] summary: Get credit usage description: Returns the current balance and ledger entries from the last 30 days. responses: "200": description: Current balance and recent ledger. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: $ref: "#/components/schemas/Usage" "401": { $ref: "#/components/responses/Unauthorized" } "500": { $ref: "#/components/responses/InternalError" } components: securitySchemes: apiKey: type: http scheme: bearer bearerFormat: grava_ description: API key shown once at creation and stored as a SHA-256 hash. headers: CreditsRemaining: description: >- Integer snapshot of the authenticated owner's credit balance. This is 0 when authentication has not succeeded. schema: type: integer parameters: DatasetId: name: datasetId in: path required: true description: Dataset UUID. schema: type: string format: uuid DatasetCursor: name: cursor in: query description: Opaque cursor returned as nextCursor by the previous page. schema: type: string DatasetLimit: name: limit in: query description: Maximum datasets to return. schema: type: integer minimum: 1 maximum: 100 default: 20 RowCursor: name: cursor in: query description: Opaque cursor returned as nextCursor by the previous page. schema: type: string RowLimit: name: limit in: query description: Maximum rows to return. schema: type: integer minimum: 1 maximum: 500 default: 100 schemas: MissingReasons: type: object additionalProperties: type: string enum: [unknown, not_found, not_applicable, extraction_failed] FieldEvidence: type: object additionalProperties: type: array items: type: object required: [observationId, entryId] properties: observationId: { type: string, format: uuid } entryId: { type: string } SourceObservation: type: object required: [id, sourceUrl, observedAt, contentDigest, entries] properties: id: { type: string, format: uuid } datasetId: { type: string, format: uuid } runId: { type: string, format: uuid } localId: { type: string } sourceKey: { type: string } scopeKey: { type: string } pageKey: { type: string } sourceUrl: { type: string, format: uri } observedAt: { type: string, format: date-time } contentDigest: { type: string } method: type: string enum: [deterministic, model] description: Whether evidence came from deterministic extraction or model-assisted research. entries: type: array items: type: object required: [id, excerpt, truncated] properties: id: { type: string } excerpt: { type: string, description: Maximum 2048 UTF-8 bytes. } truncated: { type: boolean } CellOverrides: type: object description: Latest source values for manually overridden fields. The row data contains the displayed manual value. Reset restores this source snapshot. additionalProperties: type: object required: [value, evidence] properties: value: { $ref: "#/components/schemas/DatasetRowValue" } evidence: type: array items: type: object required: [observationId, entryId] properties: observationId: { type: string, format: uuid } entryId: { type: string } missingReason: { type: string, enum: [unknown, not_found, not_applicable, extraction_failed], } RowDetail: type: object required: [ id, datasetId, rowKey, data, revisionId, evidence, missingReasons, presence, observations, ] properties: id: { type: string, format: uuid } datasetId: { type: string, format: uuid } rowKey: { type: string } identity: anyOf: - $ref: "#/components/schemas/ResearchRowIdentity" - type: "null" data: type: object additionalProperties: { $ref: "#/components/schemas/DatasetRowValue" } revisionId: { type: [string, "null"], format: uuid } evidence: { $ref: "#/components/schemas/FieldEvidence" } overrides: { $ref: "#/components/schemas/CellOverrides" } missingReasons: { $ref: "#/components/schemas/MissingReasons" } presence: { type: string, enum: [present, missing] } status: { $ref: "#/components/schemas/DatasetRowStatus" } firstSeenRunId: { type: string, format: uuid } lastSeenRunId: { type: string, format: uuid } sourceUrl: { type: string, format: uri } observedAt: { type: string, format: date-time } updatedAt: { type: string, format: date-time } observations: type: array items: { $ref: "#/components/schemas/SourceObservation" } EditRowRequest: type: object anyOf: - properties: values: { minProperties: 1 } required: [values] - required: [resetFieldIds] required: [expectedRevisionId, requestId, values] properties: expectedRevisionId: { type: string, format: uuid } requestId: { type: string, minLength: 1, maxLength: 128 } values: type: object maxProperties: 50 description: Values keyed by non-identity Field UUID. Supply values or resetFieldIds. additionalProperties: { $ref: "#/components/schemas/DatasetRowValue" } resetFieldIds: type: array minItems: 1 maxItems: 50 uniqueItems: true description: Overridden fields to restore from their latest source value; cannot overlap values. items: { type: string, format: uuid } RowRevision: type: object required: [ id, datasetId, rowId, sequence, schemaVersion, previousRevisionId, origin, kind, changedFieldIds, data, evidence, missingReasons, presence, createdAt, ] properties: id: { type: string, format: uuid } datasetId: { type: string, format: uuid } rowId: { type: string, format: uuid } sequence: { type: integer } schemaVersion: { type: integer } previousRevisionId: { type: [string, "null"], format: uuid } runId: { type: [string, "null"], format: uuid } actorId: { type: [string, "null"], format: uuid } requestId: { type: [string, "null"] } requestHash: { type: [string, "null"] } origin: { type: string, enum: [collection, user, migration] } kind: { type: string, enum: [created, updated, missing, returned, baseline], } changedFieldIds: type: array items: { type: string, format: uuid } data: type: object additionalProperties: { $ref: "#/components/schemas/DatasetRowValue" } evidence: { $ref: "#/components/schemas/FieldEvidence" } overrides: { $ref: "#/components/schemas/CellOverrides" } missingReasons: { $ref: "#/components/schemas/MissingReasons" } presence: { type: string, enum: [present, missing] } createdAt: { type: string, format: date-time } ChangePage: type: object required: [data, schemas, observations, through, nextAfter] properties: observations: type: array items: { $ref: "#/components/schemas/SourceObservation" } through: { type: integer } nextAfter: { type: [integer, "null"] } data: type: array items: allOf: - $ref: "#/components/schemas/RowRevision" - type: object properties: beforeOverrides: { $ref: "#/components/schemas/CellOverrides" } before: type: [object, "null"] additionalProperties: { $ref: "#/components/schemas/DatasetRowValue" } schemas: type: array items: type: object properties: id: { type: string, format: uuid } datasetId: { type: string, format: uuid } version: { type: integer } fields: type: array items: { $ref: "#/components/schemas/DatasetColumn" } createdAt: { type: string, format: date-time } DatasetColumnType: type: string enum: [string, number, date, url, boolean] DatasetColumn: type: object additionalProperties: false required: [name, type] description: Returned columns always include id, label and ownership. Existing IDs, extraction names, types, ownership and identity are immutable in V1. Labels and descriptions are editable. properties: id: type: string format: uuid label: type: string minLength: 1 maxLength: 120 ownership: type: string enum: [collected, user] default: collected name: type: string pattern: "^[a-zA-Z][a-zA-Z0-9_]*$" minLength: 1 maxLength: 80 type: $ref: "#/components/schemas/DatasetColumnType" key: type: boolean default: false description: type: string minLength: 1 maxLength: 500 CreateDatasetColumn: allOf: - $ref: "#/components/schemas/DatasetColumn" - not: required: [id] description: Field IDs are allocated by the application. Do not supply id when creating a Dataset. SourceRead: type: "object" properties: selector: type: "string" minLength: 1 maxLength: 500 kind: default: "text" type: "string" enum: - "text" - "attribute" attribute: type: "string" maxLength: 100 pattern: "^[a-zA-Z_][\\w-]*$" additionalProperties: false allOf: - if: required: [kind] properties: kind: { const: attribute } then: required: [attribute] description: "A read relative to one record container. Omit selector to read the container itself. Attribute reads require attribute. Supported CSS: tags, classes, IDs, attributes, descendant and child combinators." SourceField: type: "object" properties: name: type: "string" minLength: 1 maxLength: 80 pattern: "^[a-zA-Z][a-zA-Z0-9_]*$" read: "$ref": "#/components/schemas/SourceRead" required: default: false type: "boolean" required: - "name" - "read" additionalProperties: false SourceIdentity: oneOf: - type: "object" properties: kind: type: "string" const: "source-key" field: type: "string" minLength: 1 maxLength: 80 pattern: "^[a-zA-Z][a-zA-Z0-9_]*$" required: - "kind" - "field" additionalProperties: false - type: "object" properties: kind: type: "string" const: "composite" fields: minItems: 1 maxItems: 50 uniqueItems: true type: "array" items: type: "string" minLength: 1 maxLength: 80 pattern: "^[a-zA-Z][a-zA-Z0-9_]*$" required: - "kind" - "fields" additionalProperties: false description: "Stable source key or ordered typed composite key, referenced by column extraction name. Fields must be unique, collected, and exactly match the key columns." SourcePagination: oneOf: - type: "object" properties: kind: type: "string" const: "none" required: - "kind" additionalProperties: false - type: "object" properties: kind: type: "string" const: "next-link" selector: type: "string" minLength: 1 maxLength: 500 required: - "kind" - "selector" additionalProperties: false - type: "object" properties: kind: type: "string" const: "url-pattern" pattern: type: "string" minLength: 1 maxLength: 2048 pattern: "^https?://[^{}]*\\{page\\}[^{}]*$" startPage: default: 1 type: "integer" minimum: 0 maximum: 1000000 endSelector: type: "string" minLength: 1 maxLength: 500 required: - "kind" - "pattern" - "endSelector" additionalProperties: false description: "Explicit single-page scope, next-link natural end, or URL pattern with exactly one {page} and an end selector. Initial URL must match pattern at startPage. Click-only pagination and infinite scroll are unsupported." Source: type: "object" properties: version: type: "number" const: 1 url: type: "string" maxLength: 2048 format: "uri" pattern: "^https?://" render: default: "static" type: "string" enum: - "static" - "browser" itemSelector: type: "string" minLength: 1 maxLength: 500 fields: minItems: 1 maxItems: 50 type: "array" items: "$ref": "#/components/schemas/SourceField" identity: "$ref": "#/components/schemas/SourceIdentity" pagination: "$ref": "#/components/schemas/SourcePagination" maxPages: default: 10 type: "integer" minimum: 1 maximum: 50 maxItems: default: 1000 type: "integer" minimum: 1 maximum: 1000 minimumItems: default: 1 type: "integer" minimum: 0 maximum: 1000 proxy: default: "auto" type: "string" enum: - "auto" - "always" - "never" ignoreRobots: default: false type: "boolean" required: - "version" - "url" - "itemSelector" - "fields" - "identity" - "pagination" additionalProperties: false description: "Declarative record-based collection. Every collected column has exactly one relative read; collected types are string, number or boolean. The application allocates permanent Field IDs and pins the parsed Recipe. Complete coverage and passing Checks are required before acceptance and page charging. URL and redirect targets must be public. Existing array-based Recipes are held without navigation; preserve their data and create a new Dataset with explicit list configuration." CreateDatasetRequest: type: object additionalProperties: false required: [name, columns, source] properties: name: type: string minLength: 1 maxLength: 120 question: type: string minLength: 1 maxLength: 1000 columns: type: array minItems: 1 maxItems: 50 items: $ref: "#/components/schemas/CreateDatasetColumn" source: $ref: "#/components/schemas/Source" UpdateDatasetRequest: type: object additionalProperties: false minProperties: 1 description: Column updates require expectedSchemaVersion and preserve every existing Field ID. New columns must have ownership=user. properties: expectedSchemaVersion: type: integer minimum: 1 name: type: string minLength: 1 maxLength: 120 question: oneOf: - type: string minLength: 1 maxLength: 1000 - type: "null" columns: type: array minItems: 1 maxItems: 50 items: $ref: "#/components/schemas/DatasetColumn" status: allOf: - $ref: "#/components/schemas/DatasetStatus" description: Set to `active` to keep the Dataset updated on its schedule, or `paused` to stop scheduled Runs. Manual Runs work in any status. schedule: oneOf: - type: string description: Five-field cron expression. minLength: 9 maxLength: 100 - type: "null" DatasetStatus: type: string enum: [draft, active, paused] RunSummary: type: object additionalProperties: false required: [id, status, completedAt] properties: id: type: string format: uuid status: $ref: "#/components/schemas/RunStatus" completedAt: oneOf: - type: string format: date-time - type: "null" Dataset: type: object additionalProperties: false required: - id - name - question - columns - schemaVersion - status - schedule - rowCount - lastRun - createdAt - updatedAt properties: schemaVersion: type: integer minimum: 1 id: type: string format: uuid name: type: string question: oneOf: - type: string - type: "null" columns: type: array items: $ref: "#/components/schemas/DatasetColumn" status: $ref: "#/components/schemas/DatasetStatus" schedule: oneOf: - type: string - type: "null" rowCount: type: integer minimum: 0 lastRun: oneOf: - $ref: "#/components/schemas/RunSummary" - type: "null" createdAt: type: string format: date-time updatedAt: type: string format: date-time DatasetResponse: type: object additionalProperties: false required: [dataset] properties: dataset: $ref: "#/components/schemas/Dataset" DatasetRowStatus: type: string enum: [new, changed, unchanged, gone] DatasetRowValue: oneOf: - type: string - type: number - type: boolean - type: "null" ResearchRowIdentity: type: object additionalProperties: false required: [version, key, kind, anchor, aliases] properties: version: { type: integer, const: 1 } key: { type: string, pattern: "^entity:[a-f0-9]{64}$" } kind: { type: string, enum: [company, person, product, event, trend, other], } anchor: { type: string, minLength: 1, maxLength: 4096 } aliases: type: array minItems: 1 maxItems: 100 items: { type: string, minLength: 1, maxLength: 200 } DatasetRow: type: object additionalProperties: false required: - id - rowKey - identity - data - revisionId - presence - missingReasons - status - firstSeenRunId - lastSeenRunId - sourceUrl - observedAt - updatedAt properties: identity: description: Evidence-derived research identity, independent of the display name. Null for rows created before entity identity or by deterministic Recipes. anyOf: - $ref: "#/components/schemas/ResearchRowIdentity" - type: "null" revisionId: type: [string, "null"] format: uuid presence: type: string enum: [present, missing] missingReasons: $ref: "#/components/schemas/MissingReasons" id: type: string format: uuid rowKey: type: string data: description: Scalar values keyed by stable Field ID, never display label. type: object additionalProperties: $ref: "#/components/schemas/DatasetRowValue" status: $ref: "#/components/schemas/DatasetRowStatus" firstSeenRunId: type: string format: uuid lastSeenRunId: type: string format: uuid sourceUrl: type: string format: uri observedAt: type: string format: date-time updatedAt: type: string format: date-time RunStatus: type: string enum: [queued, running, succeeded, failed, blocked, held] description: >- `blocked` means the site refused every fetch attempt (a challenge page, 403, 429, a login redirect, or an empty response). Like a failed Check it writes no rows and costs no credits. CheckResult: type: object additionalProperties: false required: [checkId, passed] properties: checkId: type: string description: Stable check identifier, such as identity, extraction, minimum-items, or present:. passed: type: boolean message: type: string description: Explanation of the check outcome when available. RowDeltas: type: object additionalProperties: false required: [new, changed, gone] properties: new: type: integer minimum: 0 changed: type: integer minimum: 0 gone: type: integer minimum: 0 Run: type: object additionalProperties: false required: - id - datasetId - status - checks - rowDeltas - error - queuedAt - startedAt - completedAt properties: id: type: string format: uuid datasetId: type: string format: uuid status: $ref: "#/components/schemas/RunStatus" checks: type: array items: $ref: "#/components/schemas/CheckResult" rowDeltas: $ref: "#/components/schemas/RowDeltas" error: oneOf: - type: string - type: "null" queuedAt: type: string format: date-time startedAt: oneOf: - type: string format: date-time - type: "null" completedAt: oneOf: - type: string format: date-time - type: "null" ResearchPhase: type: string enum: [ planning, configure, pin, search, read, extract, validate, preview, accept, ] RunTraceStep: type: object additionalProperties: false required: [stepId, attemptId, phase, status, startedAt] properties: stepId: { type: string, format: uuid } attemptId: { type: string, format: uuid } phase: { $ref: "#/components/schemas/ResearchPhase" } status: { type: string, enum: [running, succeeded, failed, cancelled] } startedAt: { type: string, format: date-time } durationMs: { type: integer, minimum: 0 } candidateRows: { type: integer, minimum: 0, maximum: 25 } rejectedRows: { type: integer, minimum: 0, maximum: 25 } matchedRows: { type: integer, minimum: 0, maximum: 25, description: Supported rows on this page before cross-page deduplication., } sourceUrl: { type: string, format: uri, maxLength: 2048 } RunTrace: type: object additionalProperties: false required: [ runId, datasetId, status, queueMs, wallMs, truncated, phases, steps, discovery, harnessVersion, ] properties: runId: { type: string, format: uuid } datasetId: { type: string, format: uuid } status: { $ref: "#/components/schemas/RunStatus" } queueMs: oneOf: - { type: integer, minimum: 0 } - { type: "null" } wallMs: oneOf: - { type: integer, minimum: 0 } - { type: "null" } truncated: { type: boolean } phases: type: array maxItems: 9 items: type: object additionalProperties: false required: [phase, count, totalMs, maxMs] properties: phase: { $ref: "#/components/schemas/ResearchPhase" } count: { type: integer, minimum: 0 } totalMs: { type: integer, minimum: 0 } maxMs: { type: integer, minimum: 0 } steps: type: array maxItems: 1000 items: { $ref: "#/components/schemas/RunTraceStep" } harnessVersion: description: Research harness identifier. Null for Runs without a recorded harness version. oneOf: - { type: string, maxLength: 100 } - { type: "null" } discovery: description: >- The latest attempt's search candidates in provider order with what happened to each. Null for Runs recorded before discovery records existed or for explicit-source Runs that never searched. Snippets are never returned. oneOf: - { $ref: "#/components/schemas/RunDiscovery" } - { type: "null" } DiagnosticFinding: type: object additionalProperties: false required: [code, message] properties: code: { type: string, maxLength: 100 } message: { type: string, maxLength: 1000 } DiagnosticUnknowns: type: array maxItems: 20 items: { type: string, maxLength: 1000 } RunDiagnostics: type: object additionalProperties: false required: [ runId, datasetId, status, attemptId, harnessVersion, request, sources, cells, reconciliation, completion, cost, ] properties: runId: { type: string, format: uuid } datasetId: { type: string, format: uuid } status: { $ref: "#/components/schemas/RunStatus" } attemptId: description: The attempt the diagnostics describe; null when nothing was recorded. oneOf: - { type: string, format: uuid } - { type: "null" } harnessVersion: oneOf: - { type: string, maxLength: 100 } - { type: "null" } request: type: object additionalProperties: false required: [observations, unknowns] properties: observations: oneOf: - type: object additionalProperties: false required: [prompt, sourceUrl, plan] properties: prompt: { type: string, maxLength: 1000 } sourceUrl: oneOf: - { type: string, format: uri } - { type: "null" } plan: oneOf: - type: object additionalProperties: false required: [title, rowMeaning, nameLabel, query, attributes] properties: title: { type: string } rowMeaning: { type: string } nameLabel: { type: string } query: { type: string } attributes: type: array maxItems: 5 items: type: object additionalProperties: false required: [fieldId, label, description] properties: fieldId: { type: string, format: uuid } label: { type: string } description: { type: string } - { type: "null" } - { type: "null" } unknowns: { $ref: "#/components/schemas/DiagnosticUnknowns" } sources: type: object additionalProperties: false required: [observations, findings, unknowns] properties: observations: type: object additionalProperties: false required: [discovery, counts, candidates] properties: discovery: oneOf: - type: object additionalProperties: false required: [attemptId, query, depth, excludeDomains, credits] properties: attemptId: { type: string, format: uuid } query: { type: string } depth: { type: string, enum: [basic, advanced] } excludeDomains: type: array items: { type: string } credits: { type: number, minimum: 0 } - { type: "null" } counts: type: object additionalProperties: false required: [discovered, checked, unreadable, unknown, unattempted] properties: discovered: { type: integer, minimum: 0 } checked: { type: integer, minimum: 0 } unreadable: { type: integer, minimum: 0 } unknown: { type: integer, minimum: 0 } unattempted: { type: integer, minimum: 0 } candidates: type: array maxItems: 20 items: type: object additionalProperties: false required: [rank, url, outcome] properties: rank: { type: integer, minimum: 1 } url: { type: string, format: uri } title: { type: string } score: { type: number, minimum: 0 } outcome: type: string enum: [unattempted, unknown, unreadable, checked] findings: type: array items: { $ref: "#/components/schemas/DiagnosticFinding" } unknowns: { $ref: "#/components/schemas/DiagnosticUnknowns" } cells: type: object additionalProperties: false required: [observations, findings, unknowns] properties: observations: type: object additionalProperties: false required: [rowsFromRun, rowsInTable, fields] properties: rowsFromRun: type: integer minimum: 0 description: >- Rows this Run wrote, read from its immutable revisions, so later Runs cannot change this Run's coverage. rowsInTable: type: integer minimum: 0 description: All rows currently in the Dataset, across Runs. fields: type: array maxItems: 5 items: type: object additionalProperties: false required: [fieldId, label, populated, notFound, extractionFailed] properties: fieldId: { type: string, format: uuid } label: { type: string } populated: { type: integer, minimum: 0 } notFound: { type: integer, minimum: 0 } extractionFailed: { type: integer, minimum: 0 } findings: type: array items: { $ref: "#/components/schemas/DiagnosticFinding" } unknowns: { $ref: "#/components/schemas/DiagnosticUnknowns" } reconciliation: type: object additionalProperties: false required: [observations, findings, unknowns] properties: observations: type: object additionalProperties: false required: [rejected, quotesDropped, duplicates] properties: rejected: type: object additionalProperties: { type: integer, minimum: 0 } quotesDropped: type: integer minimum: 0 description: Rows kept on their verbatim quotes after a spliced quote was discarded. duplicates: type: array maxItems: 100 items: type: object additionalProperties: false required: [name, url, winnerUrl, offered, merged] properties: name: { type: [string, "null"] } url: { type: [string, "null"] } winnerUrl: { type: [string, "null"] } offered: type: array items: { type: string } description: Labels of supported fields the duplicate offered that were not merged (lost). merged: type: array items: { type: string } description: Labels of fields merged into the saved row from this duplicate. findings: type: array items: { $ref: "#/components/schemas/DiagnosticFinding" } unknowns: { $ref: "#/components/schemas/DiagnosticUnknowns" } completion: type: object additionalProperties: false required: [observations, findings, unknowns] properties: observations: type: object additionalProperties: false required: [status, stop, error] properties: status: { $ref: "#/components/schemas/RunStatus" } stop: oneOf: - type: object additionalProperties: false required: [rule, inputs] properties: rule: { type: string } inputs: { type: object, additionalProperties: true } - { type: "null" } error: { type: [string, "null"] } findings: type: array items: { $ref: "#/components/schemas/DiagnosticFinding" } unknowns: { $ref: "#/components/schemas/DiagnosticUnknowns" } cost: type: object additionalProperties: false required: [observations, unknowns] properties: observations: type: object additionalProperties: false required: [usage, queueMs, wallMs, phases] properties: usage: oneOf: - type: object additionalProperties: false required: [searchCredits, inputTokens, outputTokens, modelUsd] properties: searchCredits: { type: [number, "null"], minimum: 0 } inputTokens: { type: [integer, "null"], minimum: 0 } outputTokens: { type: [integer, "null"], minimum: 0 } modelUsd: { type: [number, "null"], minimum: 0 } - { type: "null" } queueMs: { type: [integer, "null"], minimum: 0 } wallMs: { type: [integer, "null"], minimum: 0 } phases: type: array maxItems: 9 items: type: object additionalProperties: false required: [phase, count, totalMs, maxMs] properties: phase: { $ref: "#/components/schemas/ResearchPhase" } count: { type: integer, minimum: 0 } totalMs: { type: integer, minimum: 0 } maxMs: { type: integer, minimum: 0 } unknowns: { $ref: "#/components/schemas/DiagnosticUnknowns" } ResearchRecipe: type: object additionalProperties: false required: [kind, version, prompt, identity, fields] properties: kind: { type: string, enum: [research] } version: { type: integer, enum: [1] } prompt: { type: string, maxLength: 1000 } url: { type: string, format: uri, maxLength: 2048 } identity: { type: string, enum: [normalized-name-v1, entity-v1] } fields: type: object additionalProperties: false required: [name, evidence, source] properties: name: { type: string, format: uuid } evidence: { type: string, format: uuid } source: { type: string, format: uuid } plan: type: object additionalProperties: false required: [title, rowMeaning, nameLabel, query, attributes] properties: title: { type: string, maxLength: 100 } rowMeaning: { type: string, maxLength: 300 } rowIdentity: type: object additionalProperties: false required: [kind, description, scopeFields] properties: kind: { type: string, enum: [company, person, product, event, trend, other], } description: { type: string, minLength: 1, maxLength: 300 } scopeFields: type: array maxItems: 5 uniqueItems: true description: Required distinguishing dimensions. Event, trend and other require subject; events also require occurrence. items: { type: string, minLength: 1, maxLength: 50 } if: properties: kind: { const: event } then: properties: scopeFields: allOf: - contains: { const: subject } - contains: { const: occurrence } else: if: properties: kind: { enum: [trend, other] } then: properties: scopeFields: contains: { const: subject } nameLabel: { type: string, maxLength: 50 } query: { type: string, maxLength: 400 } attributes: type: array minItems: 1 maxItems: 5 items: type: object additionalProperties: false required: [fieldId, label, description] properties: fieldId: { type: string, format: uuid } label: { type: string, maxLength: 50 } description: { type: string, maxLength: 300 } RunArtifact: type: object additionalProperties: false required: [ id, attemptId, stepId, kind, sourceUrl, sha256, bytes, truncated, createdAt, ] properties: id: { type: string, format: uuid } attemptId: { type: string, format: uuid } stepId: { type: string, format: uuid } kind: type: string enum: [plan.output, read.page, extract.output] sourceUrl: description: The requested URL for reads, the final URL for extraction. oneOf: - { type: string, format: uri, maxLength: 2048 } - { type: "null" } sha256: { type: string, minLength: 64, maxLength: 64 } bytes: { type: integer, minimum: 0 } truncated: type: boolean description: The body exceeded 64 KiB and was not kept. createdAt: { type: string, format: date-time } body: description: >- Present only on the single-artifact endpoint; null when truncated. RunDiscovery: type: object additionalProperties: false required: [attemptId, stepId, query, depth, excludeDomains, credits, candidates] properties: attemptId: { type: string, format: uuid } stepId: { type: string, format: uuid } query: { type: string, maxLength: 1000 } depth: { type: string, enum: [basic, advanced] } excludeDomains: type: array maxItems: 20 items: { type: string } credits: { type: number, minimum: 0 } candidates: type: array maxItems: 20 items: type: object additionalProperties: false required: [rank, url, outcome] properties: rank: { type: integer, minimum: 1, maximum: 20 } url: { type: string, format: uri, maxLength: 2048 } title: { type: string, maxLength: 200 } score: { type: number, minimum: 0 } outcome: type: string enum: [unattempted, unknown, unreadable, checked] description: >- Joined from source events on the scheduled URL, so redirects do not hide a checked page. "unknown" means the read started but no outcome was recorded (for example the attempt failed during extraction); "unattempted" means no read was recorded at all. CreditReason: type: string enum: [grant.signup, grant.monthly, grant.manual, run, purchase] LedgerEntry: type: object additionalProperties: false required: [id, delta, reason, runId, priceVersion, createdAt] properties: id: type: string format: uuid delta: type: integer reason: $ref: "#/components/schemas/CreditReason" runId: oneOf: - type: string format: uuid - type: "null" priceVersion: type: integer minimum: 1 createdAt: type: string format: date-time UsagePeriod: type: object additionalProperties: false required: [startsAt, endsAt] properties: startsAt: type: string format: date-time endsAt: type: string format: date-time Usage: type: object additionalProperties: false required: [balance, ledger, period] properties: balance: type: integer ledger: type: array items: $ref: "#/components/schemas/LedgerEntry" period: $ref: "#/components/schemas/UsagePeriod" ErrorDetail: type: object additionalProperties: false required: [path, message] properties: path: type: string message: type: string Error: type: object additionalProperties: false required: [error] properties: error: type: object additionalProperties: false required: [code, message] properties: code: type: string enum: - bad_request - unauthorized - insufficient_credits - not_found - conflict - validation_error - internal_error - queue_unavailable message: type: string details: type: array items: $ref: "#/components/schemas/ErrorDetail" responses: BadRequest: description: Malformed JSON, cursor, or query parameters. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: bad_request, message: Invalid cursor } Unauthorized: description: Missing, malformed, unknown, or revoked API key. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: unauthorized, message: A valid API key is required } InsufficientCredits: description: Balance is zero or below, so a run cannot be enqueued. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: insufficient_credits, message: No credits remaining } NotFound: description: Resource does not exist for the authenticated owner. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: not_found, message: Resource not found } Conflict: description: Request conflicts with the current resource state. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: conflict, message: Resource state conflict } ValidationError: description: Body is valid JSON but violates the request contract. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: code: validation_error message: Request validation failed details: - { path: columns.0.name, message: Required } InternalError: description: Unexpected server failure. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: internal_error, message: An unexpected error occurred } QueueUnavailable: description: Run could not be durably enqueued. headers: X-Credits-Remaining: $ref: "#/components/headers/CreditsRemaining" content: application/json: schema: { $ref: "#/components/schemas/Error" } example: error: { code: queue_unavailable, message: Run queue is unavailable }