API references

Overview

The AGP (Asynchronous Query Processor) API Server provides a RESTful interface for managing asynchronous SQL query executions against a ClickHouse cluster. It supports creating, listing, and tracking query executions, as well as retrieving results. The API is built with OpenAPI, and documentation is accessible via Swagger UI at http://localhost:8888/v1/async/docs/ when the server is running. This reference details the key endpoints, request/response formats, and parameters for the API Server, based on the AGP system described at https://github.com/agnosticeng/agp .

Base URL

All endpoints are relative to the base URL: http://localhost:8888/v1/async.

Authentication

The API requires authentication, typically via API keys or tokens, configured during deployment. Ensure your requests include the appropriate headers (e.g., Authorization: Bearer <token>).

Endpoints

1. Create Execution

Submits a new SQL query for asynchronous execution.

• Method: POST
• Path: /executions
• Request Body:

Example:

{
  "query": "SELECT * FROM iceberg('s3://bucket/hn_posts') LIMIT 5",
  "query_id": "hn_query_001",
  "tier": "premium",
  "settings": {"max_execution_time": 3600}
}

• Response:
  • 201 Created:
    json

    {
      "execution_id": "string",
      "query_id": "string",
      "status": "PENDING",
      "created_at": "2025-10-10T17:36:00Z"
    }

  • 400 Bad Request: Invalid query or parameters.
  • 401 Unauthorized: Missing or invalid authentication.
• Description: Creates an execution, queues it in PostgreSQL, and returns an execution_id for tracking. If query_id is provided and an identical query is in-flight (PENDING or RUNNING), the request is deduplicated.

2. List Executions

Retrieves a list of executions, optionally filtered by query_id.

• Method: GET
• Path: /executions
• Query Parameters:
  • query_id (optional): Filter by specific query ID.
  • status (optional): Filter by status (e.g., PENDING, RUNNING, SUCCEEDED, FAILED, CANCELED).
  • limit (optional, default: 100): Maximum number of results.
  • offset (optional, default: 0): Pagination offset.
• Response:
  • 200 OK:
    json

    [
      {
        "execution_id": "string",
        "query_id": "string",
        "status": "SUCCEEDED",
        "created_at": "2025-10-10T17:36:00Z",
        "completed_at": "2025-10-10T17:36:10Z" (optional),
        "result_url": "string" (optional, for SUCCEEDED executions)
      }
    ]

  • 400 Bad Request: Invalid query parameters.
  • 401 Unauthorized: Missing or invalid authentication.
• Description: Returns a paginated list of executions with their statuses and, for completed executions, a URL to retrieve results from the object store.

3. Get Execution Status

Polls the status of a specific execution.

• Method: GET
• Path: /executions/{execution_id}
• Path Parameters:
  • execution_id: Unique identifier of the execution (required).
• Response:
  • 200 OK:
    json

    {
      "execution_id": "string",
      "query_id": "string",
      "status": "RUNNING",
      "created_at": "2025-10-10T17:36:00Z",
      "progress": {
        "rows_processed": 1000,
        "bytes_processed": 1048576
      },
      "result_url": "string" (optional, for SUCCEEDED executions)
    }

  • 404 Not Found: Execution ID does not exist.
  • 401 Unauthorized: Missing or invalid authentication.
• Description: Retrieves the current status (PENDING, RUNNING, CANCELED, FAILED, SUCCEEDED), progress metrics (e.g., rows processed), and result URL if available.

4. Cancel Execution

Cancels a PENDING or RUNNING execution.

• Method: POST
• Path: /executions/{execution_id}/cancel
• Path Parameters:
  • execution_id: Unique identifier of the execution (required).
• Response:
  • 200 OK:
    json

    {
      "execution_id": "string",
      "status": "CANCELED"
    }

  • 404 Not Found: Execution ID does not exist.
  • 409 Conflict: Execution is already completed or canceled.
  • 401 Unauthorized: Missing or invalid authentication.
• Description: Marks the execution as CANCELED, stopping further processing if possible.

5. Retrieve Execution Results

Fetches the results of a completed execution.

• Method: GET
• Path: /executions/{execution_id}/results
• Path Parameters:
  • execution_id: Unique identifier of the execution (required).
• Response:
  • 200 OK: Returns the query results (format depends on ClickHouse output, e.g., JSON, CSV).
    json

    [
      {"column1": "value1", "column2": 123}
    ]

  • 404 Not Found: Execution ID does not exist or results are unavailable/expired.
  • 410 Gone: Results have expired in the object store.
  • 401 Unauthorized: Missing or invalid authentication.
• Description: Retrieves results from the object store for SUCCEEDED executions. Results are available until expiration, as managed by the Bookkeeper.

Error Handling

The API returns standard HTTP status codes with JSON error responses:

{
  "error": "string",
  "details": "string" (optional)
}

• 400: Invalid request parameters or query syntax.
• 401: Authentication failure.
• 404: Resource not found.
• 409: Conflict (e.g., attempting to cancel a completed execution).
• 410: Expired results.
• 500: Internal server error.

Notes

• Result Storage: Results are stored in an object store (e.g., S3) or local filesystem, with expiration managed by the Bookkeeper.
• Tier-Based Allocation: Execution priority and resource limits (e.g., CPU, timeout) depend on the tier specified during creation.
• Deduplication: Queries with the same query_id are collapsed to avoid redundant execution, enhancing efficiency.
• Swagger UI: Access detailed API documentation at http://localhost:8888/v1/async/docs/ when the server is running.

For further details, refer to the AGP repository or the OpenAPI specification (internal/api/v1/async/api.yaml).