Using the REST API

General description

Request and response format Except where noted, all endpoints accept a JSON payload. The exceptions are the endpoints that accept a query, which expect a request of type text/plain containing the query in the body as an UTF-8 encoded string.

Security

All Raw REST endpoints are protected by OAuth2 authorization. The caller must first obtain a valid OAuth2 token with the authorization service and then provide it in the “Authorization” header of the request.

Error handling

If there is an error processing a request, the server will return a 400 response with a JSON payload describing the error. The response contains the following fields:

  • errorCode - The unique identifier of the type of error. See below for valid values.

  • message - A simplified message describing the error.

  • detail - Detailed description of the error, intended for debugging. This is not always returned, depends on the configuration of the raw server.

  • extra - Extra detail used in some error responses.

  • errors - Used only to return query errors (see the Query endpoints).

The following are error codes that may be returned:

Error Code

Description

GenericError

An error that does not fit any other category. See the message field for details

AuthenticationError

Credentials provided to access an external resource are not valid (access/secret key of S3 bucket, user/password of RDBMS database, etc).

QueryErrors

Error in a query, syntax or semantic errors.

BadRequestError

The request is badly formed.

AlreadyExistsError

The entity already exists.

NotFoundError

The entity was not found.

RegexError

Error processing a regular expression in a query.

Querying

The following endpoints are used to validate and execute queries.

They all accept the same request structure, differing only in the semantic and how the response is returned to the user. The body of the request contains the query as an UTF-8 encoded string.

The request may contain one or more headers specifying the query options, following this structure:

X-RAW-${queryOption}: <optionValue> For the time being, these are not used.

Validate a query

Validates a query for syntax and semantic errors.

Request:

POST /query-validate

MediaType:

text/plain

Body contains the query in UTF-8 encoding.

Example:
curl -X POST http://localhost:54321/query-validate \
  --header "Authorization: Bearer <access token>" \
  --data 'SELECT * FROM ["Hello", "World"]'

Response:

Returns a JSON object with the result type of the query.

200 OK:
{
    "type": "collection",
    "innerType": { "type": "string"  }
}

If the query is invalid, the server will return a 400 response, with a list of errors in the response.

400 Bad Request - Invalid query response:

{
  "errorCode": "QueryErrors",
  "errors": [
    {
      "message": "expected expression",
      "positions": [
        {
          "begin": {
            "column": 3,
            "line": 1
           },
           "end": {
             "column": 3,
             "line": 1
           }
         }
      ]
    }
  ],
  "extra": {},
  "message": "Errors in query: [QueryError(expected expression: [{begin:(1, 3), end:(1, 3)}])]"
}

Execute a query

Executes a query. If the response is small (typically, less than 16KB but may vary), the response

is a plain HTTP response with the Content-Length header set. Otherwise, if the response is large, it will be streamed using Chunked Enconding transfer.

This is useful for queries returning large results, because the server will start sending the results as soon some of them are available.

Request

POST /query

MediaType:

text/plain

Body contains the query in UTF-8 encoding.

Example:
curl -X POST http://localhost:54321/query \
    --header "Authorization: Bearer <access token>" \
    --data 'SELECT * FROM ["Hello", "World"]'

Response:

The result of the query.

For small responses, a plain HTTP response:

200 OK:
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 16

"Hello"
"World"

For large responses, it uses chunked transfer-encoding:

200 OK:
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked

10
"Hello"
"World"

0

Views

Create a new view

Request

PUT /views/:name

MediaType:

text/plain

Body contains the query in UTF-8 encoding.

Example:
curl -X PUT http://localhost:54321/views/helloworld \
  --header "Content-Type: text/plain" \
  --header "Authorization: Bearer <access token>" \
  --data 'SELECT * FROM ["Hello", "World"]'

Response

{
  "innerType": {
    "type": "string"
  },
  "type": "collection"
}

List views

Request

GET /views?pattern=${pattern}&fields=${fieldNames}

Query parameters:
  • pattern - search pattern following the format used by the SQL LIKE operator (eg., % for zero or more characters, _ for single character)

  • fields - list of fields to return in the response. One or more of: name, query, type, createdAt, updatedAt

Body:

empty

Example:
curl http://localhost:54321/views \
    --header "Authorization: Bearer <access token>" \
    -G --data-urlencode "pattern=hello%" --data-urlencode "fields=name,query,createdAt"
200 OK:
[
  {
    "name": "helloworld",
    "query": "SELECT * FROM [\"Hello\", \"World\"]",
    "createdAt": "2017-11-09T09:54:28.599"
  }
]

Retrieve details of a view

Request:

GET /views/:name

Body:

empty

Example:
curl http://localhost:54321/views/helloworld \
  --header "Authorization: Bearer <access token>"

Response

200 OK:
{
  "name": "helloworld",
  "query": "SELECT * FROM [\"Hello\", \"World\"]",
  "type": {
    "innerType": {
      "type": "string"
    },
    "type": "collection"
  },
  "createdAt": "2017-11-09T09:54:28.599",
  "updatedAt": "2017-11-09T09:54:28.599"
}

Rename a view

Request

POST /views/:name

Arguments:

newName - The new name of the view.

Example:
curl -X POST http://localhost:54321/views/helloworld \
  --header "Authorization: Bearer <access token>" \
  --header "Content-Type: application/json" \
  --data '{"newName": "welcomeview"}'

Response

200 OK:
HTTP/1.1 301 Moved Permanently
Location: /views/welcomeview

Delete a view

Request

DELETE /views/:name

Body:

empty

Delete a view

Example:
curl -X DELETE  http://localhost:54321/views/helloworld \
  --header "Authorization: Bearer <access token>"

Response

Returns “204 No Content” if the view was deleted or if the view did not exist.

User Credentials Management - Dropbox tokens

Register a new dropbox token

Registers a dropbox token, failing if a token is already registered. The token should have been obtained from calling the /oauth2/token endpoint of the Dropbox API.

Request

POST /dropbox-tokens

Arguments:
  • accessToken - The access token.

  • tokenType - Always “bearer”.

  • uid - The user id.

Example:
curl -X POST http://localhost:54322/dropbox-tokens \
  --header "Authorization: Bearer <access token>" \
  --header "Content-Type: application/json" \
  --data '{
  "accessToken": "n5HcoSK_1pUAAAAAAAAMpeqUU5Cuzlm9M0kA3eG4HDNrsgKSFHtyzyrG4Eymz5_W",
  "tokenType": "bearer",
  "uid":"246215797"
}'
Response
  • 201 Created

  • 400 Bad Request

  • AlreadyExistsError - If there is already a dropbox token registered for this user.

Get a dropbox token

Request

GET /dropbox-tokens

Body:

empty

Example:
curl http://localhost:54322/dropbox-tokens \
  --header "Authorization: Bearer <access token>"

Response

200 OK:
{
  "accessToken": "n5HcoSK_1pUAAAAAAAAMpeqUU5Cuzlm9M0kA3eG4HDNrsgKSFHtyzyrG4Eymz5_W",
  "tokenType": "bearer",
  "uid": "246215797"
}

Delete dropbox token

Request

DELETE /dropbox-tokens

Example:
curl -X DELETE http://localhost:54322/dropbox-tokens \
  --header "Authorization: Bearer <access token>"
Response
  • 204 No Content

User Credentials Management - S3 Buckets

Register a new S3 bucket

Request

POST /s3buckets

Arguments:
  • name - Name of S3 bucket

  • credentials - Optional AWS credentials.
    • accessKey -

    • secretKey -

Example:
curl -X POST http://localhost:54322/s3buckets \
  --header "Authorization: Bearer <access token>" \
  --header "Content-Type: application/json" \
  --data '{
  "name": "acme-logs",
  "credentials": {
    "accessKey": "<access key>",
    "secretKey": "<secret key>"
  }
}'
Response
  • 201 Created

List registered S3 buckets

Request

GET /s3buckets

Example:
curl http://localhost:54322/s3buckets \
  --header "Authorization: Bearer <access token>"

Response

200 OK:
[
  {
    "name": "acme-logs",
    "credentials": {
      "accessKey": "<access key>",
      "secretKey": "<secret key>"
    }
  },
  {
    "name": "acme-sensor-data",
    "credentials": {
      "accessKey": "<access key>",
      "secretKey": "<secret key>"
    }
  }
]

List an S3 bucket

Request

GET /s3buckets/<bucketName>

Example:
curl http://localhost:54322/s3buckets/acme-logs \
  --header "Authorization: Bearer <access token>"

Response

200 OK:
{
  "name": "acme-logs",
  "credentials": {
    "accessKey": "<access key>",
    "secretKey": "<secret key>"
  }
}

Delete an S3 bucket

Request

DELETE /s3buckets/<bucketName>

Example:
curl -X DELETE http://localhost:54322/s3buckets/acme-logs \
  --header "Authorization: Bearer <access token>"
Response
  • 204 No Content

User Credentials Management - RDBMS Servers

Register a new RDBMS Server

Request

POST /rdbmsservers

Arguments:
  • name - Unique name used to refer to the database within raw.

  • vendor - Database vendor. Currently only postgresql is supported.

  • host - Host name of the database server.

  • port -

  • database - Name of the database

  • username - [Optional] Used for authenticating with the database server.

  • password - [Optional]

Example:
curl -X POST http://localhost:54322/rdbmsservers \
  --header "Authorization: Bearer <access token>" \
  --header "Content-Type: application/json" \
  --data '{
  "name": "db1",
  "vendor": "postgresql",
  "host": "localhost",
  "port": 5432,
  "database": "db",
  "username": "john",
  "password": "1234"
}'
Response
  • 201 Created

List registered RDBMS Servers

Request

GET /rdbmsservers

Example:
curl http://localhost:54322/rdbmsservers \
  --header "Authorization: Bearer <access token>"

Response

200 OK:
[
  {
    "connectionString": "jdbc:postgresql://localhost:5432/test",
    "name": "db1"
  },
  {
    "connectionString": "jdbc:postgresql://localhost:5432/raw",
    "name": "db2"
  }
]

List an RDBMS Server

Request

GET /rdbmsservers/<serverName>

Example:
curl http://localhost:54322/rdbmsservers/db1 \
  --header "Authorization: Bearer <access token>"

Response

200 OK:
{
  "connectionString": "jdbc:postgresql://localhost:5432/postgres",
  "name": "db1",
  "database": "postgres",
  "host": "localhost",
  "port": 5432,
  "username": "postgres",
  "password": "1234",
  "properties": {}
}

Delete an RDBMS Server

Request

DELETE /rdbmsservers/<serverName>

Example:
curl -X DELETE http://localhost:54322/rdbmsservers/db1 \
  --header "Authorization: Bearer <access token>"

Response * 204 No Content