Using the Command-Line Client

After installation open a terminal run the following command:

$ rawcli health

This will tell you if the server is running and if the configuration settings are correct.

you can get help on any command by passing the -h or --help command line argument.

For example to list all available commands type:

$ rawcli --help

To get help on any sub-command type:

$ rawcli  <command> --help

Quick Guide

In this two-step quick guide, we will register an S3 bucket in RAW - so that we can access its data - and then run a query against a JSON file stored in that bucket.

The S3 bucket and the data used in this example are public, so feel free to repeat the steps. Note that S3 buckets must be registered in RAW even if they are public. In this case however, there no access/secret keys required.

Step 1: Registering the S3 bucket:

$ rawcli buckets-register rawlabs-public-test-data eu-west-1

Step 2: Running a query against S3 file:

$ rawcli query 'select * from read("s3://rawlabs-public-test-data/patients.json")'

Obtaining the output

Obtaining the output of the query above as a JSON document:

$ rawcli query -f json 'select * from read("s3://rawlabs-public-test-data/patients.json")'

Obtaining the output of the query above as SQLite database and saving it to a file.

$ rawcli query -f sqlite 'select * from read("s3://rawlabs-public-test-data/patients.json")' > patients.sqlite
$ sqlite3 patients.sqlite
SQLite version 3.22.0 2018-01-22 18:45:57
Enter ".help" for usage hints.
sqlite> SELECT * FROM output;

To know the list of available output formats, run:

$ rawcli query -h

… and check option -f.

Running large queries

Running a RQL query that is stored in a file:

$ rawcli query -i query.sql

Credentials

Registering S3 bucket with credentials, where ‘access_key’ and ‘secret_key’ are the S3 credentials:

$ rawcli buckets-register my-private-bucket eu-west-1 access_key secret_key

List registered S3 buckets:

$ rawcli buckets-list

Similarly, look into rawcli -h for information on how to store Dropbox or relational database credentials.

Kafka

To create a kafka consumer using raw use the kafka-create command. The positional arguments are:

name

Name of the Kafka table to create.

host

Kafka hostname.

port

Kafka port.

topic

Kafka topic.

interval

Polling interval (in milliseconds).

location

Location to store data.

query

Query to process kafka events.

Optional arguments:

-I INDEXES, –indexes INDEXES

Indexes to create

-i INPUT_FILE, –input-file INPUT_FILE

Input file containing a query.

for example

$ rawcli kafka-create my_table kafka-host 1234 my_topic 1000 hdfs://name-node/some_location --input-file query.rql

Example of a query to process kafka events

\(key: binary, value: binary, topic: string, partitionNo: int, offset: long, timestamp: timestamp, timestampType: int) -> {
    typealias t := record(name: string, age: int);
    parse_json[t](decode_string(value, "utf-8"))
}