What is RAW?
Overview
RAW is a software platform to enable fast and easy data delivery and sharing via APIs. RAW allows a Data Producer to:
- Query multiple data sources including: files, databases and APIs
- Integrate and transform
- Deliver and manage the data product output as APIs
- Share access to the APIs with any number of Data Consumers
RAW runs in the Cloud using a scalable infrastructure that is fully managed, so that you can just concentrate on delivering business functionality. RAW is built using DataOps principles that facilitate greater quality and decreased delivery time and faster iterations.
Create a query using RAW SQL, Configure an API endpoint using the query in a YAML file. Push both to your GitHub repository, and your API is ready via our DataOps infrastructure.
System Components
RAW SQL - A SQL-based declarative language for querying and manipulating data. RAW SQL has numerous benefits over standard SQL, including better support for non-relational, complex and nested data structures, i.e. JSON, XML. With RAW SQL you can query APIs and Log files together with databases in a single query without having to land data and transform it first. For more information see our RAW SQL Reference Guide.
YAML API Configuration - RAW uses a simple YAML file mechanism to configure API endpoints to execute RAW SQL queries. Security, Resource Groups and Metadata are defined. For more information see our YAML Reference Guide
VS Code Development Environment - RAW uses the popular Microsoft Visual Studio Code, as an IDE to enable RAW SQL query code to be developed, APIs to be configured and executed from within a single place. For more information see our RAW VS Code Extension Reference Guide
RAW DataOps Environment - RAW harnesses GitHub for code management and DevOps using GitHub Actions for CI/CD. RAW syncs with your GitHub Repository automatically so that APIs are delivered immediately on a Git Push, including any branches for testing purposes For more information see our GitHub Integration Reference Guide
RAW Execution Engine - A cloud-based query engine designed for scale-out execution for distributed, heterogenous data and complex data types. Based on unique monoid comprehension calculus, the engine is built with state-of-the-art 'JIT' code generation techniques, creates high-performance code employing data and code caches and associated algorithms.
Resource Groups - RAW Execution Engines are available in multiple sizes using cloud infrastructure to ensure that the needs of the RAW SQL query can be matched better with the class of service. Resource Groups include appropriate CPU, Memory and Storage and are categorized as Standard(S), Large(L) and Extra-Large(XL)
RAW Admin UI - A web-based administration portal for Producers. Here you can monitor your Repositories/APIs, administer users and share data access, as well as view logs for access, resource usage, Git sync, etc. For more information see our Admin Console Reference Guide
RAW Catalog UI - A web portal displaying the API Catalog for authenticated Consumers to browse. For more information see our Catalog Console Reference Guide
RAW System Component Diagram
Users
Users of a single RAW environment are members belonging to an Organization in RAW. An Organization may have multiple Repositories, which are Git-based. Members may assume different Roles, assigned by the Administrator:
Global Roles and Permissions
These roles are at the Organizational level:
Role | Permissions |
---|---|
Billing Manager | Manage all aspects of Billing |
User Manager | Manage Users within the Organization |
Repository Manager | Manage Repositories within the Organization |
Administrator | All permissions |
Local Roles and Permissions
These roles are per Repository:
Role | Permissions |
---|---|
Repository Manager | All permissions within a Repository |
Credentials Manager | Manage Data Source credentials within a Repository |
Developer | Develop, Run and Test Queries within a Repository |
Consumer Roles
Consumers are users with execution permissions. They see all endpoints of the organization in the current model.