14. Data Catalog

The following entities can be created in RAW:

  • Views - similar to virtual views in traditional databases;

  • Materialized Views - similar to materialized views in traditional databases, but refreshed “on-demand”;

  • Packages - used to organize code. Similar to Python packages;

  • Tables - storage optimized exclusively for tabular data structures, based on an MVCC algorithm.

%load_ext raw_magic
The raw_magic extension is already loaded. To reload it, use:
  %reload_ext raw_magic

The Jupyter client includes magics to list and show details for existing entities. These magics are: * views_list, views_show <name> for views; * materialized_views_list, materialized_views_show <name> for materialized views; * tables_list, tables_show <name> for tables; * packages_list, packages_show <name> for packages;

Some examples:

%views_show airports
SELECT * FROM READ("https://raw-tutorial.s3.amazonaws.com/airports.csv")
%packages_show package1
airports := SELECT * FROM READ("https://raw-tutorial.s3.amazonaws.com/airports.csv");

airports_by_country := SELECT Country, * FROM airports GROUP BY Country;

airports_in_city(city: string) := SELECT * FROM airports WHERE City = city;

In addition, the Web Admin UI includes extra search capabilities. In particular, user comments that are part of the source code of a view, materialized view, or of declarations and functions in packages are also searchable.

(These search capabilities are not yet available in the CLI, Python API or Jupyter.)

For instance a view defined as follows:

%%views hello_world

// This is a query that does hello world!...
["Hello", "World"]

… would be visible in the Web Admin UI as:


(Note the Comment field.)