14. Data Catalog

The following entities can be created in RAW:

  • Views - similar to virtual views in traditional databases;

  • Materialized Views - similar to materialized views in traditional databases, but refreshed “on-demand”;

  • Packages - used to organize code. Similar to Python packages;

  • Tables - storage optimized exclusively for tabular data structures, based on an MVCC algorithm.

[4]:
%load_ext raw_magic
The raw_magic extension is already loaded. To reload it, use:
  %reload_ext raw_magic

The Jupyter client includes magics to list and show details for existing entities. These magics are: * views_list, views_show <name> for views; * materialized_views_list, materialized_views_show <name> for materialized views; * tables_list, tables_show <name> for tables; * packages_list, packages_show <name> for packages;

Some examples:

[11]:
%views_list
airports
airports_1
airports_per_country
view1
[10]:
%views_show airports
SELECT * FROM READ("https://raw-tutorial.s3.amazonaws.com/airports.csv")
[13]:
%materialized_views_list
airports_materialized
trips_materialized
[14]:
%packages_list
package1
package2
package3
[15]:
%packages_show package1
airports := SELECT * FROM READ("https://raw-tutorial.s3.amazonaws.com/airports.csv");

airports_by_country := SELECT Country, * FROM airports GROUP BY Country;

airports_in_city(city: string) := SELECT * FROM airports WHERE City = city;

In addition, the Web Admin UI includes extra search capabilities. In particular, user comments that are part of the source code of a view, materialized view, or of declarations and functions in packages are also searchable.

(These search capabilities are not yet available in the CLI, Python API or Jupyter.)

For instance a view defined as follows:

%%views hello_world

// This is a query that does hello world!...
["Hello", "World"]

… would be visible in the Web Admin UI as:

Data%20Catalog.png

(Note the Comment field.)