Securing endpoints

There are two types of APIs in RAW:

• public APIs, which can be accessed by anyone in the public, without having to be authenticated or authorized;
• private APIs, which required the user to be authenticated and authorized.

This property can be chosen in the endpoint settings.

note

Public APIs are truly public: anyone can use them without your intervention. Since you may be charged for their usage always ensure that you truly want to create a public API and that the URL is only known to relevant parties.

Scopes

For private APIs, scopes can be used to provide fine-grained security.

Scopes are essentially a list of strings you create. You can associate scopes to Users and to APIs keys, and then to endpoints. When a user or an API key consumes an endpoint, the platform checks whether the list of scopes required by the endpoint matches the list of scopes associated with the user or API key.

For instance, if a user or API key has scopes sales and marketing, while the endpoint requires scope sales, then the user or API key can access the endpoint. If a new user with scope hr tries to reach the endpoint, access will be denied. Scopes are lists of free-form strings, so you can organize them anyway you prefer.

Filter data dynamically based on scopes

It is possible to access the scopes of the caller using the built-in Snapi function Environment.Scopes.

For example the following function will return a different output depending if the user calling it has the scope "special" in his permissions.

special() =
    if (List.Contains(Environment.Scopes(), "special"))
    then "you are special"
    else "you are not special"

This can be particularly helpful to build API services that hide, modify, filter or otherwise change their behaviour based on the specific set of scopes the caller (user or API key) has. For instance, you can build a service that hides certain rows or columns of data based on the scopes the user has, implementing a form of fine-grained security.

Example: Filtering data using scopes

Let us illustrate how to create an API where the data displayed will depend on the scopes/permissions of the user calling the endpoint.

For this example we read a log file which has 3 levels of log statements INFO, WARN and ERROR. The data access rules are:

• Users without relevant scopes can only see INFO statements.
• Users with the scope monitoring can see INFO and WARN statements.
• Users with the scope admin scope can see all statements.

info

If you want to try this example, you can deploy the following endpoint:

Changing output of data depending of user scopes

Parse a log file and show subset of data depending of user scopes.

Overview

This example illustrates how to create a REST API where the data displayed will depend on the scopes/permissions of the user calling the endpoint.

For this example we will read a log file which has 3 levels of log statements INFO, WARN and ERROR. The data access rules are:

• Users without relevant scopes can only see INFO statements.
• Users with the scope monitoring can see INFO and WARN statements.
• Users with the scope admin scope can see all statements.

Here's an excerpt of the log file:

2015-01-01T05:54:15 WARN vibration close to treshold, check instrumentation panel ASAP.
2015-01-01T05:54:58 INFO calibration at 100%, checking inner sub-systems.
2015-01-01T05:55:41 ERROR voltage not measured for more than 25 seconds, reboot machine.
2015-01-01T05:56:24 INFO cleaning procedure schedulled soon, performing sub task 111.
2015-01-01T05:57:07 INFO task 155 schedulled soon, preparing next task.

Parsing the file

We can use String.ReadLines and Regex.Groups to extract the timestamp, level and message from each line of text:

parse() = 
    let
        lines = String.ReadLines(
            "s3://raw-tutorial/ipython-demos/predictive-maintenance/machine_logs.log"
        ),
        parsed = Collection.Transform(lines, l ->
            let
                groups = Regex.Groups(l,"""(\d+-\d+-\d+T\d+:\d+:\d+) (\w+) (.*)"""),
                timestamp = Timestamp.Parse(List.Get(groups, 0),"yyyy-M-d\'T\'H:m:s"),
                level = List.Get(groups, 1),
                message = List.Get(groups, 2)
            in
                {timestamp: timestamp, level: level, message: message}
        )
    in
        parsed

The output of this function looks like this:

[
  {
    "timestamp": "2015-01-01T05:54:15.000",
    "level": "WARN",
    "message": "vibration close to treshold, check instrumentation panel ASAP."
  },
  {
    "timestamp": "2015-01-01T05:54:58.000",
    "level": "INFO",
    "message": "calibration at 100%, checking inner sub-systems."
  },
  {
    "timestamp": "2015-01-01T05:55:41.000",
    "level": "ERROR",
    "message": "voltage not measured for more than 25 seconds, reboot machine."
  },
  {
    "timestamp": "2015-01-01T05:56:24.000",
    "level": "INFO",
    "message": "cleaning procedure schedulled soon, performing sub task 111."
  },
  {
    "timestamp": "2015-01-01T05:57:07.000",
    "level": "INFO",
    "message": "task 155 schedulled soon, preparing next task."
  }
]

Usage

/aws/s3/logs-scopes

Code

main() =
    let
        lines = String.ReadLines("s3://raw-tutorial/ipython-demos/predictive-maintenance/machine_logs.log"),
        parsed = Collection.Transform(
            lines,
            (l) ->
                let
                    groups = Regex.Groups(l, "(\\d+-\\d+-\\d+T\\d+:\\d+:\\d+) (\\w+) (.*)"),
                    timestamp = Timestamp.Parse(List.Get(groups, 0), "yyyy-M-d\'T\'H:m:s"),
                    level = List.Get(groups, 1),
                    message = List.Get(groups, 2)
                in
                    {timestamp: timestamp, level: level, message: message}
        )
    in
        Collection.Filter(
            parsed,
            (x) ->
                if List.Contains(Environment.Scopes(), "admin") then
                    true
                else
                    if List.Contains(Environment.Scopes(), "monitoring") then
                        x.level == "WARN" or x.level == "INFO"
                    else
                        x.level == "INFO"
        )

Here's an excerpt of the log file:

2015-01-01T05:54:15 WARN vibration close to treshold, check instrumentation panel ASAP.
2015-01-01T05:54:58 INFO calibration at 100%, checking inner sub-systems.
2015-01-01T05:55:41 ERROR voltage not measured for more than 25 seconds, reboot machine.
2015-01-01T05:56:24 INFO cleaning procedure schedulled soon, performing sub task 111.
2015-01-01T05:57:07 INFO task 155 schedulled soon, preparing next task.

Parsing the file

We can use String.ReadLines and Regex.Groups to extract the timestamp, level and message from each line of text:

parse() = 
    let
        lines = String.ReadLines(
            "s3://raw-tutorial/ipython-demos/predictive-maintenance/machine_logs.log"
        ),
        parsed = Collection.Transform(lines, l ->
            let
                groups = Regex.Groups(l,"""(\d+-\d+-\d+T\d+:\d+:\d+) (\w+) (.*)"""),
                timestamp = Timestamp.Parse(List.Get(groups, 0),"yyyy-M-d\'T\'H:m:s"),
                level = List.Get(groups, 1),
                message = List.Get(groups, 2)
            in
                {timestamp: timestamp, level: level, message: message}
        )
    in
        parsed

The output of this function looks like this:

[
  {
    "timestamp": "2015-01-01T05:54:15.000",
    "level": "WARN",
    "message": "vibration close to treshold, check instrumentation panel ASAP."
  },
  {
    "timestamp": "2015-01-01T05:54:58.000",
    "level": "INFO",
    "message": "calibration at 100%, checking inner sub-systems."
  },
  {
    "timestamp": "2015-01-01T05:55:41.000",
    "level": "ERROR",
    "message": "voltage not measured for more than 25 seconds, reboot machine."
  },
  {
    "timestamp": "2015-01-01T05:56:24.000",
    "level": "INFO",
    "message": "cleaning procedure schedulled soon, performing sub task 111."
  },
  {
    "timestamp": "2015-01-01T05:57:07.000",
    "level": "INFO",
    "message": "task 155 schedulled soon, preparing next task."
  }
]

Filtering by scope

Now that we have parsed the file we can start implementing our data restriction rules. We use the built-in function Environment.Scopes to get the calling user or API key scopes.

This allows us filter the data accordingly:

Collection.Filter(
    parsed,
    (x) ->
        if List.Contains(Environment.Scopes(), "admin")
            then true
        else if List.Contains(Environment.Scopes(), "monitoring") 
            then x.level == "WARN" or x.level == "INFO"
        else
            x.level == "INFO"
)

Here is all the code together:

main() =
    let
        lines = String.ReadLines(
            "s3://raw-tutorial/ipython-demos/predictive-maintenance/machine_logs.log"
        ),
        parsed = Collection.Transform(
            lines,
            (l) ->
                let
                    groups = Regex.Groups(
                        l,
                        "(\\d+-\\d+-\\d+T\\d+:\\d+:\\d+) (\\w+) (.*)"
                    ),
                    timestamp = Timestamp.Parse(
                        List.Get(groups, 0),
                        "yyyy-M-d\'T\'H:m:s"
                    ),
                    level = List.Get(groups, 1),
                    message = List.Get(groups, 2)
                in
                    {timestamp: timestamp, level: level, message: message}
        )
    in
        Collection.Filter(
            parsed,
            (x) ->
                if List.Contains(Environment.Scopes(), "admin")
                    then true
                else if List.Contains(Environment.Scopes(), "monitoring") 
                    then x.level == "WARN" or x.level == "INFO"
                else
                    x.level == "INFO"
        )

In this example, we can create API keys or invite users, and assign them the scope monitoring or admin and set the expiration time, e.g. one month. These clients can then see ERROR or WARN messages only for 1 month.