Skip to main content

Location

Library of functions to obtain information on remote locations.

Functions​

Describe​

Describes a location.

Syntax:
Location.Describe(location: location, sampleSize: optional int)
Parameters​

  • location: The location to describe.

  • sampleSize: The number of data units to sample from the location for the inference. If set to negative value, the whole file will be read.

Returns​
  • record(format: string, comment: string, type: string, properties: list(record(name: string, value: string)), is_collection: bool, columns: list(record(col_name: string, col_type: string, nullable: bool)), sampled: bool): A record describing the location.
Example:

Location.Describe("http://.../dataset.json")



Location.Describe("http://.../dataset.json", sampleSize = 1000)

FromString​

Transforms an url to a location.

Syntax:
Location.FromString(url: string)
Parameters​
  • url: The url to resolve to a location.
Returns​
  • location: The location object.
Example:

Location.FromString("s3://my-bucket/folder/file")

Details​

List of supported url protocols:

  • s3 e.g. 's3://my-bucket/folder/file'.
  • http/https e.g 'https://example.com/service'.
  • dropbox e.g. 'dropbox://credential/my-folder/file'.

Ll​

Lists a location but returns also metadata information per entry. The location must be a file system or an S3 bucket.

Syntax:
Location.Ll(location: location)
Parameters​
  • location: The location to list.
Returns​
  • list(record(url: string, metadata: record(modified: timestamp, size: int, blocks: list(record(hosts: list(string), offset: long, length: long))): The list of files in the location with metadata information.
Example:
Location.Ll("s3://my-bucket/folder/")

Details​

URLs with wildcards are also supported. For information about the use of wildcards see the Locations with wildcards documentation.

Ls​

Lists a location. The location must be a file system or an S3 bucket.

Syntax:
Location.Ls(value: location)
Parameters​
  • value: The location to list.
Returns​
  • list(string): The list of files in the location.
Example:
Location.Ls("s3://my-bucket/folder/")

Details​

Urls with wildcards are also supported. For information about the use of wildcards see the Locations with wildcards documentation.