2.5 Modules

As programs get larger, it becomes intractable to manage the entirety of a program in a single file: we often want to separate programs into independent pieces. Doing so has a number of benefits, giving us the abilities to:

• provide library code to students (unseen to them).

• logically separate responsibilities of a program into different files to make it easier for programmers to understand the code’s layout.

• provide a boundary over which it’s relatively simple to substitute one implementation of a library for another.

• provide a boundary where the two sides may be implemented in different languages, but can still share values.

• provide only a subset of defined names to a different part of a program.

• manage names when the same (good) names may be in use in different parts of the same program.

• support incremental compilation.

Not all of these require a module system, but a module system can help with all of them, and these use cases (and others) motivate Pyret’s.

This section describes the components of Pyret’s module system and how to use it.

2.5.1 Quick Start

The shortest way to get started using the module system is to understand three key ideas:

• How to provide names from a module

• How to tell Pyret to locate one module from a different module

• How to include names from a located module

Here’s a simple example that demonstrates all three.

In a file called list-helpers.arr:

provide: * end

concat :: <A> List<A>, List<A> -> List<A>
fun concat(l1, l2):
  l.append(l1, l2)
end

In a file in the same directory called list-user.arr:

include file("list-helpers.arr")

check:
  concat([list: 1, 2], [list: 3, 4]) is [list: 1, 2, 3, 4]
end

The provide: * end declaration tells Pyret to make all the names of functions and values defined in the list-helpers.arr module available to any module that imports it. The include file("list-helper.arr") declaration tells Pyret to find the module at the path given in quotes, and then make all the names it provides available in the top-level of the module the include appears in.

In general, include and provide: * end are handy ways to provide a collection of definitions to another context across modules.

Note that provide: * end only provides values. If you want to provide type definitions, then you may use a related declaration:

In a file called mypos-provider.arr:

provide:
  *,       # This line provides `pos2d`, `pos3d`, `is-pos2d` and `is-pos3d`
  type *   # This line provides the type named `MyPos`
end

data MyPos:
  | pos2d(x, y)
  | pos3d(x, y, z)
end

favorite-spot = pos3d(3, 4, 5)

In a file in the same directory called mypos-user.arr:

include file("mypos-provider.arr")

cases(MyPos) favorite-spot:
  | pos2d(x, y) => "Two dimensions"
  | pos3d(x, y, z) => "Three dimensions"
end

The type * declaration is needed for the cases expression to recognize the name MyPos as a type name.

2.5.2 Finding Modules

Each import or include statement indicates a dependency of the current module on some other module. The syntax of each import and include statement tells Pyret how to find the module.

There are currently five types of dependencies supported, though the compiler can be configured to support some, all, or other types of dependency; for example, the gdrive dependencies (below) only work in code.pyret.org, where it is assumed the user is authenticated to Google Drive.

The meaning of the currently supported forms are:

include string-dict

include file("path/to/a/file.arr")

include js-file("path/to/a/file.arr.js")

include my-gdrive("stored-in-gdrive.arr")

include shared-gdrive("public-in-gdrive.arr", "ABCDEF12345")

2.5.3 Detailed Control of Names

In larger programs, or in more sophisticated libraries for students, it is often useful to have quite precise control over which names are provided and included across module boundaries. A programmer may want to provide only a subset of the names defined in a module to maintain an abstraction, or to avoid cluttering namespaces with definitions intended only for use internal to a module.

To this end, Pyret supports several forms for controlling names of various kinds:

• Importing another module and referring to it by name. This approach for name control is simple but syntactically heavyweight on the importing side.

• More precisely exporting only a subset of the values defined in a module

• Providing other kinds of definitions besides values

• More precisely importing only a subset of the values provided by a module

• Importing other kinds of definitions besides values

2.5.4 import and Module Identifiers

In Quick Start we showed provide: * end and include <dependency> as a quick way to get names from one module to another. This is convenient and often a good choice. However, there are situations where this is inadequate. For example, what if we wish to use functions from two different list-helper libraries, but some of the names overlap?

Consider:

# list-helpers.arr
provide: * end
fun concat(l1, l2): l1.append(l2) end
fun every-other(l): ... end

# list-helpers2.arr
provide: * end
concat :: <A> List<List<A>> -> List<A>
fun concat(list-of-lists):
  for fold(acc from empty, l from list-of-lists): acc.append(l) end
end
fun is-odd-length(l): ... end

# in a separate file
include file("list-helpers.arr")
include file("list-helpers2.arr")

concat(???)

In this example, the name concat could have one of two meanings. Since this is ambiguous, this program results in an error that reports the conflict between names.

Neither of the list-helpers modules is wrong: they each define a function named concat that is internally consistent within each helper module. Instead, the client module that uses both helpers simply needs more control in order to use the right behavior from each. One way to get this control is to use import, rather than include, which allows the programmer to give a name to the imported module. This name can then be used with . to refer to the names within the imported module.

import file("list-helpers.arr") as LH1
import file("list-helpers2.arr") as LH2

check:
  LH1.concat([list: 1, 2], [list: 3, 4]) is [list: 1, 2, 3, 4]
  LH2.concat([list: [list: 1, 2], [list: 3]]) is [list: 1, 2, 3]
end

Using import to define a module identifier is a simple way to unambiguously refer to individual modules’ exported names, and avoids conflicting names. It is always a straightforward way to resolve these ambiguities.

For situations where these issues become too onerous, Pyret provides more ways to control names.

2.5.5 Providing Fewer (and More) Names

It is not required that a module provide all of its defined names. To provide fewer names than provide: * end, a module can use one or more provide blocks. The overall set of features allowed is quite broad, and simple examples follow:

‹provide-block›: provide: [‹provide-spec› (, ‹provide-spec›)* [,]] end | ... ‹provide-spec›: ‹provide-value-spec› | ... ‹name-spec›: * [‹hiding-spec›] | ‹module-ref› | ‹module-ref› as NAME ‹provide-value-spec›: ‹name-spec› ‹hiding-spec›: hiding ( [(NAME ,)* NAME] ) ‹module-ref›: (NAME .)* NAME

A provide block contains one or more provide specifications indicating what to names supply, and any module may specify zero or more provide blocks as needed. For now we focus on just one category of provide specifications, which provide names of values (functions, variables, etc.).

• A module might provide all of its values:

  provide:
    *
  end

• A module might define several names of values, and only provide a few:

  # A module that includes this one will only see concat and is-odd-length, and
  # won't see concat-helper because it is not provided
  provide:
    concat,
    is-odd-length
  end
  
  fun concat-helper(element, lst): ... end
  fun concat(list1, list2): ... end
  fun is-odd-length(l): ... end

• A module might provide all of its values and exclude a few:

  # This module provides the same exports as the one above
  provide:
    * hiding (concat-helper)
  end
  
  fun concat-helper(element, lst): ... end
  fun concat(list1, list2): ... end
  fun is-odd-length(l): ... end

• A module might rename some of the values it exports:

  # This module provides two names: is-odd-length and append
  provide:
    * hiding (concat-helper, concat),
    concat as append
  end
  
  fun concat-helper(element, lst): ... end
  fun concat(list1, list2): ... end
  fun is-odd-length(l): ... end

2.5.5.1 Re-exporting values

A module can also re-export values that it imported, and it can do so using module ids:

‹provide-block›: ... | provide from ‹module-ref› : [‹provide-spec› (, ‹provide-spec›)* [,]] end

For example, this module exports both one name it defines, and all the names from string-dict:

import string-dict as SD
provide from SD: * end
provide: my-string-dict-helper end
fun my-string-dict-helper(): ... end

Combining provides from multiple modules can be an effective way to put together a library for students. For example, an introductory course in data science may benefit from a helper library that gives access to the image, chart, and table libraries:

import tables as T
import chart as C
import image as I
provide from T: * end
provide from C: * end
provide from I: * end

A student library that includes this module would have access to all of the names from these three modules.

2.5.6 Providing more than just values

Modules can give names to other things besides values: they may define new types or new datatypes, or they may import another module and give it a name. The syntax above can be used to provide them as well.

2.5.6.1 Types

‹provide-spec›: ... | ‹provide-type-spec› | ... ‹provide-type-spec›: type ‹name-spec›

Providing a type definition is analogous to providing a value definition:

# A module that includes this one will see the name NumPair as a type
provide:
  type NumPair
end

type NumPair = {Number; Number}

Just as with ‹provide-value-spec› above, a ‹provide-type-spec› can use type * as shorthand to supply all of its types, optionally hiding some of them, or renaming some types.

2.5.6.2 Modules

‹provide-spec›: ... | ‹provide-module-spec› | ... ‹provide-module-spec›: module ‹name-spec›

Providing a module id is also quite similar

# A module that includes this one will see the name SD as a module id,
# just as if it had written `import string-dict as SD` itself.
provide:
  module SD
end

import string-dict as SD

2.5.6.3 Data definitions

‹provide-spec›: ... | ‹provide-data-spec› ‹provide-data-spec›: data ‹data-name-spec› [‹hiding-spec›] ‹data-name-spec›: * | ‹module-ref›

Providing a data definition is more sophisticated, since data definitions implicitly define types and values. The following two programs are equivalent in meaning:

provide:
  data MyPosn
end

data MyPosn:
  | pos2d(x, y)
  | pos3d(x, y, z)
end

means the same thing as the much longer

provide:
  # constructors
  pos2d, pos3d,
  # type tester
  is-MyPosn,
  # variant testers
  is-pos2d, is-pos3d,
  # the type declaration itself
  type MyPosn
end

data MyPosn:
  | pos2d(x, y)
  | pos3d(x, y, z)
end

Similarly to how we could hide some names before, providing data definitions also permits hiding names:

provide:
  data MyPosn hiding (pos2d, pos3d)
end

data MyPosn:
  | pos2d(x, y)
  | pos3d(x, y, z)
end

will not provide the constructors for this data definition. Clients will therefore not be able to construct new values of this data type, but they will still be able to manipulate any values they do receive. This can be useful in combination with providing other functions that do construct these values. For instance, the following program will only allow clients to construct points with positive coordinates, by combining providing data, hiding some elements of it, and providing other values and renaming them:

provide:
  data MyPosn hiding (pos2d, pos3d)
  smart-pos2d as pos2d,
  smart-pos3d as pos3d
end

data MyPosn:
  | pos2d(x, y)
  | pos3d(x, y, z)
end

fun smart-pos2d(x, y):
  pos2d(num-max(x, 0), num-max(y, 0))
end
fun smart-pos3d(x, y, z):
  pos2d(num-max(x, 0), num-max(y, 0), num-max(z, 0))
end

Clients of this module will see the "smart" versions of these functions with the same names as the constructors, and will therefore never be able to construct invalid values.

As with values and types, a module may use data * as a shorthand to export all the data definitions it contains. Note that trying to write data D1 as D2 is not allowed syntax. You could export with a different name for the type alias (e.g. type D1 as D2) if you want to refer to one type with a different name in another context.

2.5.7 Including Fewer (and More) Names

There are forms for include with the same structure as provide for including particular names from other modules. All include forms take a module id and a list of specifications of names to include. (That module id must first have been imported and given a name.)

‹import-stmt›: include ‹import-source› | include from ‹module-ref› : [‹include-spec› (, ‹include-spec›)* [,]] end | import ‹import-source› as NAME | import NAME (, NAME)* from ‹import-source› ‹import-source›: ‹import-special› | ‹import-name› ‹import-special›: NAME ( STRING (, STRING)* ) ‹import-name›: NAME ‹include-spec›: ‹include-name-spec› | ‹include-type-spec› | ‹include-data-spec› | ‹include-module-spec› ‹include-name-spec›: ‹name-spec› ‹include-type-spec›: type ‹name-spec› ‹include-data-spec›: data ‹data-name-spec› [‹hiding-spec›] ‹include-module-spec›: module ‹name-spec›

Some examples:

# This program puts just two names from the builtin string-dict module into
# scope.
import string-dict as SD
include from SD:
  mutable-string-dict,
  make-mutable-string-dict
end

# This program imports and renames two values from the string-dict module
import string-dict as SD
include from SD:
  mutable-string-dict as dict,
  make-mutable-string-dict as make-dict
end

# This program includes all the value names from the string-dict module
import string-dict as SD
include from SD: * end

It is an error to include the same name with different meanings. For example, we could not include the map function from the lists library and import the string-dict constructor as map:

import lists as L
import string-dict as SD
include from L: map end
include from SD: mutable-string-dict as map end

However, it is not an error to include the same name multiple times if it has the same meaning:

# in "student-helpers.arr"
provide from L: map, filter, fold end
import lists as L

# in "student-code.arr"
include file("student-helpers.arr")
import lists as L
include from L: map end
# map included again here, but it's OK because the other map is the same

2.5.8 Importing more than just values

Just as we can provide types, data definitions, and module ids, so too we can import them or include them from other modules. The syntax is analogous to the providing syntax: a client module may write

import string-dict as SD
include from SD:
  *         # includes all values defined in the string-dict module
  type *    # includes all types defined in the string-dict module
  data *    # includes all data definitions from the string-dict module
  module *  # includes any modules provided by the string-dict module
end

2.5.9 Converting between shorthand and expanded syntax

Pyret used to have, and supports for backwards compatibility, a few other syntaxes for modules. The short guide below shows how to convert from this old syntax to the new. Of the old syntax, we only recommend the continued use of include <some-module>, which is a convenient first line of many student-facing starter files.

provide *

provide:
  *
end

provide-types *

provide:
  type *
end

include <some-module>

import <some-module> as <some-name>
include from <some-name>:
  *,
  type *,
  data *,
  module *
end

import name1, ... from <some-module>

import <some-module> as <some-name>
include from <some-name>:
  name1, ...
end