spec Guide

The spec library (API docs) specifies the structure of data, validates or conforms it, and can generate data based on the spec.

To use spec, declare a dependency on Clojure 1.9.0 or higher:

To start working with spec, require the clojure.spec.alpha namespace at the REPL:

Or include spec in your namespace:

Each spec describes a set of allowed values. There are several ways to build specs and all of them can be composed to build more sophisticated specs.

Any existing Clojure function that takes a single argument and returns a truthy value is a valid predicate spec. We can check whether a particular data value conforms to a spec using conform:

The conform function takes something that can be a spec and a data value. Here we are passing a predicate which is implicitly converted into a spec. The return value is "conformed". Here, the conformed value is the same as the original value - we’ll see later where that starts to deviate. If the value does not conform to the spec, the special value :clojure.spec.alpha/invalid is returned.

If you don’t want to use the conformed value or check for :clojure.spec.alpha/invalid, the helper valid? can be used instead to return a boolean.

Note that again valid? implicitly converts the predicate function into a spec. The spec library allows you to leverage all of the functions you already have - there is no special dictionary of predicates. Some more examples:

Sets can also be used as predicates that match one or more literal values:

Until now, we’ve been using specs directly. However, spec provides a central registry for globally declaring reusable specs. The registry associates a namespaced keyword with a specification. The use of namespaces ensures that we can define reusable non-conflicting specs across libraries or applications.

Specs are registered using s/def. It’s up to you to register the specification in a namespace that makes sense (typically a namespace you control).

A registered spec identifier can be used in place of a spec definition in the operations we’ve seen so far - conform and valid?.

You will see later that registered specs can (and should) be used anywhere we compose specs.

Once a spec has been added to the registry, doc knows how to find it and print it as well:

The simplest way to compose specs is with and and or. Let’s create a spec that combines several predicates into a composite spec with s/and:

We can also use s/or to specify two alternatives:

This or spec is the first case we’ve seen that involves a choice during validity checking. Each choice is annotated with a tag (here, between :name and :id) and those tags give the branches names that can be used to understand or enrich the data returned from conform and other spec functions.

When an or is conformed, it returns a vector with the tag name and conformed value:

Many predicates that check an instance’s type do not allow nil as a valid value (string?, number?, keyword?, etc). To include nil as a valid value, use the provided function nilable to make a spec:

explain is another high-level operation in spec that can be used to report (to *out*) why a value does not conform to a spec. Let’s see what explain says about some non-conforming examples we’ve seen so far.

Let’s examine the output of the final example more closely. First note that there are two errors being reported - spec will evaluate all possible alternatives and report errors on every path. The parts of each error are:

For the first reported error we can see that the value :foo did not satisfy the predicate string? at the path :name in the spec :domain/name-or-id. The second reported error is similar but fails on the :id path instead. The actual value is a keyword so neither is a match.

In addition to explain, you can use explain-str to receive the error messages as a string or explain-data to receive the errors as data.

Clojure programs rely heavily on passing around maps of data. A common approach in other libraries is to describe each entity type, combining both the keys it contains and the structure of their values. Rather than define attribute (key+value) specifications in the scope of the entity (the map), specs assign meaning to individual attributes, then collect them into maps using set semantics (on the keys). This approach allows us to start assigning (and sharing) semantics at the attribute level across our libraries and applications.

For example, most Ring middleware functions modify the request or response map with unqualified keys. However, each middleware could instead use namespaced keys with registered semantics for those keys. The keys could then be checked for conformance, creating a system with greater opportunities for collaboration and consistency.

Entity maps in spec are defined with keys:

This registers a :acct/person spec with the required keys :acct/first-name, :acct/last-name, and :acct/email, with optional key :acct/phone. The map spec never specifies the value spec for the attributes, only what attributes are required or optional.

When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. We’ll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.

Let’s take a moment to examine the explain error output on that final example:

Much existing Clojure code does not use maps with namespaced keys and so keys can also specify :req-un and :opt-un for required and optional unqualified keys. These variants specify namespaced keys used to find their specification, but the map only checks for the unqualified version of the keys.

Let’s consider a person map that uses unqualified keys but checks conformance against the namespaced specs we registered earlier:

Unqualified keys can also be used to validate record attributes:

One common occurrence in Clojure is the use of "keyword args" where keyword keys and values are passed in a sequential data structure as options. Spec provides special support for this pattern with the regex op keys*. keys* has the same syntax and semantics as keys but can be embedded inside a sequential regex structure.

Sometimes it will be convenient to declare entity maps in parts, either because there are different sources for requirements on an entity map or because there is a common set of keys and variant-specific parts. The s/merge spec can be used to combine multiple s/keys specs into a single spec that combines their requirements. For example consider two keys specs that define common animal attributes and some dog-specific ones. The dog entity itself can be described as a merge of those two attribute sets:

One common occurrence in Clojure is to use maps as tagged entities and a special field that indicates the "type" of the map where type indicates a potentially open set of types, often with shared attributes across the types.

As previously discussed, the attributes for all types are well-specified using attributes stored in the registry by namespaced keyword. Attributes shared across entity types automatically gain shared semantics. However, we also want to be able to specify the required keys per entity type and for that spec provides multi-spec which leverages a multimethod to provide for the specification of an open set of entity types based on a type tag.

For example, imagine an API that received event objects which shared some common fields but also had type-specific shapes. First we would register the event attributes:

We then need a multimethod that defines a dispatch function for choosing the selector (here our :event/type field) and returns the appropriate spec based on the value:

The methods should ignore their argument and return the spec for the specified type. Here we’ve fully spec’ed two possible events - a "search" event and an "error" event.

And then finally we are ready to declare our multi-spec and try it out.

Let’s take a moment to examine the explain error output on that final example. There were two different kinds of failures detected. The first failure is due to the missing required :event/timestamp key in the event. The second is from the invalid :search/url value (a number instead of a string). We see the same parts as prior explain errors:

The multi-spec approach allows us to create an open system for spec validation, just like multimethods and protocols. New event types can be added later by just extending the event-type multimethod.

A few helpers are provided for other special collection cases - coll-of, tuple, and map-of.

For the special case of a homogenous collection of arbitrary size, you can use coll-of to specify a collection of elements satisfying a predicate.

Additionally, coll-of can be passed a number of keyword arg options:

Following is an example utilizing some of these options to spec a vector containing three distinct numbers conformed as a set and some of the errors for different kinds of invalid values:

While coll-of is good for homogenous collections of any size, another case is a fixed-size positional collection with fields of known type at different positions. For that we have tuple.

Note that in this case of a "point" structure with x/y/z values we actually had a choice of three possible specs:

In this example, coll-of will match other (invalid) values as well (like [1.0] or [1.0 2.0 3.0 4.0]), so it is not a suitable choice - we want fixed fields. The choice between a regular expression and tuple here is to some degree a matter of taste, possibly informed by whether you expect either the tagged return values or error output to be better with one or the other.

In addition to the support for information maps via keys, spec also provides map-of for maps with homogenous key and value predicates.

By default map-of will validate but not conform keys because conformed keys might create key duplicates that would cause entries in the map to be overridden. If conformed keys are desired, pass the option :conform-keys true.

You can also use the various count-related options on map-of that you have with coll-of.

Sometimes sequential data is used to encode additional structure (typically new syntax, often used in macros). spec provides the standard regular expression operators to describe the structure of a sequential data value:

Like or, both cat and alt tag their "parts" - these tags are then used in the conformed value to identify what was matched, to report errors, and more.

Consider an ingredient represented by a vector containing a quantity (number) and a unit (keyword). The spec for this data uses cat to specify the right components in the right order. Like predicates, regex operators are implicitly converted to specs when passed to functions like conform, valid?, etc.

The data is conformed as a map with the tags as keys. We can use explain to examine non-conforming data.

Let’s now see the various occurrence operators *, +, and ?:

Finally, we can use alt to specify alternatives within the sequential data. Like cat, alt requires you to tag each alternative but the conformed data is a vector of tag and value.

If you need a description of a specification, use describe to retrieve one. Let’s try it on some of the specifications we’ve already defined:

Spec also defines one additional regex operator, &, which takes a regex operator and constrains it with one or more additional predicates. This can be used to create regular expressions with additional constraints that would otherwise require custom predicates. For example, consider wanting to match only sequences with an even number of strings:

When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context. For example to describe a sequence like [:names ["a" "b"] :nums [1 2 3]], you need nested regular expressions to describe the inner sequential data:

If the specs were removed this spec would instead match a sequence like [:names "a" "b" :nums 1 2 3].

Now is a good time to step back and think about how spec can be used for runtime data validation.

One way to use spec is to explicitly call valid? to verify input data passed to a function. You can, for example, use the existing pre- and post-condition support built into defn:

When the function is invoked with something that isn’t valid :acct/person data, the pre-condition fails. Similarly, if there was a bug in our code and the output was not a string, the post-condition would fail.

Another option is to use s/assert within your code to assert that a value satisfies a spec. On success the value is returned and on failure an assertion error is thrown. By default assertion checking is off - this can be changed at the REPL with s/check-asserts or on startup by setting the system property clojure.spec.check-asserts=true.

A deeper level of integration is to call conform and use the return value with destructuring to pull apart the input. This will be particularly useful for complex inputs with alternate options.

Here we conform using the config specification defined above:

Here configure calls conform to produce data good for destructuring the config input. The result is either the special ::s/invalid value or an annotated form of the result:

In the success case, the parsed input is transformed into the desired shape for further processing. In the error case, we call explain-data to generate error message data. The explain data contains information about what expression failed to conform, the path to that expression in the specification, and the predicate it was attempting to match.

The pre- and post-condition example in the previous section hinted at an interesting question - how do we define the input and output specifications for a function or macro?

Spec has explicit support for this using fdef, which defines specifications for a function - the arguments and/or the return value spec, and optionally a function that can specify a relationship between args and return.

Let’s consider a ranged-rand function that produces a random number in a range:

We can then provide a specification for that function:

This function spec demonstrates a number of features. First the :args is a compound spec that describes the function arguments. This spec is invoked with the args in a list, as if they were passed to (apply fn (arg-list)). Because the args are sequential and the args are positional fields, they are almost always described using a regex op, like cat, alt, or *.

The second :args predicate takes as input the conformed result of the first predicate and verifies that start < end. The :ret spec indicates the return is also an integer. Finally, the :fn spec checks that the return value is >= start and < end.

Once a spec has been created for a function, the doc for the function will also include it:

We’ll see later how we can use a function spec for development and testing.

Higher order functions are common in Clojure and spec provides fspec to support spec’ing them.

For example, consider the adder function:

adder returns a function that adds x. We can declare a function spec for adder using fspec for the return value:

The :ret spec uses fspec to declare that the returning function takes and returns a number. Even more interesting, the :fn spec can state a general property that relates the :args (where we know x) and the result we get from invoking the function returned from adder, namely that adding 0 to it should return x.

As macros are functions that take code and produce code, they can also be spec’ed like functions. One special consideration however is that you must keep in mind that you are receiving code as data, not evaluated arguments, and that you are most commonly producing new code as data, so often it’s not helpful to spec the :ret value of a macro (as it’s just code).

For example, we could spec the clojure.core/declare macro like this:

The Clojure macroexpander will look for and conform :args specs registered for macros at macro expansion time (not runtime!). If an error is detected, explain will be invoked to explain the error:

Because macros are always checked during macro expansion, you do not need to call instrument for macro specs.

Here’s a bigger set of specs to model a game of cards:

We can validate a piece of this data against the schema:

Or look at the errors we’ll get from some bad data:

The error indicates the key path in the data structure down to the invalid value, the non-matching value, the spec part it’s trying to match, the path in that spec, and the predicate that failed.

If we have a function deal that doles out some cards to the players we can spec that function to verify the arg and return value are both suitable data values. We can also specify a :fn spec to verify that the count of cards in the game before the deal equals the count of cards after the deal.

A key design constraint of spec is that all specs are also designed to act as generators of sample data that conforms to the spec (a critical requirement for property-based testing).

spec provides a set of development and testing functionality in the clojure.spec.test.alpha namespace, which we can include with:

In this guide we have covered most of the features for designing and using specs and generators. We expect to add some more advanced generator techniques and help on testing in a future update.