Ollie Charles recently popped into #ghc to ask about a small program which was taking a long time to compile. In fact, the compiler was taking so long in the simplifier he had to increase the tick factor (a measure of how much work the simplifier is allowed to do) to get compilation to finish. Oleg and I quickly set to work working out what was going on in his program.
It turned out that a misplaced INLINE
pragma was causing a lot of simplification work to be duplicated. Removing the pragma allowed the compiler to operate faster whilst producing the same code.
When a lot of time is spent in the simplifier it is usually because the core programs have grown quite large. Core programs can grow large for a number of reasons but one of primary reasons is due to excessive inlining caused by INLINE
pragmas.
The first tool we have at our disposal is ddumpsimplstats
which outputs a summary of each step the simplifier takes. Looking at this summary is a good way to work out roughly where the problem lies.
In this case, the statistics file was quite large. The first bit I always check is the “UnfoldingDone” section which details how many times each definition has been inlined. Here is the relevant snippet from the top of that section.
14620 UnfoldingDone
596 $
574 contramapF
546 $fNumInt_$c+
485 $fStorableWord8_$cpoke
485 castPtr
485 $fStorableWord8_$calignment
485 word8
485 $s>$<
485 castPtr1
484 thenIO
484 thenIO1
484 ord
484 $fBitsInt_$c.&.
484 plusPtr
484 $fStorableWord19
463 char7
331 $s>*<1
331 pairF
220 returnIO
220 returnIO1
220 $s>$<
220 contramapB
The first thing to notice about these numbers is that there are groups of definitions which have all been inlined the same number of times. This is indicative of a misplaced INLINE
pragma as a large unoptimised definition will then be inlined many times and then simplified individually at each call site rather than once at the definition site. Of particular suspicion is the large block of definitions which are each inlined exactly 484 times.
By looking at the definitions of each of the identifiers in this list, we can then work out what is going on. To cut to the chase, inspecting the definition of char7
from the Data.ByteString.Builder.Prim.ASCII
module we can see where a lot of the work is coming from.
  Encode the least 7bits of a 'Char' using the ASCII encoding.
{# INLINE char7 #}
char7 :: FixedPrim Char
char7 = (\c > fromIntegral $ ord c .&. 0x7f) >$< word8
The definition of char7
is concise but composed of combinators which will be keen to get inlined later on. The definitions of ord
, .&.
and >$<
are all small.
By using an INLINE
pragma, the unoptimised unfolding is included in the interface file so this complex definition will be inline verbatim into each call site. We can inspect the unfolding by using the showiface
flag on the .hi
file for the module.
8334ad079da5b638008c6f8feefdfa4a
char7 :: FixedPrim Char
{ HasNoCafRefs, Strictness: m, Inline: INLINE (satargs=0),
Unfolding: InlineRule (0, False, False)
($s>$<
@ Char
@ Word8
(\ (c :: Char) >
$ @ 'PtrRepLifted
@ Int
@ Word8
(\ (x :: Int) >
case x of wild { I# x# > W8# (narrow8Word# (int2Word# x#)) })
($fBitsInt_$c.&. (ord c) (I# 127#)))
word8) }
Which very closely resembles the source definition.
Removing the INLINE
pragma we get a nice, small optimised definition which crucially is still small enough that GHC inlines it at call sites.
5e7820a4ab4b18cf2032517105d2cc56
char7 :: FixedPrim Char
{ HasNoCafRefs, Strictness: m,
Unfolding: (FP
@ Char
1#
char1
`cast`
(<Char>_R >_R <Ptr Word8>_R >_R Sym (N:IO[0] <()>_R))) }
Look! No calls to >$<
, .&.
, ord
or any other complicated functions. We have optimised the definition once at the definition site so that we don’t have to repeatedly do so at each call site. We didn’t even need to look at the program to spot the problem.
This is currently a problem because INLINE
is used for two different reasons.
RULES
where it is important to inline the literal rhs of a definition so that the rules reliably fire.For the first case, the unoptimised unfoldings are important but for the second this leads to a lot of duplicated work. In this case, I could see that there were no rules defined which were relevant to the definition of char7
so I ruled out the first scenario. I then verified that GHC considered the optimised version of char7
small enough to include in interface files and inline by using showiface
. Ruling out both of these possibilities, it then seemed sensible to remove the pragma.
It would be good to add a new pragma which instructs GHC to inline an optimised unfolding across modules rather than the unoptimised version so that the second scenario can be reliably achieved.
What is an indexed optic? It is an optic which gives you access to an index whilst performing updates.
It is a simple clear generalisation of a lens but the implementation looks quite complicated. This is due to the desire to reuse the same combinators for both nonindexed and indexed variants. We we will start by explaining a simplified implementation of indexed optics before the technique used in order to reuse the same combinators as ordinary optics.
As a first approximation, we will augment the updating function with an additional index which we will then subsequently refine.
type PrimitiveIndexedTraversal i s t a b
= forall f . Applicative f => (i > a > f b) > (s > f t)
Implementing optics by hand gives a good intuition for the types involved.
pair :: PrimitiveIndexedTraversal Int (a, a) (b, b) a b
pair iafb (a0, a1) = (,) <$> iafb 0 a0 <*> iafb 1 a1
The implementation is exactly the same as a normal traversals apart from we also pass an index to each call of the worker function. Note that we have a lot of choice about which indices we choose. We could have indexed each field with a boolean or in the opposite order. For lists, we need to use a helper function which passes an index to each recursive call.
list :: PrimitiveIndexedTraversal Int [a] [b] a b
list iafb xs = go xs 0
where
go [] _ = pure []
go (x:xs) n = (:) <$> iafb n x <*> go xs (n+1)
There are all the usual combinators to work with indexed traversals as normal traversals but one of the most useful ones to see what is going on is itoListOf
which converts an indexed traversal into a list of indexvalue pairs.
itoListOf :: ((i > a > Const [(i, a)] b) > s > (Const [(i, a)] t))
> s > [(i, a)]
itoListOf t s = getConst $ t (\i a > Const [(i, a)]) s
> itoListOf pair (True, False)
[(0, True), (1, False)]
We monomorphise the argument so that we we don’t have to use a variant of cloneTraversal
in order to work around impredicative types.
We can also turn an ordinary traversal into an indexed traversal by labeling each element with the order in which we traverse it. In order to do so we need to define an applicative functor which when traversed with will perform the labelling ultimately returning an indexed traversal.
newtype Indexing f s = Indexing { runIndexing :: (Int > (Int, f s))}
instance Functor f => Functor (Indexing f) where
fmap f (Indexing fn) = Indexing (over (_2 . mapped) f . fn)
instance Applicative f => Applicative (Indexing f) where
pure x = Indexing (\i > (i, pure x))
(Indexing fab) <*> (Indexing fa)
= Indexing (\i >
let (i', ab) = fab i
(i'', a) = fa i'
in (i'', ab <*> a))
Then traversing with this applicative functor supplies the index to each function call which we can pass to our indexed updating function.
indexing :: Traversal s t a b > PrimitiveIndexedTraversal Int s t a b
indexing t p s = snd $ runIndexing
(t (\a > Indexing (\i > (i + 1, p i a) )) s) 0
A common pattern is to use indexing
and traverse
together to create indexed traversals for Traversable
functors. It is so common that it is given a special name traversed
.
traversed :: Traversable t => PrimitiveIndexedTraversal (t a) (t b) a b
traversed = indexing traverse
> itoListOf traversed (Just 5)
[(0, 5)]
However, there are two problems with this representation.
.
) in order to compose indexed optics together.Considering the first problem, in order to compose an indexed optic with an ordinary optic using function composition we would need to be able to unify i > a > b
with s > t
.
Given an ordinary optic op
and an indexed optic iop
with the following types:
op : (a > f b) > (s > f t)
iop : (i > c > f d) > (u > f v)
op . iop
is the only composition which type checks. It yields an indexed traversal which keeps track of the index of the inner component.
> itoListOf (traverse . list) (True, [1,2,3])
[(0, 1), (1, 2), (2, 3)]
However, composition the other way around doesn’t work and further with this representation indexed optics do not compose together with .
. In order to compose indexed optics together with .
we need to be able to unify the argument and result type of the lens together. In order to do this, we abstract away from the the indexed argument of the updating function for any Indexable
profunctor.
class Indexable i p where
index :: p a b > i > a > b
Using this class, the type for indexed traversals becomes:
type IndexedTraversal i s t a b =
forall f p . (Applicative f, Indexable i p) => p a (f b) > s > f t
There are instances for the newtype wrapped indexed functions which we were using before
newtype Indexed i a b = Indexed { runIndexed :: i > a > b }
instance (Indexable i (Indexed i)) where
index (Indexed p) = p
but also for (>)
which ignores the index. This means that we can seamlessly use optics with or without the index by instantiating p
with either Indexed
or >
.
instance (Indexable i (>)) where
index f _ = f
Now op . iop
yields an indexed traversal, iop . op
forces us to instantiate p = (>)
and so yields a traversal which has forgotten the index. Perhaps most surprisingly, the composition iop . iop
typechecks as well, but again we loose information as we are forced to instantiate p = (>)
and thus forget about the indexing of the outer traversal.
This is a double edged sword. Composing using .
leads to more code reuse as the same combinators can be used for both indexed and nonindexed optics. On the other hand, composing indexed optics using .
is nearly always the wrong thing if you care about the indicies.
Composing together indexed optics with the normal lens composition operator .
leads to unexpected results as the indices are not combined appropiately. The index of the innermost optic of the composition is preserved whilst the outer indexing is thrown away. It would be more desirable to combine the indicies together in order to retain as much information as possible.
To that end we define <.>
which can compose indexed optics together whilst suitably combining their indices.
(<.>) :: Indexable (i, j) p
=> (Indexed i s t > r)
> (Indexed j a b > s > t)
> p a b > r
(istr <.> jabst) p
= istr (Indexed (\i s >
jabst (Indexed (\j a >
indexed p (i, j) a)) s))
The definition monomorphises the argument again in order to avoid inpredicativity problems.
> itoListOf (list <.> pair) [(1,2), (2, 3)]
[((0,0),1),((0,1),2),((1,0),2),((1,1),3)]
Generalisations are possible which combine indicies in other ways but this simple combination function highlights the essence of the approach.
Now that is a question which I will have to defer to reddit comments. I couldn’t find many libraries which were using the indexing in interesting ways.
]]>The inliner and specialiser are the two parts of the optimiser which are crucial to writing performant functional programs. They ensure that we can write programs at a highlevel of abstraction which are simplified when eventually used with concrete arguments.
The inliner’s job is to replace a function with its definition. This removes one layer of indirection and most importantly allows other optimisations to fire. The specialiser is important for optimising code which uses type classes. Type classes are desugared into dictionary passing style but the specialiser removes a layer of indirection by creating new functions with the relevant dictionaries already supplied.
This document will explain the basics of these two parts of the optimiser and some userfacing options which can be used to control them.
INLINABLE
pragma do?Toplevel definitions can be marked INLINABLE
.
myComplicatedFunction :: (Show a, Num a) => ...
myComplicatedFunction = ...
{# INLINABLE myComplicatedFunction #}
This causes exactly two things to happens.
Note that GHC is no more keen to inline an INLINABLE
function than any other.
INLINE
pragma do?The INLINE
pragma can be applied to toplevel definitions. It behaves like the INLINABLE
pragma but makes GHC very keen to inline the function.
mySimpleFunction :: ...
mySimpleFunction = ...
{# INLINE mySimpleFunction #}
It is a sledgehammer and without care you can make the compiler take a long time and produce a lot of code. Most “ticks exhausted” panics are due to library authors misusing INLINE
pragmas.
Liberally sprinkling all your definitions with INLINE
is likely make the compiler take a very long time to compile your program. It is not beneficial to inline every function, inlining a function which is not optimised further only increases overall code size without improving performance.
One situation where it is useful to use an INLINE
pragma is when the definition of the function contains functions which are mentioned in RULES
. In this case, it is essential that the optimiser is quite aggressive so that the RULES
can fire.
GHC will decide to include some small unfoldings in interface files. When it does this, it first optimises the definitions so that they are not repeatedly optimised at each use site after being inlined. Unfoldings included by INLINE
or INLINABLE
are unoptimised so that they interact better with RULES
.
An interface file stores all information about a module which is needed by other modules.
The key to crossmodule inlining and specialisation is making sure that we have the definitions of functions we want to inline at hand. Information is only passed between modules by interface files, therefore we must include the unfoldings of definitions in interface files if we want to inline them across modules.
The extension for interface files is .hi
, you can see what’s in an interface file by using the showiface
flag.
The unfolding of a function f
is what f
is replaced by when it is inlined. This is usually the definition of f
.
Not all definitions are included in interface files by default, doing so might create quite large files. There’s no point including an unfolding of very large definitions which we will never inline in other modules.
Unfoldings end up in interface files in three ways:
INLINE
or INLINABLE
are included in interface files.fexposeallunfoldings
include all unfoldings of all definitions in a module unless they are marked as NOINLINE
.Specialisation is the process of removing typeclass dictionary arguments by creating a new typespecialised definition for an overloaded function. Once specialised, dictionary methods can be easily inlined which usually creates more efficient code.
For example, if we define the overloaded function foo
foo :: Show a => a > a > Bool
foo x y = show x == show y
the following core definition will be produced:
foo = \ @a $dShow x y >
eqString (show $dShow x) (show $dShow y)
There are now 4 parameters to foo
, the first argument is a type (denoted by @
), the second argument is the dictionary for Show
(denoted by the $d
prefix) and the last two are the arguments x
and y
of the original definition.
The class constraint is translated into a dictionary. Each time a class method is used, we must dynamically lookup which definition to use in the supplied class dictionary.
If we know which type a
is instantiated with, we can specialise the definition of foo
and produce much better code.
qux :: Bool > Bool > Bool
qux = foo @Bool
Using foo
at a specific type produces a new definition foo_$sfoo
which is defined as:
foo_$sfoo :: Bool > Bool > Bool
foo_$sfoo = foo @Bool $fShowBool
Further optimisations then inline foo
and then the dictionary selector show
which produces the following more direct program.
foo_$sfoo =
\ x y >
case x of {
False >
case y of {
False > foo4;
True > foo3
};
True >
case y of _ {
False > foo2;
True > foo1
}
}
Specialisation occurs when an overloaded function is called at a specific type. The specialised definition is placed in the module where the call happens but also exported so that it can be reused if there is another upstream callsite where specialisation would take place.
By default, functions are not specialised across modules.
There are two ways to make functions specialise across modules:
INLINABLE
or INLINE
.fspecialiseaggressively
when compiling the client module. An unfolding must still be available to perform specialisation.Further to this, observe that for specialisation to occur across modules, the unfolding must be made available in interface files.
Notice this subtle point, the INLINABLE
pragma guarantees the precise conditions for a function to be specialised across modules.
SPECIALISE
pragma?The SPECIALISE
pragma is used to create a specialised copy of an overloaded function even if it is not used with that type in the module.
module A where
class C ...
foo :: C a => a > a
{# SPECIALISE foo :: Text > Text #}
This example will create a new function, foo_$sfoo :: Text > Text
which will be used whenever foo
is applied to a Text
value even in modules which import A
.
This is useful to prevent GHC creating many copies of the same specialised function if you have a very flat module structure.
In general, if we were to inline recursive definitions without care we could easily cause the simplifier to diverge. However, we still want to inline as many functions which appear in mutually recursive blocks as possible. GHC statically analyses each recursive groups of bindings and chooses one of them as the loopbreaker. Any function which is marked as a loopbreaker will never be inlined. Other functions in the recursive group are free to be inlined as eventually a loopbreaker will be reached and the inliner will stop.
Note: Do not apply INLINE
pragmas to loopbreakers, GHC will never inline a loop breaker regardless of which pragma you attach. In fact, with a debugging compiler, core lint will warn about using an INLINE
pragma on a loopbreaker.
Loopbreakers are discussed in detail in section 4 of Secrets of the Glasgow Haskell Compiler inliner.
GHC uses a heuristic to decide which definitions it would be least beneficial to inline and to choose those as loop breakers. For example, it is very beneficial to inline simple expressions and dictionary selector functions so they are given high scores. Discounts are also available if an unfolding is available thus marking a definition as INLINABLE
or INLINE
will usually cause GHC to not choose it.
fspecialiseaggressively
removes the restrictions about which functions are specialisable. Any overloaded function will be specialised with this flag. This can potentially create lots of additional code.
fexposeallunfoldings
will include the (optimised) unfoldings of all functions in interface files so that they can be inlined and specialised across modules.
Using these two flags in conjunction will have nearly the same effect as marking every definition as INLINABLE
apart from the fact that the unfoldings for INLINABLE
definitions are not optimised.
Sometimes people ask if GHC is smart enough to unroll a recursive definition when given a static argument. For example, if we could define sum
using direct recursion:
sum :: [Int] > Int
sum [] = 0
sum (x:xs) = x + sum xs
A user might expect sum [1,2,3]
to be optimised to 6. However, GHC will not inline sum
because it is a selfrecursive definition and hence a loopbreaker. The compiler is not smart enough to realise that repeatedly inlining sum
will terminate.
However, there is a trick that can be used in order to tell GHC that an argument is truly static. We replace the value argument with a type argument. Then by defining suitable type class instances, we can recurse on the structure of the type as we would on a normal value. This time however, GHC will happily inline each “recursive” call as each call to sum
is at a different type.
{# LANGUAGE DataKinds #}
{# LANGUAGE KindSignatures #}
{# LANGUAGE TypeOperators #}
{# LANGUAGE TypeApplications #}
{# LANGUAGE NoImplicitPrelude #}
{# LANGUAGE ScopedTypeVariables #}
{# LANGUAGE InstanceSigs #}
{# LANGUAGE PolyKinds #}
module Sum where
import Prelude (Integer, (+))
import GHC.TypeLits
data Proxy x = Proxy
class Sum (xs :: [Nat]) where
sum :: proxy xs > Integer
instance Sum '[] where
sum _ = 0
instance (KnownNat x, Sum xs) => Sum (x ': xs) where
sum :: Proxy (x ': xs) > Int
sum _ = natVal (Proxy @x) + sum (Proxy @xs)
main = sum (Proxy @'[1,2,3])
Inspecting the core we find that the definition of main
is simplified to the constant value 6
.
Note that this is slightly different to the static argument transformation which applies to a multiparameter recursive functions where one of the arguments is the same for each recursive call. In this case, there are no arguments which remain constant across recursive calls.
This technique is due to Andres Löh.
Thanks to Reid Barton, Ashok Menon and Csongor Kiss for useful comments on a draft.
]]>Something I have never seen articulated is why the Foldable
type class exists. It is lawless apart from the free theorems which leads to adhoc definitions of its methods. What use is the abstraction if not to enable us to reason more easily about our programs? This post aims to articulate some justification stemming from the universality of folds.
In brief, here is the argument.
Foldable
type class is a way to exploit this universality without having to define all of our data types as the fixed points of base functors.To recall, the type class is meant to capture the intuitive notion of a fold. Folds are a way of consuming a data type by summarising values in a uniform manner before combining them together.
We now recall the basics of initial algebra semantics for polynomial data types.^{1} We can represent all data types as the fixed point of a polynomial functor. In Haskell we can represent the fixed point by the data type Mu
.
data Mu f = Mu { out :: (f (Mu f)) }
 Inital algebra
in :: f (Mu f) > Mu f
in = Mu
We then specify data types by first defining a control functor \(F\) and then considering the initial \(F\)algebra for the functor. The initial \(F\)algebra is given by \((\mu F, in)\). The injection function \(in\) wraps up one level of the recursion. The projection function out
strips off one layer of the recursion.^{2}
We can define the type of lists by first specifying a control functor \(ListF = 1 + (A \times \_)\) and then defining lists in terms of this base functor and Mu
.
data ListF a r = Nil  Cons a r
type ListM a = Mu (ListF a)
We call types which are definable in this manner inductive types.
We do not usually define data types in this style as programming directly with them is quite cumbersome as one must wrap and unwrap to access the recursive structure.
However, defining data in this manner has some useful properties. The one that we care about is that it is possible to define a generic fold operator for inductive types.
cata :: (f a > a) > Mu f > a
cata f = f . fmap (cata f) . out
What’s more, due to initiality, cata f
is the unique function of this type. We have no choice about how we define a fold operator after we have described how to interpret the control functor.
Fleshed out in some more detail, Given a functor \(F\), for any other algebra \((B, g : F B \to B)\) there exists a unique map \(h\) to this algebra from \((\mu F, in)\). Our definition of cata
is precisely the way to construct this unique map.
Languages such as Haskell allow users to define data types in a more adhoc fashion by specifying the recursive structure themselves rather than in terms of a base functor.
data List a = Nil  Cons a (List a)
It can be easily seen that Mu (ListF a) ~= List a
and thus we can exploit the the uniqueness of the fold function and define a canonical fold function specialised to our newly defined data type.
foldr :: (a > b > b) > b > List a > b
foldr
is a specialisation of cata
to lists. It is perhaps clearer to see the correspondence if we rewrite the function to explicitly take a list algebra as it’s first argument.
data ListAlg a b = ListAlg { z :: () > b , cons :: (a, b) > b }
foldr :: ListAlg a b > List a > b
Specifying a function ListF a b > b
is precisely the same as specifying ListAlg a b
as it amounts to specifying functions \(1 \to b\) and \(a \times b \to b\).
So, for each data type we define we can specialise the cata
operator in order to define a canonical fold operator. However, the issue now is that each one of our fold operators has a different type. It would be useful to still be able to provide a consistent interface so that we can still fold any inductive type. The answer to this problem is Foldable
.
This highlights the essential tension with the Foldable
type class. It exists in order to be able to continue to define a generic fold operation but without the cost of defining our data types in terms of fixed points and base functors.
Foldable
The method foldr
is the only method needed to define an instance of Foldable
.^{3}
class Foldable f where
foldr :: (a > b > b) > b > t a > b
It turns out that (a > b > b)
and a single constant b
are sufficient for specifying algebras for inductive types. Inductive types are built from polynomial base functors so we can describe an algebra by first matching on the summand and then iteratively applying the combining function to combine each recursive position. If there are no recursive positions, we instead use the zero value z
.
Defining the instance for lists is straightforward:
instance Foldable [] where
foldr _ z [] = z
foldr f _ (x:xs) = f x (foldr f xs)
As another example, we consider writing an instance for binary trees which only contain values in the leaves. It is less obvious then how to implement foldr
as the usual fold (foldTree
below) has a different type signature.
data Tree a = Branch (Leaf a) (Leaf a)  Leaf a
foldTree :: (b > b > b) > (a > b) > Tree a > b
We can do so by cleverly instantiating the result type b
when performing the recursive calls.
instance Foldable Tree where
foldr :: (a > b > b) > b > Tree a > b
foldr f z t =
case t of
Leaf a > f a z
Branch l r > foldr (\a c > f a . c) id l
(foldr f z r)
The first recursive call of foldr
returns a function of type b > b
which tells us how to combine the next recursive occurence. In this case there are only two recursive positions but this process can be iterated to combine all the recursive holes.^{4}
The definition for foldMap
usually provides a more intuitive interface for defining instances but it is harder to motivate which is why we prefer foldr
.
foldMapTree :: Monoid m => (a > m) > Tree a > m
foldMapTree f (Leaf a) = f a
foldMapTree f (Branch l r) = foldMapTree f l <> foldMapTree f r
However, instances for inductive types defined in this uniform manner are less powerful than folds induced by an \(F\)algebra. The problem comes from the fact that all the recursive holes much be combined in a uniform fashion.
The question I have, is it possible to define middle
using the Foldable
interface?
data Tree3F a r = Leaf a  Branch r r r
type Tree3 a = Mu (Tree3F a)
middleFold :: Tree3F a a > a
middleFold (Leaf a) = a
middleFold (Branch _ m _) = m
middle :: Tree3 a > a
middle = cata middleFold
The definition of Foldable
is motivated by the wellunderstood theory of inductive data types. The pragmatics of Haskell lead us to the seemingly quite adhoc class definition which has generated much discussion in recent years. The goal of this post was to argue that the class is better founded than people think and to explain some of the reasons that it leads to some uncomfort.
My argument is not about deciding whether an adhoc definition is lawful, it is explaining the motivation for the class in a way which also explains the lawlessness. The class definition is a compromise because of the practicalities of Haskell. The only way in which we can know a definition is sensible or not is by inspecting whether the adhoc definition agrees with the canonical definition given by cata
.
foldr
There are some free theorems for foldr
which is natural in a
and b
.
foldr :: (a > b > b) > b > t a > b
Naturality in b
amounts to, for all functions g : b > c
.
g (foldr f z t) = foldr (\x y > f x (g y)) (g z)
and naturality in a
amounts to, for all functions f : c > a
.
foldr f z . fmap h = foldr (\x y > f (h x) y) z
These are included for completeness.
From now on when we say data type, we mean polynomial data type.↩
Lambek’s lemma tells us that the two functions are each other’s inverse.↩
Notice that we are only able to fold type constructors of kind * > *
, this is an arbritary choice, motivated by the fact that most containers which we wish to fold are polymorphic in one way. (For example, lists and trees).↩
Note that I didn’t think of this definiton myself but arrived at it purely calculationally from the definition of foldMap
and foldr
.↩
Pattern synonyms can’t (safely) cause any additional type refinement than their definition dictates. This means that they can’t be used to provide a GADTlike interface when the underlying representation is not a GADT. The purpose of this note is to explain this restriction.
The defining feature of GADTs is that the constructors can bind dictionaries for constraints.
data CaptureShow a where
CaptureShow :: Show a => CaptureShow a
When the constructor is matched upon, the constraints are provided to the local environment.
When the constraints are equality constraints, this causes type refinement. We learn more about the result type from performing the match.
We can use pattern synonyms to abstract GADTs. The second set of constraints is the set of provided constraints.
pattern MyCaptureShow :: () => Show a => CaptureShow a
pattern MyCaptureShow = CaptureShow
But, the set of provided constraints must be exactly those constraints which the underlying constructor provides. This is different to required constraints which can be more specific than needed.
Why is this the case? One might expect that if additional constraints were specified then the pattern synonym could bind the needed dictionaries when building and release them when matching. However, not all values which can be destructed with a pattern synonym must be constructed with a pattern synonym.
For example, we would be able to specify an unsatisfiable constraint in the provided context.
pattern Unsat :: () => ('True ~ 'False) => Int
pattern Unsat = 0
If we did the same in a GADT it would be impossible to construct such a value, similary here we can’t use Unsat
to construct an Int
as we will never be able to satisfy the equality constraint. However, if it were possible to define such a pattern synonym we would be able to use it to match on 0
. Doing so would provide the bogus constraint.
There is a more immediate reason why this will never work. For type class constraints, the dictionaries must be bound when the constructor is used to construct values. If the pattern synonym is not used to construct the value then we can’t conjure it up out of thin air when we need it.
This wasn’t obvious to me, which is why it is now written down. Pattern synonym signatures are surprisingly tricky.
David Feuer and Edward Yang conspired to show that using unsafeCoerce
it was possible to provide additional type equalities. The key to the approach is to use a dummy GADT which is used to actually do the refining. Our more efficient representation is upcasted to this GADT, then by matching on the constructor, we cause refinement. Here is Edward’s code:
{# LANGUAGE KindSignatures #}
{# LANGUAGE DataKinds #}
{# LANGUAGE PatternSynonyms #}
{# LANGUAGE ViewPatterns #}
{# LANGUAGE TypeOperators #}
{# LANGUAGE GADTs #}
{# OPTIONS_GHC fwarnincompletepatterns #}
module GhostBuster where
import GHC.TypeLits
import Unsafe.Coerce
newtype Vec a (n :: Nat) = Vec { unVec :: [a] }
 "Almost" Vec GADT, but the inside is a Vec
 (so only the toplevel is unfolded.)
data Vec' a (n :: Nat) where
VNil' :: Vec' a 0
VCons' :: a > Vec a n > Vec' a (n + 1)
upVec :: Vec a n > Vec' a n
upVec (Vec []) = unsafeCoerce VNil'
upVec (Vec (x:xs)) = unsafeCoerce (VCons' x (Vec xs))
pattern VNil :: () => (n ~ 0) => Vec a n
pattern VNil < (upVec > VNil') where
VNil = Vec []
pattern VCons :: () => ((n + 1) ~ n') => a > Vec a n > Vec a n'
pattern VCons x xs < (upVec > VCons' x xs) where
VCons x (Vec xs) = Vec (x : xs)
headVec :: Vec a (n + 1) > a
headVec (VCons x _) = x
mapVec :: (a > b) > Vec a n > Vec b n
mapVec f VNil = VNil
mapVec f (VCons x xs) = VCons (f x) (mapVec f xs)
If we were to change the definition of the nil case of mapVec
to use VCons
instead then it wouldn’t type check.
There have been four small but significant improvements to pattern synonyms which are going to appear in GHC 8.0.
This work closes up some holes which were left in the implementation of pattern synonyms and should provide library authors with a new and flexible method of abstraction.
More information about pattern synonyms can be found in the GHC 8.0 user guide.
The biggest update extends pattern synonyms to allow the construction of pattern synonyms which behave like record data constructors.
Since GHC 7.8 you have been able to define prefix and infix pattern synonyms which behave like normal data constructors. With the addition of record pattern synonyms most data constructors can be replicated by pattern synonyms.^{1}
To make this clear, consider the data constructor Just
. We can use this constructor in two contexts, in a pattern match or in an expression context to construct a value.
If we defined the pattern synonym MyJust
, we can use it in precisely the same contexts as Just
.
pattern MyJust :: a > Maybe a
pattern MyJust a = Just a
Similarly, record data constructors can be used in seven contexts.
Usage  Example 

As a constructor 

As a constructor with record syntax 

In a pattern context 

In a pattern context with record syntax 

In a pattern context with field puns 

In a record update 

Using record selectors 

Record pattern synonyms are defined as follows and can also be used in these seven contexts.
pattern MyPoint :: Int > Int > (Int, Int)
pattern MyPoint{x, y} = (x,y)
Projection functions, x
and y
are defined like record selectors for ordinary constructors.
x :: (Int, Int) > Int
y :: (Int, Int) > Int
Because we defined this pattern synonym, tuples can now be updated with record update syntax.
> (0,0) { x = 5 }
(5, 0)
Since pattern synonyms are a lot like data constructors, they should be able to be imported just like data constructors. To put it another way a user should be unaware whether they are using a pattern synonym or a data constructor.
However, before GHC 8.0, there has been quite an awkward distinction between the two. Data constructors couldn’t be imported or exported separated from the type which they construct. On the other hand, pattern synonyms could only be imported and exported individually by using the pattern
keyword. This meant that consumers had to be aware that whether they were importing a pattern synonym or not! No good!
Now there are two ways which we can export the pattern synonym P :: A
.
Separately, as before, by using the pattern
keyword.
module Foo (pattern P) where
Bundled with the relevant type constructor
module Foo ( A(P) ) where
or to export all of A
’s constructors along with the pattern synonym P
.
module Foo ( A(.., P) ) where
In this second case, if another module imports Foo
then P
can be imported alongwith A
.
 Will import P
import Foo (A (..))
or
 Will import P
import Foo (A (P))
ErrorCall
This problem reared its head in one of the first serious uses of pattern synonyms. In GHC 8 a pattern synonym ErrorCall
is introduced into the base library to smooth over changes in the internal representation caused by Eric Seidel’s work on call stacks.
The datatype ErrorCall
previously just had one synonymous constructor.
data ErrorCall = ErrorCall String
After the refactoring, the single constructor was renamed to ErrorCallWithLocation
but Eric wanted to smooth over the transition by providing a pattern synonym which would behave much like before.
data ErrorCall = ErrorCallWithLocation String String
pattern ErrorCall :: String > ErrorCall
pattern ErrorCall s < ErrorCallWithLocation s _ where
ErrorCall s = ErrorCallWithLocation s ""
However clients importing ErrorCall(..)
found that despite the careful efforts of the library author this change broke their code. The problem being that by default, it is necessary to explicitly import pattern synonyms.
With this feature, we can now bundle the new ErrorCall
pattern synonym in the export list of the module so that users importing ErrorCall(..)
will also import the pattern synonym.
module GHC.Exception ( ErrorCall(.., ErrorCall) ) where
Pattern synonyms can also have type signatures. The syntax is very similar to normal type signatures but there are two sets of constraints rather than the usual one which correspond to “required” and “provided” constraints.
pattern P :: required => provided => type
In the common case that there are no provided constraints, it is possible to omit the first set of constraints.
pattern P :: required => type
and in the even more common case when there are no constraints, both can be omitted.
pattern P :: type
Required constraints are constraints which are required in order to make a match. For example, we could provide the quite silly pattern synonym which uses show
to check whether a pattern should match. As show
is from the Show
typeclass, we have to add it to the required constraints.
pattern IsTrue :: Show a => a
pattern IsTrue < ((== "True") . show > True)
Provided constraints are constraints which are made available on a successful match. This usually occurs when matching on a GADT with an existential type.
In fact, it only makes sense for provided constraints to mention existentially quantified type variables which explains why they are less often used.
data T where
MkT :: (Show b) => b > T
pattern ExNumPat :: () => Show b => b > T
pattern ExNumPat x = MkT x
Pattern synonym signatures aren’t new for GHC 8.0 but the order of required and provided constraints has been switched.^{2}
The final small change is that GHC can also warn about any pattern synonym which doesn’t have a type signature. The warning is turned on by the flag fwarnmissingpatsynsigs
and is also enabled by Wall
.
There is one exception which is a datatype which has record constructors which share field names. data A = B { a :: Int }  C { a :: Int, b :: Int }
↩
In previous versions, provided constraints appeared before required constraints and if only one set was given then it was assumed to be the provided rather than required constraints.↩
Users upgrading to hlint 1.9.23 will now be able to take advantage of the new refactor
flag which by invoking the refactor
executable supplied by applyrefact
can automatically apply suggestions.
cabal install hlint
cabal install applyrefact
hlint refactor myfile.hs
To take advantage of this flag users also need to install the refactor
executable which is provided by applyrefact
. Users can directly invoke refactor
if they wish but it is easier to invoke it via hlint, passing any extra options with the refactoroptions
flag.
HLint will by default apply all suggestions without any prompts to the user and output the result on stdout.
If you’re feeling cautious, the s
flag will prompt you before applying each refactoring.
If you’re feeling brave, the i
flag will perform the refactoring in place.
The final option which is useful for tool writers is the pos
flag which specifies the region to which hints should be applied to.
There are plugins availible for vim, emacs and atom. Moritz Kiefer has already helped me with the emacs plugin, pull requests to clean up the other two plugins would be greatly appreciated.
If you find that the program gives you an unexpected result then please report a bug to the issue tracker. Please also include the output of hlint serialise
and the input file.
There are a few known problems with CPP and a few corner cases which are not handled by the underlying library but for most users, a painfree experience is expected.
Tabs and trailing whitespace will be removed (this is by design and not going to change). Line endings are changed to unix line endings.
Not all hlint suggestions are supported. A notable exception being the eta reduction refactoring as it proved quite difficult to implement correctly.
Sometimes hlint suggests a refactoring which uses a name which isn’t in scope. The tool is quite dumb! It just takes whatever hlint suggests and applies it blindly. It might be good to add an option to hlint to avoid spitting out hints which do this.
Over the last few months Alan Zimmerman and myself have been working on what will hopefully become a new foundation for refactoring Haskell programs. We have called this foundation ghcexactprint
and it is now available on hackage to coincide with the release of GHC 7.10.2.
Thompson and Reinke originally described 22 refactorings in their catalogue of functional refactorings which motivated the development of HaRe. Missing from their list is the identity transformation, in the context of refactoring this is perhaps the most important. When applying any kind of change to small segment of your project it is undesirable for the tooling to modify another part of your file unexpectedly. The identity refactoring seems to be the easiest of all but it is important that any refactoring tool behaves in this way.
The biggest challenge of the last few months was being able to perform the identity refactoring whilst retaining a suitable representation to be able to easily transform programs. Now that we believe that ghcexactprint
is robust, the challenge is to build the promised refactoring tools upon it.
If your only concern is to perform the identity transformation then perhaps the representation you choose is not very important. One can easily choose to operate directly on the source files for instance. The representation becomes much more important when considering other refactoring operations.
When deciding on a representation, you have to take into account more complicated transformations such as renamings, insertions and deletions which are made much easier by manipulating an AST. Most current source manipulations are based on haskellsrcexts
which provides a separate parser and as a result lacks support for many modern language extensions. We instead chose to base ghcexactprint
directly on the GHC parser so that we don’t have to worry (much) about adding support for future language extensions and secondly, working with the GHC AST directly is essential if you want to perform typeaware refactorings.
We work in two stages.
After receiving the output of the GHC parser, the first stage converts all the absolute source positions into relative positions.
not True = False
For example, in the preceding declaration, using absolute source positions we might describe the the declaration starts at (0,0)
, the first argument starts at (0,4)
and the RHS of the definition starts at (0,12)
. Using relative positioning like in the library, we instead describe the position of each element relative to the position of the previous element. For example, True
is 0 lines and 1 column further than not
and False
is 0 lines and 1 column further than =
.
To keep track of this information we introduce a separate data structure which we call annotations. This data structure keeps track of the relative positions of all AST elements and keywords as well as any necessary information which is not included in the AST in order to exactly reproduce the original source file.
We hope this approach will make it easy to perform transformations without worrying too much about where exactly everything will end up. With this style, replacing expressions is easy as we don’t have to worry about updating anything at all, just by replacing the expression in the AST produces the correct output.
Secondly, we perform the reverse of this transformation and produce a source file given an AST and the previously worked out relative positions. Thus a refactoring becomes two things:
Much of the last 10 months was spent modifying GHC and ghcexactprint
in order to be able roundtrip any source file successfully. The library has been through many iterations with varying levels of success but finally has reached an understandable and usable core. After testing the program on around 50,000 source files from Hackage, we can say with confidence that at least the foundations are in place for further tooling to be built.
A glance at the test suite will show some particularly tricky examples. The IOHCC also provided a good source of test material..
import Data.Char
e=181021504832735228091659724090293195791121747536890433
u(f,m)x=i(m(x), [],let(a,b)=f(x) in(a:u(f,m)b))
(v,h)=(foldr(\x(y )>00+128*y+x)0,u( sp(25),((==)"")))
p::(Integer,Integer )>Integer > Integer NotInt
p(n,m)x =i(n==0 ,1,i(z n ,q(n,m)x, r(n,m)x))
i(n,e,d )=if(n) then(e) else (d) 23+3d4f
(g,main ,s,un)= (\x>x, y(j),\x>x*x,unlines))
j(o)=i(take(2)o== "e=","e="++t (drop(42)o),i(d>e,k,l)o)
l=un.map (show.p (e,n).v.map( fromIntegral{g}.ord)).h
k=co.map(map(chr .fromIntegral ).w.p(d,n). read).lines
(t,y)=(\ (o:q)> i(o=='' ,'1','' ): q,interact)
q(n,m)x= mod(s( p( div(n)2, m{jl})x) )mhd&&gdb
(r,z,co) =(\(n, m)x>mod(x*p(n1, m)x)m,even ,concat)6
(w,sp)=( u(\x>( mod(x)128,div(x )128),(==0 )),splitAt)
d=563347325936+1197371806136556985877790097563347325936
n=351189532146914946493104395525009571831256157560461451
Replacements, deletions and rearrangements are very easy using this framework. Most of the time they can be specified by generic traversals (with any generics library) The last piece of the puzzle is to make it just as easy to add new items into the AST. A challenge that I and Alan Zimmerman will now begin to work on.
If you are interested in using the tooling then we are both active in #haskellrefactorer
.
There are a few things which remain difficult to process.
This simple example inserts a type signature making sure to move comments to the right place. We hope to build a higher level interface on top of this low level manipulation.
Along with the changes that Alan made to GHC to keep track of all the locations of keywords during parsing there have been two other improvements to enable this work.
The following example takes advantage of the former of these two additions.
 test.hs
module Foo where
baz "one" = 1
baz "two" = 2
{# LANGUAGE NamedFieldPuns #}
module InsertSignature where
import Language.Haskell.GHC.ExactPrint
import Language.Haskell.GHC.ExactPrint.Parsers
import Language.Haskell.GHC.ExactPrint.Types
import Language.Haskell.GHC.ExactPrint.Utils
import qualified Data.Map as Map
import qualified HsSyn as GHC
import qualified RdrName as GHC
import qualified SrcLoc as GHC
type Module = GHC.Located (GHC.HsModule GHC.RdrName)
main :: IO ()
main = do
Right (as, m) < parseModule "test.hs"
(finalAs, finalM) < addSignature "baz" "baz :: String > Int" as m
putStrLn $ exactPrintWithAnns finalM finalAs
addSignature :: String  ^ Function to add a signature for
> String  ^ Type signature
> Anns
> Module
> IO (Anns, Module)
addSignature funid tsig as (GHC.L l m) = do
 Parse new AST element
Right (sigAnns, sig) < withDynFlags (\d > parseDecl d "template" tsig)
let (before, (bind: after)) = break findFunBind (GHC.hsmodDecls m)
 Add new annotations to the map
newAs = Map.union as sigAnns
 Modify the annotations to
 1. Retain the original spacing
 2. Make sure that comments are placed correctly.
Just Ann{annEntryDelta, annPriorComments} = Map.lookup (mkAnnKey bind) newAs
finalAnns = Map.adjust (\sigAnn > sigAnn { annEntryDelta = annEntryDelta
, annPriorComments = annPriorComments })
(mkAnnKey sig)
. Map.adjust (\bindAnn > bindAnn { annEntryDelta = DP (1, 0)
, annPriorComments = [] })
(mkAnnKey bind) $ newAs
finalMod = m { GHC.hsmodDecls = before ++ [sig, bind] ++ after }
return (finalAnns, GHC.L l finalMod)
where
findFunBind :: GHC.LHsDecl GHC.RdrName > Bool
findFunBind (GHC.L _ (GHC.ValD b@(GHC.FunBind {})))
 showGhc (GHC.unLoc (GHC.fun_id b)) == funid = True
findFunBind _ = False
module Foo where
baz :: String > Int
baz "one" = 1
baz "two" = 2
By hitting that critical sweet spot of solving an interesting problem and having a catchy name, most know, if not understand Wouter Swierstra’s data types à la carte.
As the name suggests, the à la carte approach involves composing together data types to form bigger, custom types along with functions where we can precisely specify the necessary pieces. The approach relies heavily on a clever trick with type classes which is reviewed below. With more modern GHC extensions, the implementation can be made much more explicit, this is what this post will explore.
For a general overview of the approach there is no better introduction than the original functional pearl.
To recap, in order to ease writing composed data types, we define a type class a :<: b
to express that a
is a subtype of b
. To do this we require three instances.
 Coproduct
data (f :+: g) e = Inl (f e)  Inr (g e)
class f :<: g where
inj :: f a > g a
instance f :<: f where
inj = id
instance f :<: (f :+: g) where
inj = Inl
instance (f :<: h) => f :<: (g :+: h) where
inj = inj . Inr
Together these three instances model a linear search through our nested coproduct whilst picking out the correct injection.
Closed type families as introduced by Richard Eisenberg (2013) are on a simple level are restricted functions which act on types. With this observation, it seems that they can be used to reimplement most type class tricks in a more explicit manner and indeed this is the case. Patrick Bahr (2014) was the first to explore this idea in connection with compositional data types, this post closely follows his exposition.
Patrick incrementally builds a solution to provide a more robust subtyping constraint. He proceeds in three stages.
Today I will talk about the first two stages, leaving the slightly more intricate third for another time.
Implementing the subtyping relation using type families brings out quite a subtle point which is implicit in the original pearl. The type class :<:
has two purposes, the first is to check whether we are able to construct an injection from f
to g
by computing at the type level. The second is to work out the correction injection from a
to b
at the term level. Type families make this dependency explicit.
Thus, our first step will be to check whether such an injection exists from f
to g
.
As type class resolution operators without backtracking  we can’t express any kind of branching computation in type class instances. This led to the convention where we were forced to make sure that :+:
associated to the right.^{1} Hence, it was easy to think about our composed data types as lists of types (with :+:
being a type level cons and :<:
searching through the list). As type families allow us to backtrack, this restriction is needless. Instead it is much easier to think about our constructed data types as trees.
Our goal becomes to find a type family which searches these trees. To do this, we consider how we would do so at the term level.
data Tree a = Tip a  Branch (Tree a) (Tree a
elem :: Eq v => a > Tree a > Bool
elem v (Tip x) = if v == x then True else False
elem v (Branch l r) = (elem v l)  (elem v r)
We can define a very similar type family which searches a composite coproduct at the type level. Note that we will make extensive implicit use of the DataKinds
extension. For example in the following example, True
and False
are the promoted constructors.^{2}.
type family Elem e f :: Bool where
Elem e e = True
Elem e (l :+: r) = Or (Elem e l) (Elem e r)
Elem e f = False
type family Or a b :: Bool
Or False False = False
Or a b = True
This is no use to us as we must also calculate the injection, to do this, we need to know the path we took to find the value in the tree. By introducing a new data type Res
and Crumbs
we can construct the trail necessary. The modified definition of Elem
and Or
follow naturally.
data Crumbs = Here  L Crumbs  R Crumbs
data Res = Found Crumbs  NotFound
type family Elem e f :: Res where
Elem e e = Found Here
Elem e (l :+: r) = Choose (Elem e l) (Elem e r)
Elem e f = NotFound
type family Choose e f :: Res where
Choose (Found a) b = Found (L a)
Choose a (Found b) = Found (R b)
Choose a b = NotFound
Again, this is very similar to the termlevel definition but is more verbose thanks to the lack of typelevel type classes.
Now we have the path to the type, we must also construct the injection. In fact, another type class is what we need but this time thanks to our additional work we don’t need to rely on convention nor OverlappingInstances
.
class MakeInj (res :: Res) f g where
mkInj :: Proxy res > f a > g a
Notice one difference here is that as we are going to explicitly use the computed type level path, we need someway to make this information accessible to our type class. One way to do this is to use a proxy variable. At first glace it may seem useless to define a data type in this fashion but it allows us to pass around type information in exactly the way we want to here.
instance MakeInj (Found Here) f f where
mkInj _ = id
instance MakeInj (MakeInj (Found p) f l) => MakeInj (Found (L p)) f (l :+: r) where
mkInj _ = Inl . mkInj (Proxy :: Proxy (Found p))
instance MakeInj (MakeInj (Found p) f r) => MakeInj (Found (R p)) f (l :+: r) where
mkInj _ = Inr . mkInj (Proxy :: Proxy (Found p))
Notice how the proxy is used to direct the construction.
Finally we can define :<:
.
type f :<: g = MakeInj (Elem f g) f g
That is to say, f
is a subtype of g
if there is an injection from f
to g
. Notice that there is an injection if Elem f g
is not NotFound
.
Using closed type families might seem like more work but the approach is inherently more powerful. There is also something to be said about making our search strategy explicit. Without knowledge of the type class resolution mechanism it can be confusing why a certain subtyping would fail. With this implementation, even if unfamiliar with the nuances of type families, it is much clearer.
Another wart was the possibility of ambiguous injections if the same constructor was specified twice in a type signature. One example where it would be impossible to guarantee a consistent injection would be Lit :+: Lit
. We want to disallow such signatures at compile time as they are always a programming error.
To do this we can extend the Res
data type to include a new constructor Ambiguous
and altering the definition of Choose
.
data Res = Found Crumbs  NotFound  Ambiguous
type family Choose e f :: Res where
Choose (Found x) (Found y) = Ambiguous
Choose Ambiguous x = Ambiguous
Choose x Ambiguous = Ambiguous
Choose (Found a) b = Found (L a)
Choose a (Found b) = Found (R b)
Choose a b = NotFound
The choice here to use a new constructor Ambiguous
is stylistic, we could have easily reused the NotFound
constructor. The reason for doing so is that the error messages produced by a fine grained result type are better. For example now attempting to compile the following expression results in an error which specifically mentions ambiguity.
lit2 :: Expr (Lit :+: Lit)
lit2 = lit 2
No instance for (MakeInj 'Ambiguous Lit (Lit :+: Lit))
arising from a use of ‘lit’
In the expression: lit 2
In an equation for ‘lit2’: lit2 = lit 2
Failed, modules loaded: none.
We’ll wrap things up there for today but Patrick goes on to explore the possibility of further extending the subtyping relation to deal with more complicated injections. Even with the machinery he developed, intuitively correct subtyping relationships (such as Add :+: Mult :<: Add :+: Mult :+: Lit
) fail to hold. To avoid any suspense, he shows how to further extend :<:
to allow such relationships.
These ideas and more are implemented in the compdata package.
Bahr, Patrick. 2014. “Composing and decomposing data types: a closed type families implementation of data types à la carte.” Proceedings of the 10th ACM SIGPLAN Workshop on …, 71–82. http://dl.acm.org/citation.cfm?id=2633635.
Eisenberg, RA, D Vytiniotis, SP Jones, and Stephanie Weirich. 2013. “Closed type families with overlapping equations (extended version),” 1–21. http://research.microsoft.com/enus/um/people/simonpj/papers/extf/axiomsextended.pdf.
For instance a :<: (a :+: b) :+: c
would not typecheck.↩
In fact, it is possible to promote simple functions using the singletons package by Richard Eisenberg.↩
Pattern synonyms are a great feature.
It is common to define closed data types. Later if one wants to annotate their syntax tree or pass along any other additional information this can cause significant problems. A clean solution is to redefine the data type using open recursion and then taking the fixed point.^{1} This approach allows great flexibility and access to uniform recursion schemes but used to come at a cost. If the library writer had exposed the constructors, all existing user code would break with these changes. Since GHC 7.10, patterns allow us to completely swap out the underlying data type whilst maintaining the original interface.
Here’s how. Consider Hutton’s Razor.
data Hutton = Int Int  Add Hutton Hutton
Typically there will be many different operations defined as traversals of the syntax tree. One might typically be an evaluator.
eval :: Hutton > Int
eval (Int n) = n
eval (Add n m) = eval n + eval m
Later, we might want to define the unfixed version of Hutton
.
data HuttonF a = IntF Int  AddF a a deriving Functor
newtype Fix f = Fix (f (Fix f))
type Hutton = Fix HuttonF
Unfortunately, we can no longer use any of our existing functions. Fortunately, using pattern synonyms we can regain this functionality.
pattern (Int n) = Fix (IntF n)
pattern (Add x y) = Fix (AddF x y)
This is as far as GHC 7.8 will take us. To move further we need to take advantage of explicitlybidirectional pattern synonyms which are set to be introduced in GHC 7.10. These allow us to explicitly specify the constructor as well as the pattern. Useless in this case but what if we also wanted to annotate out AST?
To do so it is usual to define a product type, Annotate
,
data Annotate f = Annotate Label (f (Annotate f))
which we can then take the fixed point to get an annotated AST.
data Annotated = Annotate Hutton
Now we want to be able to keep using our existing functions for both annotated and unannotated values. The catch is that we can only have one pattern which corresponds to each constructor. To get around this we can use view patterns and a type class in order to use the same pattern for both our annotated and unannotated AST.
View
provides an injection and projection function. Then defining suitable instances for Annotated
and Hutton
, we can reuse all of our previous definitions.
class View a where
proj :: a > HuttonF a
inj :: HuttonF a > a
pattern I n < (proj > IF n) where
I n = inj (IF n)
pattern Add a b < (proj > AddF a b) where
Add a b = inj (AddF a b)
instance View Hutton where
proj = unFix
inj = Fix
instance View Annotated where
proj (Annotated _ e) = e
inj v = Annotated (mkLabel v) v
mkLabel :: HuttonF Annotated > Label
So far so good. Let’s test out our sample program.
eval :: View a => a > Int
eval (I n) = n
eval (Add a b) = eval a + eval b
p1 = Add (I 5) (I 6)
main = do
let p2 = p1 :: Hutton
let p3 = p1 :: Annotated
print (eval p2)
print (eval p3)
> ./patterns
11
11
To take stock for a moment  using technology available in GHC 7.8 we were able to swap out our standard data type and replace it with an unfixed variant. Then with machinery available to us in GHC 7.10, we were able to vary this underlying type as long as we could provide suitable mapping functions.
Looking back at our definition of inj
for Annotated
, depending on what we want the label to be, it could make little sense to fill in the annotation whilst we are building the AST. It seems sensible to separate our View
typeclass into two separate classes, one for projection and one for injection as they might not both make sense for all data types. Indeed if we introduce one more example, it is clear that they don’t.
data Holey f = Hole  Expr (f (Holey f))
type HuttonHole = Holey HuttonF
This union type represents syntax trees which might have holes in. Clearly there is no total definition of proj
so we redefine our type classes as follows.
class Project a where
proj :: a > HuttonF a
class Inject a where
inj :: HuttonF a > a
instance Inject HuttonHole where
inj = Expr
Now we should be able to use this data type very naturally to construct trees which might contain holes and later fill in the holes with whatever we please.
hole :: HuttonHole
hole = Hole
p4 :: HuttonHole
p4 = Add (I 5) hole
fillHole :: (Hole > Hutton) > HuttonHole > Hutton
fillHole f Hole = f Hole
fillHole _ (Expr v) = v
Unfortunately we have pushed pattern synonyms a little bit too far. Instead of sweet sweet compilation, we are greeted with the following error message for each of the patterns.
Could not deduce (Inject a) arising from a use of ‘inj’
from the context (Project a)
bound by the type signature for Main.I :: Project a => Int > a
at patterns.hs:1:1
Possible fix:
add (Inject a) to the context of
the type signature for Main.I :: Project a => Int > a
In the expression: inj' (IF n)
In an equation for ‘I’: I n = inj (IF n)
GHC baulks at the fact that we have different class constraints for the pattern and constructor. I don’t think there is any reason that they need to have the same constraints but it might be desirable so that constructors, even synonyms, continue to match up with patterns.
There are many powerful abstractions in Haskell which allow a library maintainer to provide a consistent interface across interactions. Unfortunately, if they ever chose to expose their internal syntax tree to the enduser, it was previously very difficult to maintain backwards compatibility. Pattern synonyms provide the perfect tool to do just this but they don’t appear to stretch quite far enough to tackle all interesting use cases.
]]>