refactoring the site

Written: 2021-09-06 to 2021-09-06
by Norman Liu

It’s been a WHILE since I’ve updated this site, really sorry to the 5 people that follow me. Life’s been pretty busy.

I finally won the hiring lottery back in May, and I’m pretty pleased to say that I’m gainfully employed and no longer starving for recruiter attention creative outlets. My stack at work doesn’t include Haskell (yet), but a ton of Scala and Rust. Scala’s been a blast so far, especially coming from a functional background. Plenty of concepts map directly between the two worlds (type classes -> implicits, type families -> abstract types/defs, trash outdated IO API’s -> java.nio, etc).

There’s plenty to be desired though, and most of my developer pains tend to stem from a few sources.

  1. Long compile times.

I always thought GHC was kinda slow when compiling projects for the first time, given that this blog takes ~10 minutes to build from scratch (most of that stems from Pandoc and JuicyPixels). Recompiling is fast though, with a rebuild rarely taking longer than a second. Scala projects take exponentially longer, to the point where it starts to really impact your developer experience. On a medium-sized codebase (~100 classes or so) with heavy use of implicits, recompiling a single file can take me 10 seconds or more. Some quick profililing during a long build showed me that most of that time is spent on implicit resolution, with the compiler having to shovel hundreds of thousands of rejected implicits into the garburetor GC. This issue is made worse with heavy use of libraries like shapeless, which enable SYB-style generic programming. Implicits in Scala 3 have been heavily revised, although it’ll be a while until people start migrating en masse to the new ecosystem. I haven’t benchmarked Scala 3 vs Scala 2’s compile times on anything larger than small example projects, although older articles seem to indicate that Scala 3 is slower.

This issue sometimes extends to my IDE, as well. I use Metals in VSCode for most of my work, and editor feedback sometimes slows to a crawl when the compiler has to re-resolve implicits and macro annotations. This is arguably more annoying than the long compile times alone, given your workflow tends to directly depend on the responsiveness of your development environment. Personally, I have a habit of saving after every line, to make sure what I’m writing is coherent and that my types check out. On those rare occasions where Metals takes 10+ seconds to tell me whether or not I’m being a dumbass, it’s incredibly easy to lose my train of thought and the pace of development slows down to a crawl.

  1. Verbosity.

Scala is EXPONENTIALLY better than Java due to ADT’s, improved syntax, and its facilities for programming in a properly functional style. It still tends to show its warts in unexpected areas, largely due to limitations of running on the JVM and conscious decisions by its authors. For instance, a simple sum type is a one-liner in ML-style languages:

data SumType a = Foo Int | Bar Char | SumType a a

but unifying disjoint structures in Scala is a little bit more involved:

sealed trait SumType[A] extends Enumeration {
  final case class Foo(thisIntNeedsAName: Int) extends SumType[A]
  final case class Bar(alsoNeedsAName: Char) extends SumType[A]
  final case class SumType(left: A, right: A) extends SumType[A]
}

Ow, my pinky fingers hurt already. This is a vanilla Scala 2 implementation, by the way. There are libraries like enumeratum that provide macros for easier declaration of sum types. Enums in Scala 3 have also been vastly improved, although you still have to dance the extends shuffle.

On the flip side, you get GADTs for free:

{-# LANGUAGE GADTs #-}
data GADT a where
  SomeInt :: Int -> GADT Int
  SomeChar :: Char -> GADT Char
  SomeMaybe :: Maybe a -> GADT (Maybe a)
  -- and so on

unwrapGADT :: GADT a -> a
unwrapGADT (SomeInt i) = i
unwrapGADT (SomeChar c) = c
unwrapGADT (SomeMaybe mb) = mb
sealed trait GADT[A] {
  def unwrapGADT: A
}

object GADT {
  final case class SomeInt(unwrapGADT: Int) extends GADT[Int]
  final case class SomeChar(unwrapGADT: Char) extends GADT[Char]
  final case class SomeMaybe(unwrapGADT: Option[A]) 
    extends GADT[Maybe[A]]
}
  1. Janky implicits (and more verbosity).

Implicits are also first-class, compared to type class instances which sort of float around in the ether and can only be accessed through their members. This is either a hindrance or a plus, depending on the context.

Let’s give an example of some type class, like Functor. It’s not built into Scala, but provided by cats, which is pretty widely used across projects.

class Functor f where
  fmap :: (a -> b) -> f a -> f b

data Id a = Id {getId :: a} 

instance Functor Id where
  fmap f (Id a) = Id (f a)
abstract class Functor[F[_]] {
  def fmap[A, B](func: A => B)(fa: F[A]): F[B]
}

final case class Id[A](getId: A) extends AnyVal

object Id {

  implicit def functorIdent: Functor[Ident] = new Functor[Ident] {
    def fmap[A, B](func: A => B)(fa: Ident[A]): Ident[B] = Ident(func(fa.getId))
  }

}

The verbosity is out of control, and thousands of Scala developers will begin to flood hospitals with symptoms of RSI in the next few decades. You need to name all your instances (just like Purescript, blech), and instances are namespaced not by module (package objects), but by companion objects. At least associated implicits are imported along with their source classes, but this leads to a bloatload of imports with more complex uses of implicits. Naming instances also requires you to declare types twice (_: Functor[_] = new Functor[_] {...}), which is pretty unavoidable under most linting rules. Top level declarations usually need types, and you can’t infer the type of an instance even when you assign a type to the expression. This, for example, won’t type-check:

implicit def functorIdent: Functor[Ident] = new Functor {
  ...
}

and you end up with esoteric errors about type variance since the instance’s type is inferred to be Nothing (a subclass of every type). Another issue is actually using these instances. You can write parametrically polymorphic functions constrained by the presence of implicits, just like type class constraints in Haskell:

def mapCompose[F[_]: Functor]
  (ab: A => B, bc: B => C)
  (fa: F[A]): F[C] = {...}

which means “Give me two functions, one from a type A to B, the other from a type B to C, as well as a value of type A wrapped inside a container for which there’s an implicit Functor object floating around, and I’ll give you the result of mapping the composition of these functions into that container.” That’s a mouthful, thank god natural language programming never took off. The key is that this function will work for any class with an associated Functor instance, but we can’t actually use that instance unless we name it.

def mapCompose[F[_]: Functor]
  (ab: A => B, bc: B => C)
  (fa: F[A]): F[C] = {
    val functor: Functor[F] = implicitly
    functor.fmap {a => bc(ab(a))} ( fa )
  }

We can name the implicit value directly in the arguments, but that gets ugly:

def mapCompose[F[_]]
  (ab: A => B, bc: B => C)
  (fa: F[A])
  (implicit functor: Functor[F]): F[C]= {...}

On the plus side, libraries like cats provide package objects with all the implicits you need, so invoking type class methods is a lot cleaner:

Functor[F].fmap(...)

Implicits are immensely powerful - they’re type classes, TypeFamilies, ImplicitParams, FlexibleInstances, and the Reader monad baked right into the language. You can even define macros that are invoked implicitly, which transforms your build times into a Samuel Beckett homage. I guess it’s no different from Template Haskell, although implicits being built-into the language leads to much easier and pervasive misuse. You also can’t declare implicits at top level, although you can use package objects to make implicits visible across an entire namespace level. Scala 3’s given and using features promote implicits to simple, first-class, scoped, contextual abstractions, which gives me a ton of ideas for Koka-like effect systems. Scala 2 is still stuck with raw implicits for now.


In short, work is great. Being able to use cutting-edge tech in production is exciting, and there’s no better feeling than being surrounded by big-brain coworkers with plenty of experience and new knowledge to impart. I’m just sharing some gripes about Scala that I’ve faced in the last few months, and that have no doubt been improved/outright solved in recent releases.

Oh yeah, I have zero gripes about Rust. Rust is love. Rust is life.