I’ve spent the last 1.5 years parttime building classy.school (currently only available in Dutch). It’s a SaaS application for music schools in Belgium.
I've built this application in a programming language that is almost completely unknown to most programmers. That language is called Ur/Web. It’s a shame that it's so obscure because according to me it’s both technically amazing and extremely practical. In this post I’ll talk about:
Before we begin, a short introduction. Ur/Web is a purely functional, statically typed programming language designed specifically for making web applications. It was created by Adam Chlipala, Associate Professor of Computer Science at MIT.
Ur/Web is not for everybody. I'm in a position that makes Ur/Web an amazing solution to my problems. This position might not be yours.
I've added comparable code in JavaScript/TypeScript for the sections where it makes sense. I’ll do this in a minimal amount of lines of JS/TS, since it’s the mainstream programming language I am most familiar with.
// server.ts
import * as express from "express";
let app = express();
app.get("/users", (req, res) => {
res.send({id: 1, userName: “Simon”});
});
// client.ts
fetch(“/users”).then(users => users.forEach(u => console.log(u.firstName)));
// prints undefined, the property is called userName
Keeping frontend and backend code in sync is pretty hard. Often the data model that is being used on the backend somehow deviates from what’s used in frontend code, for a variety of reasons. It’s been a drag on every relatively big project I’ve worked on. There are many (partial) solutions for this. In most mainstream languages, you can rely on manual work and dilligence (bad) or on code generation and IDE functionality (better). Even code generation has serious downsides. To me, the most important one is that the methods that I know of (eg: Swagger) don’t have support for vital data structures. Tagged unions, option types, dates (with and without timezones), times, etc are often not supported, or not supported well.
-- users.ur
fun getUsers () = return ({Id = 1, UserName = "Simon"} :: [])
fun main (): transaction page = return <xml>
<body>
<button onclick={fn _ => users <- rpc (getUsers ());
List.app (fn u => debug u.FirstName) users}/>
</body>
</xml>
(* /home/simonvancasteren/ur-proj/blog/main.ur:6:29: (to 6:100) Stuck unifying these records after canceling matching pieces:
Have: ([FirstName = string]) ++ <UNIF:U98::{Type}>
Need: [Id = int, UserName = string]
*)
Ur/Web frontend and backend code is always in sync. You use one codebase for all your tiers and the typechecker makes sure everything lines up. Marshalling of values is done by the compiler. You can’t define your own marshalling. I was used to writing serializers and deserializers in PureScript, but I’ve come to appreciate what Ur/Web does. So far I’ve not had any problems with marshalling between frontend and backend code.
const pgp = require('pg-promise')();
const db = pgp('postgres://postgres@localhost:5432/blog');
let q = 'select userName from users'
db.each(q, [], users => users.forEach(u => console.log(u.firstName)));
// prints undefined, the property is called userName
When interacting with databases, the same kind of synchronization problems tend to occur. The database schema is somehow not in sync with the application code. Some mainstream languages try to solve this with typed ORM's or IDE functionality. Visual Studio works pretty well in this regard.
table users: { Id: int
, UserName: string
}
fun main2 (): transaction page =
users <- queryL1 (SELECT users.UserName
FROM users);
List.app (fn u => debug u.FirstName) users;
return <xml></xml>
(* /home/simonvancasteren/ur-proj/blog/main.ur:18:4: (to 22:3) Stuck unifying these records after canceling matching pieces:
Have: ([FirstName = string]) ++ <UNIF:U48::{Type}>
Need: [UserName = string]
*)
In Ur/Web, you define tables with the "table" keyword. You then write embedded SQL code to query (or insert/update) these tables. Table definitions get output as a .sql file containing CREATE TABLE... statements. The embedded SQL code gets executed in your DB (currently supports sqlite, mysql and postgresql). I say embedded SQL becuase the compiler parses this syntax and translates it into its own AST. This allows it to do typechecking and avoid SQL injection. Later on the actual SQL code is generated from that AST.
The type system level encoding of SQL in UrWeb is awesome. It has support for pretty much everything I need in my day to day programming. Among others: INNER JOIN, OUTER JOIN, UNION, GROUP BY, ORDER BY, window functions, etc. All VERY strictly typed. The type-level encoding is beautiful and enough to warrant its own blog post.
Ur/Web takes care of marshalling base datatypes in and out of SQL and allows embedding non-base types (like tagged unions) with the use of the serialize keyword. This explicit serialization can get annoying though. Also, some types are not supported (like the various date and time types of PostgreSQL) and using them means adding them to the compiler. I've managed without this, but a good solution for this would be great.
fun main3 (): transaction page =
countSource <- source 0;
return <xml>
<body>
<span>Hello world</span>
<dyn signal={count <- signal countSource;
return <xml> {[count]} </xml>}/>
<button onclick={fn _ => count <- get countSource; set countSource (count + 1)}>
Add to counter
</button>
</body>
</xml>
Two things here I want to mention. First of all, you tag dynamic parts of your page with the "dyn" (or "active") tags. This allows Ur/Web to do server-side rendering for everything else. The "Hello world" span will be sent as HTML to the client, so no JavaScript rendering for that part. The "dyn" tag will not be server-side rendered, but rather rendered using JavaScript as the content of this tag can change during the lifecycle of the page. This is great for a few reasons, but I like it mostly for performance. Ur/Web's server side is insanely fast. And client-side rendering code only needs to rerender limited parts of the page.
Second thing I want to mention here is the frontend "framework". It works extremely well. I'm mostly a frontend programmer and have experience in just about every single JavaScript frontend of the last 10 years. I am more productive with this than with any alternative I've tried. It's very simple for small dynamic things and can scale up to really high complexity. In the more complex cases I often use an "Elm Architecture" approach, which is easily implemented on top of the sources and signals of Ur/Web's frontend framework.
No code here, just two things. First of all, server-side performance is amazing. Urweb has been doing really well on Techempower's benchmarks up until round 16, at which point something broke, not sure what, probably Docker-related. Other languages that focus on functional programming, correctness, etc. often incur a performance penalty. Ur/Web is the combobreaker: its performance is at the top of the charts.
Secondly, client-side performance is also very good. In terms of code generation, the generated code size seems large at first. My bundle ends up at around 1.7MB of javascript. However, after brotli level 4 compression it ends up at around 140kB, and level 11 compression even brings it down to 90kB. This is extremely small for the amount of logic that is being shipped (no code splitting or lazy loading is being done) and is mostly thanks to the way that Ur/Web frontend code is encoded into a special format. Parsing is also very fast because a lot of code is shipped as strings and eval'd later.
Frontend runtime performance has not been an issue for me at all, but I haven't done any extensive benchmarks, simply because I've not felt any kind of slowdown at all in client-side rendering speed. Not having to rerender the whole page (as explained in this section) definitely helps a lot, as does the source/signal architecture.
// server.ts
const myValue = anUnknownFunction();
// Thankfully, TypeScript can tell me what type myValue has.
// But, does anUnknownFunction have side effects? Can I call this twice and expect the same result?
// I can't tell without inspecting its source code.
Knowing whether or not a function causes side effects is very important information, on par with its type and its name. Very few languages support this kind of "impure" tagging today. Only the Haskell family of languages does so and it's something I miss every day when writing TypeScript. There is no real substitute for this in mainstream languages. The only thing you can do is tag impure functions yourself, but without compiler support, this is a losing battle.
val anUnknownFunction (): transaction string = ...
(* Does this function have side effects? *)
(* Yes, its type says so, and the compiler enforces "bubbling" of side-effects *)
(* In case a function doesn't contain a transaction type, I'm sure it's pure and calling it with the same arguments will give me the same value every time *)
Luckily, Ur/Web does have a compiler-enforced side-effect type in the form of the transaction type. Not much to say about it, apart from the fact that it completely changes how I write programs, in the best way possible.
This might be a point of contention. Even though Ur/Web is not used by a lot of people currently, there is some really good documentation available. It does require a bit of searching however to figure out where you should be looking for what.
Ur/Web's main website can be found at www.impredicative.com/ur. Its design is what you'd call, uhm, "spartan". Once you're used to it, it's actually very effective. Tons of good information on just a couple pages. Unfortunately todays programmers are used to fancy websites introducing programming tools in a way that is similar to commercial marketing pages. Anything that is not fancy and "modern-looking" is deemed irrelevant.
I myself am also guilty of this: It took me at least five seperate visits to the Ur/Web website before I actually dug in to figure out what it was about. The first four times I bounced immediately, thinking the project was irrelevant and/or abandoned. That's unfortunate!
Go to www.impredicative.com/ur/demo for a look at how to do all kinds of basic and more complicated things in Ur/Web. It starts of slowly and is really easy to follow. This should be your first stop when looking into Ur/Web. In my opinion, this should probably be called "the tutorial".
Go to www.impredicative.com/ur/tutorial for what's officially known as the tutorial. It's a different format as the demo, but goes through some different concepts. This tutorial is actually much more work to go through, since it introduces you to Ur/Web's advanced type system. Again it's very well written, but don't despair if it takes you some time to get through it.
Go to github.com/urweb/urweb/tree/master/tests to look at all the tests in the compiler repo. This helped alot starting out, especially concerning syntax and the embedded SQL stuff. I like to learn from examples, and these tests are just that.
Go to www.impredicative.com/ur/manual.pdf to download the pdf manual. This is an amazing document that contains pretty much everything there is to know about Ur/Web. How to install and use the compiler, Syntax, Semantics, the FFI, the Stdlib and even an overview of every compiler phase! This is such a valuable document.
There is no forum, discord or IRC channel. But there is a mailing list that I turn to if I really don't know how to do something. There is a small but dedicated core of people that go out of their way to answer your questions there. Not having an easy way to search through this is annoying though.
When I started out with Ur/Web, it was pretty hard to figure out what the compiler was actually trying to tell me. One reason is that often it'll spit huge amounts of type definitions in your face, representing the context of the error seen through the eyes of the typechecker. 99% of the time all this is useless and you can just look at the very first and very last lines of an error message. Ideally this should be something that the compiler error reporting itself handles, but currently it doesn't.
Another reason is that some pretty common errors get (initially) difficult to understand error descriptions. I have a list of common compiler phrases and some possible causes for this. Again, this is something that the compiler can improve upon, but so far it doesn't.
For me personally, in my daily programming of Ur/Web stuff, I don't have much issues figuring out what the compiler is trying to tell me in 90% of cases. But if you're just starting out, this will be difficult.
I really miss having type-under-cursor and autocompletion-with-types functionality in Ur/Web. This is implemented very well in most mainstream languages. TypeScript even has context-sensitive autocompletion and type-under-cursor using their data-flow analysis. This works great! I really wish Ur/Web would have that as well. The basic building blocks are there: Ur/Web has a "daemon" that caches type information, so hooking into this to perform both these features seems possible. Not implemented yet though.
UPDATE: A few months ago I developed an initial version of an lsp server for the Ur/Web compiler. Check out the Ur/Web manual for help in setting it up with your favorite editor!
Importing other Ur/Web projects into your project is very easy. Your project file references another project file and you've got yourself an import. However, the compiler just looks at the file system to find these projects. Getting these other projects there, building/linking them and keeping them up to date (aka. Package Management) is left to you.
Personally, I haven't had much problems with it. I am however always very very conservative with including third-party code into my projects. My project has only four dependencies at the moment. I've built a small custom solution on top of Nix to manage this, but it's not something that can be reused by others and still includes some manual work. I know of https://github.com/grwlf/urweb-build but I was unhappy with the complexity of this tool. Building on Nix however is always a good idea in my opinion though.
I'll be clear here: The amount of Ur/Web packages available is tiny. I haven't found this to be a big problem though: The language and standard library take care of most of my needs. Remember: you don't need a web server library, you don't need a front end framework and you don't need an ORM or data access library. Both FFI's are pretty easy to use (though more examples would be very useful) and I've used them both to integrate with C libraries like OpenSSL and browser functionality like requestAnimationFrame.
At the time of writing, this is my biggest gripe with Ur/Web. The project mentioned above, which I have to admit is quite a big project, has gotten slower to build.
Thankfully, thanks to the lsp server mentioned above, the typechecking feedback loop is still very fast. It takes about 5 seconds to get typechecker feedback on my very biggest file. Most files are instant. If you're working with Ur/Web, use it!
However, actually building the executable has become a bit slow. It takes about 2 minutes to build the whole executeable from scratch. This is something that I'd love to see improved. I've done some compiler experiments to get this feedback loop faster, but this still needs a lot of work. If anyone has any ideas on how to improve this, let me know. I'm definitely interested in helping out with this!
I hope this will give you some insight into what Ur/Web is and what benefits and drawbacks it brings. I'm happy to answer any questions about it, you can find my information here. I realize this post glosses over thousands of bigger and smaller details, but it's already more than long enough... If you're missing something, let me know and I'll see if I can make a post about it.