Welcome to the Golang port of Overleaf!
This repository is a monorepo for all the backend services and other useful command line utils. The frontend code remains in a fork of the Node.js monorepo.
- The backend code is written in a statically typed language
- Monolith-first approach, eliminating latency overhead and network chatter -- just run this single binary, and you are good (compiles can still be scaled horizontally, other components as well - but they are self-contained and do not expose internal APIs on the network)
- Mongodb was replaced with PostgreSQL (an automated data migration exists via
cmd/m2pq
, you can find a mongodb implementation and edgedb implementation in the git history too) - Postgres enables single round-trip data fetching for complex pages like the project dashboard or the project editor page
- Postgres enables atomic updates to the filetree and strong unique constrains inside the tree
- No admin capabilities in the web app (command line utils exist instead)
- Highly optimized compile process (<200ms are possible for incremental compiles, <20ms for synctex aka navigating between code/PDF location, zero content is fetched from Postgres for incremental compiles)
- Homegrown real-time service with minimal overhead for connection startup (zero server round trips inside the websocket for "joining" a project)
- JWT based authentication and authorization for many endpoints (epochs in the DB enable invalidation of tokens ahead of their expiry)
- Highly optimized document updating process (merged redis keys reduce redis calls by an order of magnitude, re-using of already fetched content for processing multiple updates and flushing them all the way to history/DB)
- Worker-stealing for spellchecker (this allows less popular languages to take over idle worker from a more popular language)
- Chaining of network proxies for fetching external resources (multiple hops between proxies allow stronger firewalling of services, e.g. hop to local service and then hop to external cloud -- only that local service needs internet access)
- Stronger CSP and zero inline scripts in HTML templates
- Minify and compile HTML templates at boot-time in well under 1s
- Caching for documentation content and images
- Website translations on a single domain
- No local filesystem backend for storing binary files (all major providers expose an S3 compatible API today and min.io can be used for self-hosting)
There are little to no run-time defaults and strict validation of options is in place for flagging missed/invalid directives in your config.
You can use the config generator which has opinionated options and auto-detection for Docker details (rootful, rootless, Mac).
With Golang installed and docker-compose v2:
go run ./cmd/config-generator | tee .env
docker compose up
Without Golang installed
-
Without Golang installed on Linux and docker-compose v2:
docker run --rm -v `pwd`:`pwd` -w `pwd` golang:1.22.6 \ go build -tags osusergo,netgo ./cmd/config-generator ./config-generator | tee .env docker compose up
-
Without Golang installed on an Intel based Mac and docker-compose v2:
docker run --rm -v `pwd`:`pwd` -w `pwd` \ -e GOOS=darwin -e GOARCH=amd64 golang:1.22.6 \ go build -tags osusergo,netgo ./cmd/config-generator ./config-generator | tee .env docker compose up
-
Without Golang installed on a M1/M2 based Mac and docker-compose v2:
docker run --rm -v `pwd`:`pwd` -w `pwd` \ -e GOOS=darwin -e GOARCH=arm64 golang:1.22.6 \ go build -tags osusergo,netgo ./cmd/config-generator ./config-generator | tee .env docker compose up
Note: For more advanced usages (e.g. email delivery), see
$ go run ./cmd/config-generator --help
or$ ./config-generator --help
.
Docker-compose will build all the images, binaries and frontend assets, run the minio setup, create the postgres schema and finally start it all up!
The first startup may take a while to complete. Watch out for a message that says "Setup finished!".
You can now open http://localhost:8080/register in your favorite browser and create your first account!
Tip: Use
$ docker compose up --detach
to keep the services running in the background.