Skip to content

Atrocit/farstorm

Repository files navigation

Farstorm

Farstorm (First Access Retrieval Structurally Typed ORM) is an ORM for PostgreSQL, breaking away from exciting ORM patterns. It was born out of frustration with the state of tools that try to provide database access in the JS/TS world. Broadly you can classify those as one of two types: either it is an ORM and it tries to abstract SQL and the database away, or it is a query builder. The thing is that we don't need query builders too badly, since most of our queries are trivial, and just writing the SQL isn't too painful. And we really don't need neavy handed ORMs with their rigid lifecycles, bad support for transactions, DSLs to define entities, horrible Hibernate-style badly supported annotations, and TypeScript breaking architectures. This project tries to automate the tedious serialization and deserialization, fetching of relations, and validating that your entities match the schema in the database, and nothing else.

So why should you care? Because this library does four things we've yet to see any other library do:

  • It keeps track of the entities loaded into memory within your transaction, and uses that to prevent the n+1 problem
  • It does not use a class based approach. This means you can use spread operators, and construct your entities piece-by-piece until it matches the schema and has enough information to persist it. No mutating necessary, you can treat your entities as fully immutable if you wish.
  • It provides a way to listen for updates on entities in batch. If you change 100 objects in a transaction, it does not call listeners 100 times individually, but once, at the end of the transaction with the list of the 100 entities in one go. This is absolutely crucial for performant code when doing operations on large numbers of entities. It is also crystal clear on whether the listener runs inside the main transaction, or in its own separate transaction.
  • It provides better typesafety for your entities than anything out there, without a codegen step or domain specific language (DSL)

How to use

Any node and npm-compatible project can use this library by installing farstorm. Here is a code snippet that shows how you can use it:

const farstorm = new Farstorm({
	type: 'postgresql',
	host: process.env['DB_HOST'] ?? 'localhost',
	port: Number(process.env['DB_PORT'] ?? '5432'),
	username: process.env['DB_USERNAME'] ?? '',
	password: process.env['DB_PASSWORD'] ?? '',
	database: process.env['DB_NAME'] ?? '',
	appName: 'myApplicationName',
	ssl: false,
	poolSize: 5,
}, {
	'TodoItem': defineEntity({
		fields: {
			id: defineIdField(),
			description: defineField('string', false),
			createdAt: defineAutogeneratedField('Date'),
			finishedAt: defineField('Date', true),
		},
		manyToOne: {
			assignee: { entity: 'User', nullable: true },
		}
	}),
	'User': defineEntity({
		fields: {
			id: defineIdField(),
			fullName: defineField('string', false),
		},
		oneToMany: {
			todoItems: { entity: 'TodoItem', inverse: 'assignee' },
		},
	})
});

// The pattern below would be an awful N+1 pattern in most other ORMs,
//  but will only execute 2 queries here, no matter how many todos and unique assignees exist
await farstorm.inTransaction(async orm => {
	const openTodoItems = await orm.findMany('TodoItem', { where: sql`finished_at is null` });
	for (const todoItem of openTodoItems) {
		console.log('Open item: ', todoItem.description, ' -- for user ', (await todoItem.assignee)?.fullName ?? 'UNASSIGNED');
	}
});

Drivers

  • PostgreSQL: Fully supported out of the box via the postgresql driver (uses the pg package).
  • PgLite: An optional in-process PostgreSQL-compatible driver. If you use type: 'pglite', you must install the PGlite package yourself, as it is not a required dependency of Farstorm.

Installation when using PgLite:

  • npm: npm install @electric-sql/pglite
  • yarn: yarn add @electric-sql/pglite

Example usage with PgLite:

import { Farstorm, defineEntity, defineIdField, defineField, sql } from 'farstorm';

const orm = new Farstorm({
	type: 'pglite',
	// optional: databasePath or file config per @electric-sql/pglite docs
}, {
	// Define your entities here as usual
});

If @electric-sql/pglite is not installed and you instantiate the PgLite driver, Farstorm will throw a clear runtime error indicating you need to install it.

License

This projected is licensed under the ISC License, a permissive open source license. The short summary is that you can use this project for any purpose you want, but none of the authors or contributors are liable for anything.

Documentation

Aside from the comments in the code, there currently is no documentation to speak of. Sorry.

Goals

  • Provide a clear API to access the database, with few footguns
  • Allow enabling the strictest TypeScript type checks in tsconfig.json, specifically strictNullChecks and strictPropertyInitialization
  • Allow constructing entities dynamically, instead of forcing rigid classes. If { x: 1, y: 2 } is a valid entity, then calling orm.save({ x: 1, y: 2 }) should work, as well as orm.save({ ...({ x: 1 }), ...({ y: 2 }) })
  • Fix or avoid the n+1 query problem
  • Fix or avoid the eager loading multiplication problem
  • Provide a decent event listener system with clear semantics around executing changes inside/outside of the triggering transaction
  • Clearly map ORM operations to database operations, there should be as little magic as possible going on

Non-goals

  • While the structure of the package is such that in theory drivers for multiple databases can be added, it is not currently a goal to support databases other than PostgreSQL at this time. Lots of the code is tightly coupled to the PostgreSQL dialect of SQL. Due to their compatibility, databases like pglite and Amazon Aurora may be supported.
  • Exhaustively supporting all database features and query styles. The fallback case is always to use a function to query the database using raw SQL.

Limitations

There are a number of things Farstorm does badly, or decisions that may render it unsuitable for you. We may resolve these in the future, and help on these is certainly appreciated:

  • Currently we only support bigserial id columns, and bigint foreign key columns. If you use UUIDs, or you don't name your primary key for every table id, you are out of luck for now
  • There is no built-in many-to-many support. You can fake it by using two one-to-many relations ofcourse, and the moment you need more metadata added in the relation, this is probably the approach you'd want anyway.
  • There is no query builder or type checked custom queries. It's on the wishlist, but if you write loads of complex queries and want something to do static typing on that, you're in the wrong place right now.
  • Transactions are mandatory. You cannot execute a query without starting a transaction. This is a deliberate design choice, and not likely to change in the future. We don't think it's a big deal because you probably should use transactions ANYWAY, but it's worth mentioning.
  • Due to the complexity of the types, especially on large schemas with many interlinked entities, TypeScript may get a bit sluggish.

Versioning

Farstorm follows the Semantic Versioning, or SemVer. This means the version numbering scheme consists of these three segments: MAJOR.MINOR.PATCH, which increase according to semver guidelines.

For all three cases the CHANGELOG.md file should be updated to explain what has changed in a concise and human-readable way.

Releasing

Assuming you have write access to the repository for Farstorm, including write access to the package hosting, releasing can be done like this:

Step 1: Build by running yarn build
Step 2: Run yarn changeset version, the new version will automatically be determined by the changesets
Step 3: Review and commit the changes made by changeset Step 4: Run yarn changeset publish Step 5: git push; git push --tags the newly created commit onto master, including the version tags

About

Farstorm - First Access Retrieval Structurally Typed ORM

Resources

License

Stars

Watchers

Forks

Packages

No packages published