Improve website's grid layout and add Payload CMS benchmarks results.

ignatz · ignatz · commit 51e55aecaee9 · 2024-11-18T15:32:42.000+01:00
diff --git a/docs/src/components/SplitCard.astro b/docs/src/components/SplitCard.astro
@@ -0,0 +1,31 @@
+---
+import { Card } from "@astrojs/starlight/components";
+
+export interface Props {
+  title: string;
+  icon?: any;
+  reverse?: boolean;
+}
+
+const { title, icon, reverse } = Astro.props as Props;
+---
+
+<Card title={title} icon={icon}>
+  <div
+    class:list={[
+      "flex",
+      "flex-col",
+      reverse ? "md:flex-row-reverse" : "md:flex-row",
+      "gap-2",
+      "md:gap-8",
+    ]}
+  >
+    <div class="md:w-[45%]">
+      <slot name="first" />
+    </div>
+
+    <div class="md:w-[55%]">
+      <slot name="second" />
+    </div>
+  </div>
+</Card>
diff --git a/docs/src/content/docs/index.mdx b/docs/src/content/docs/index.mdx
@@ -22,6 +22,9 @@ hero:
 
 import { Image } from "astro:assets";
 import { Aside, Card, CardGrid } from "@astrojs/starlight/components";
+import SplitCard from "@/components/SplitCard.astro";
+
+import Roadmap from "./_roadmap.md";
 
 import screenshot from "@/assets/screenshot.webp";
 import dotnetLogo from "@/assets/dotnet_logo.svg";
@@ -30,64 +33,60 @@ import tsLogo from "@/assets/ts_logo.svg";
 
 import { Duration100kInsertsChart } from "./reference/_benchmarks/benchmarks.tsx";
 
-<div class="flex justify-center">
-  <div class="md:w-[80%] flex flex-col gap-4">
+<div class="flex flex-col gap-4">
 
-    <div>
-      <h3>Total time for 100k insertions</h3>
+  <SplitCard title="Performance" icon="rocket">
+    <div slot="first">
+      Blazingly fast thanks to its constituents:
 
-      <a href="/reference/benchmarks">
-        TrailBase adds minimal overhead compared to in-process SQLite and
-        beats other excellent choices:
-      </a>
+      * Rust: one of the lowest overhead languages,
+      * Axum: one of the fastest HTTP servers,
+      * SQLite/Libsql: one of the fastest full-SQL databases,
+      * V8: one of the fastest JS engines.
 
-      <div class="w-full h-[260px] mt-4">
+      TrailBase APIs are [6-7x faster than PocketBase's and 15x faster than SupaBase's
+      needing only a fraction of the footprint](/reference/benchmarks), allowing
+      you to serve millions of customers from a tiny box.
+      TrailBase JS runtime is ~40x faster than PocketBase's.
+    </div>
+
+    <div slot="second">
+      <a href="/reference/benchmarks">Total time for 100k insertions:</a>
+
+      <div class="w-full h-[300px]">
         <Duration100kInsertsChart client:only="solid-js" />
       </div>
     </div>
+  </SplitCard>
+
+  <Card title="Admin Dashboard" icon="setting">
+    TrailBase ships with a builtin admin dashboard UI, see demo above, that
+    lets you quickly configure your instance and visually explore your data.
+    Following TrailBase's mantra of not getting in your way, the UI is
+    entirely optional letting you fall back to a purely config &
+    migration-based setup for integration tests or managing an entire fleet
+    of deployments.
 
-    #### Live Demo
+    Check out the **live demo**:
 
-      <div class="relative" >
+    <div class="flex justify-center">
+      <div class="max-w-[680px] relative" >
         <a href="https://demo.trailbase.io/_/admin">
           <Image class="z-0 rounded-xl" src={screenshot} alt="Screenshot of TrailBase's admin dashboard" />
         </a>
 
         <div class="z-1 w-full h-full absolute top-0 flex justify-center items-center pointer-events-none">
           <a class="pointer-events-auto no-underline flex flex-col items-center bg-gray-200 dark:bg-accent-900 px-4 py-2 rounded" href="https://demo.trailbase.io/_/admin">
+            <strong>Live Demo</strong>
             <span>login: admin@localhost</span>
             <span>password: secret</span>
           </a>
         </div>
       </div>
+    </div>
+  </Card>
 
-    <Aside type="caution" title="Early Days">
-      TrailBase is very young.
-      You can expect many new features but also moving APIs until things settle.
-      We'll make sure to follow semantic versioning and welcome brave, early
-      adopters.
-    </Aside>
-
-  </div>
-</div>
-
-<div class="w-full py-8">
-
-  <CardGrid stagger>
-
-    <Card title="Performance" icon="rocket">
-      Blazingly fast thanks to its constituents:
-
-      * Rust: one of the lowest overhead languages,
-      * Axum: one of the fastest HTTP servers,
-      * SQLite/Libsql: one of the fastest full-SQL databases.
-      * V8: one of the fastest JS runtimes
-
-      TrailBase APIs are [6-7x faster than PocketBase and 15x faster than SupaBase
-      needing only a fraction of the footprint](/reference/benchmarks), allowing
-      you to serve millions of customers from a tiny box.
-      TrailBase JS runtime is ~40x faster than PocketBase's.
-    </Card>
+  <CardGrid>
 
     <Card title="Simple" icon="heart">
       TrailBase is a small, single file, static binary that is incredibly easy
@@ -100,15 +99,6 @@ import { Duration100kInsertsChart } from "./reference/_benchmarks/benchmarks.tsx
       let you move faster, more confidently and pivot when necessary.
     </Card>
 
-    <Card title="Admin Dashboard" icon="setting">
-      TrailBase ships with a builtin admin dashboard UI, see demo above, that
-      lets you quickly configure your instance and visually explore your data.
-      Following TrailBase's mantra of not getting in your way, the UI is
-      entirely optional letting you fall back to a purely config &
-      migration-based setup for integration tests or managing an entire fleet
-      of deployments.
-    </Card>
-
     <Card title="Authentication" icon="open-book">
       TrailBase comes with an authentication system and UI built in supporting
       both password-based and Social/OAuth (Google, Discord, ...) sign-ups.
@@ -151,9 +141,10 @@ import { Duration100kInsertsChart } from "./reference/_benchmarks/benchmarks.tsx
     </Card>
 
   </CardGrid>
+
 </div>
 
-import Roadmap from "./_roadmap.md";
+<div class="h-8" />
 
 <div class="flex justify-center">
   <div class="md:w-[80%] flex flex-col items-center gap-4">
diff --git a/docs/src/content/docs/reference/_benchmarks/benchmarks.tsx b/docs/src/content/docs/reference/_benchmarks/benchmarks.tsx
@@ -18,6 +18,7 @@ type Datum = {
 };
 
 const colors = {
+  payload: "rgb(0, 101, 101)",
   supabase: "rgb(62, 207, 142)",
   pocketbase0: "rgb(230, 128, 30)",
   pocketbase1: "rgb(238, 175, 72)",
@@ -57,37 +58,41 @@ function transformMillisecondTicks(
   }
 }
 
-const durations100k = [
-  {
+const durations100k = {
+  payload: {
+    label: "Payload v3+SQLite",
+    data: [656.09],
+    backgroundColor: colors.payload,
+  },
+  supabase: {
     label: "SupaBase",
     data: [151],
     backgroundColor: colors.supabase,
   },
-  {
+  pocketbase_ts: {
     label: "PocketBase TS",
     data: [67.721],
     backgroundColor: colors.pocketbase0,
   },
-  // {
-  //   label: "PocketBase Dart (AOT)",
-  //   data: [62.8136],
-  // },
-  {
+  pocketbase_dart_aot: {
+    label: "PocketBase Dart (AOT)",
+    data: [62.8136],
+  },
+  pocketbase_dart_jit: {
     label: "PocketBase Dart (JIT)",
     data: [61.687],
     backgroundColor: colors.pocketbase1,
   },
-  {
+  trailbase_ts: {
     label: "TrailBase TS",
     data: [16.742],
     backgroundColor: colors.trailbase0,
   },
-  // {
-  //   label: "TrailBase Dart (AOT)",
-  //   data: [11.1],
-  // },
-  {
-    // label: "TrailBase Dart (JIT)",
+  trailbase_dart_aot: {
+    label: "TrailBase Dart (AOT)",
+    data: [11.1],
+  },
+  trailbase_dart_jit: {
     label: "TrailBase Dart",
     data: [9.4247],
     backgroundColor: colors.trailbase1,
@@ -96,22 +101,46 @@ const durations100k = [
   //   label: "TrailBase Dart (JIT + PGO)",
   //   data: [10.05],
   // },
-  // {
-  //   label: "TrailBase Dart (INT PK)",
-  //   data: [8.5249],
-  //   backgroundColor: colors.trailbase2,
-  // },
-  {
+  trailbase_dart_jit_int_pk: {
+    label: "TrailBase Dart (INT PK)",
+    data: [8.5249],
+    backgroundColor: colors.trailbase2,
+  },
+  drizzle: {
     label: "In-process SQLite (Drizzle)",
     data: [8.803],
     backgroundColor: colors.drizzle,
   },
-];
+};
 
 export function Duration100kInsertsChart() {
   const data: ChartData<"bar"> = {
-    labels: ["Time [s] (lower is better)"],
-    datasets: durations100k as ChartDataset<"bar">[],
+    labels: ["Time in seconds (lower is faster)"],
+    datasets: [
+      durations100k.supabase,
+      durations100k.pocketbase_ts,
+      durations100k.pocketbase_dart_jit,
+      durations100k.trailbase_ts,
+      durations100k.trailbase_dart_jit,
+      durations100k.drizzle,
+    ] as ChartDataset<"bar">[],
+  };
+
+  return <BarChart data={data} />;
+}
+
+export function Duration100kInsertsChartMoreResults() {
+  const data: ChartData<"bar"> = {
+    labels: ["Time in seconds (lower is faster)"],
+    datasets: [
+      durations100k.payload,
+      durations100k.supabase,
+      durations100k.pocketbase_ts,
+      durations100k.pocketbase_dart_jit,
+      durations100k.trailbase_ts,
+      durations100k.trailbase_dart_jit,
+      durations100k.drizzle,
+    ] as ChartDataset<"bar">[],
   };
 
   return <BarChart data={data} />;
diff --git a/docs/src/content/docs/reference/benchmarks.mdx b/docs/src/content/docs/reference/benchmarks.mdx
@@ -4,7 +4,7 @@ description: Performance comparison with similar products.
 ---
 
 import {
-  Duration100kInsertsChart,
+  Duration100kInsertsChartMoreResults,
   PocketBaseAndTrailBaseReadLatencies,
   PocketBaseAndTrailBaseInsertLatencies,
   SupaBaseMemoryUsageChart,
@@ -53,13 +53,13 @@ Ultimately, nothing beats benchmarking your own workload and setup.
 _Total Time for 100k Insertions_
 
 <div class="flex justify-center">
-  <div class="h-[300px] w-[90%]">
-    <Duration100kInsertsChart client:only="solid-js" />
+  <div class="h-[400px] w-[90%]">
+    <Duration100kInsertsChartMoreResults client:only="solid-js" />
   </div>
 </div>
 
 The graph shows the overall time it takes to insert 100k messages into a mock
-"chat-room" table setup. Less time is better.
+*chat-room* table setup. Less time is better.
 
 Unsurprisingly, in-process SQLite is the quickest [^2].
 All other setups add additional table look-ups for access checking, IPC
@@ -69,11 +69,15 @@ and the cost a project would pay by adopting any of the systems over in-process
 SQLite.
 
 The data suggests that depending on your setup (client, data, hardware)
-TrailBase can insert 100k records 9 to 16 times faster than SupaBase[^4] and
-roughly 6 to 7 times faster than PocketBase [^1].
+TrailBase can insert 100k records almost 70 times faster than Payload[^4], 9 to
+16 times faster than SupaBase[^5], and roughly 6 to 7 times faster than
+PocketBase [^1].
+
+{/*
 The fact that our TS/node.js benchmark is slower than the Dart one, suggests a
 client-side bottleneck that could be overcome by tuning the setup or trying
 other JS runtimes with lower overhead HTTP clients.
+*/}
 
 Total time of inserting a large batch of data tells only part of the story,
 let's have a quick look at resource consumption to get an intuition for
@@ -90,7 +94,7 @@ _TrailBase & PocketBase Utilization_
 The graph shows the CPU utilization and memory consumption (RSS) of both
 PocketBase and TrailBase. They look fairly similar apart from TrailBase
 finishing earlier. They both load roughly 3 CPUs with PocketBase's CPU
-consumption being slightly more variable [^5].
+consumption being slightly more variable [^6].
 The little bump after the TrailBase run is likely due to SQLite check-pointing.
 
 Both only consume about 140MB of memory at full tilt, which makes them a great
@@ -218,8 +222,8 @@ The benchmarks are available on [GitHub](https://github.com/trailbaseio/trailbas
 
 [^2]:
     Our setup with drizzle and node.js is certainly not the fastest possible.
-    For example, we could drop down to using raw SQLite in C or another
-    low-level language.
+    For example, we could drop down to SQLite in C or another low-level
+    language with less FFI overhead.
     That said, drizzle is a great popular choice which mostly serves as a
     point-of-reference and sanity check.
 
@@ -228,14 +232,23 @@ The benchmarks are available on [GitHub](https://github.com/trailbaseio/trailbas
     For the benchmarks at hand we're using a loopback network device.
 
 [^4]:
+    We picked Payload as representative of popular Node.js CMS, which
+    [itself claims](https://payloadcms.com/blog/performance-benchmarks)
+    to be many times faster than popular options like Strapi or Directus.
+    We were using a v3 pre-release, as recommended, also using the
+    SQLite/drizzle database adapter marked as beta.
+    We manually turned on WAL mode and filed an issue with payload, otherwise
+    stock payload was ~210x times slower.
+
+[^5]:
     The SupaBase benchmark setup skips row-level access checks. Technically,
     this is in its favor from a performance standpoint, however looking at the
     overall load on its constituents with PG being only a sliver, it probably
     would not make much of an overall difference nor would PG17's vectorization,
     which has been released since the benchmarks were run. That said, these
     claims deserve re-validation.
 
-[^5]:
+[^6]:
     We're unsure as to what causes these 1-core swings.
     Runtime-effects, such as garbage collection, may have an effect, however we
     would have expected these to show on shorter time-scales.
