Refactoring SMRT Schema Management: A Comprehensive Guide

Oct 20, 2025 by ADMIN 58 views

Hey guys! Today, we're diving deep into a crucial refactor discussion focusing on SMRT schema management. This is super important because, as it stands, our current system has some architectural quirks that can cause headaches down the road. We're talking about issues like table names breaking in production when the code is minified, schema creation logic scattered across multiple places, and the absence of a unified ALTER TABLE API. All these point to a deeper problem: a lack of a unified, explicit schema management system within SMRT. So, let's roll up our sleeves and figure out how to fix this!

Problem Statement: Identifying the Core Issues

To truly tackle this refactor, let's first get crystal clear on the problems we're trying to solve. Our current SMRT schema management has some fundamental architectural issues. These issues expose the fact that SMRT currently lacks a unified, explicit schema management system.

Implicit Schema Derivation: This is a big one, guys! Relying on class names to derive table names might seem convenient, but it's a recipe for disaster when we start minifying our code for production. Imagine your class Image turning into something like a after minification. Suddenly, your table names are all messed up! This implicit derivation becomes a major bottleneck, especially when deploying to production environments where code optimization is crucial. The fragility introduced by this approach means that a simple change in the build process could lead to unexpected and hard-to-debug errors. It's like building a house on sand – looks good initially, but not so stable in the long run. We need a robust solution that doesn’t rely on potentially changing class names.
Fragmented Schema Generation: This one's about code duplication and maintainability. Currently, the logic for creating schemas is scattered across three different code paths. Think about the headache of updating schema logic – you'd have to hunt down every instance and make sure they're all consistent. This fragmentation not only increases the risk of inconsistencies but also makes the codebase harder to understand and maintain. It's like having three different instruction manuals for the same device – confusing and inefficient. A unified approach would streamline the process and make our lives much easier.
No ALTER TABLE API: Our lack of a centralized ALTER TABLE API means schema alteration logic is duplicated across different database adapters. This is another maintenance nightmare waiting to happen! When we need to modify a table, we shouldn't have to repeat the logic for each adapter. A unified API would provide a consistent interface for altering tables, simplifying the process and reducing the chances of errors. Think of it as having to reinvent the wheel every time you need to change a tire – a lot of unnecessary work! A dedicated API would abstract away the database-specific details, making our code more portable and maintainable.

Proposed Solution: A Unified Approach to Schema Management

Okay, so we've identified the problems. Now, let's talk solutions! Our main goal is to create a unified, explicit schema management system for SMRT. This involves several key steps, each addressing one or more of the issues we just discussed.

1. Auto-Capture Table Names in Decorator (Addresses #1)

To tackle the minification issue, we'll introduce a clever trick using decorators. Decorators are a powerful feature in TypeScript that allows us to add metadata or modify class behavior at design time. We'll use a decorator to automatically capture the class name before minification happens. This ensures that we have the correct table name, even after the code is optimized for production. We'll also allow for explicit overrides, giving developers the flexibility to specify custom table names when needed.

// Automatic capture (decorator runs before minification)
@smrt()
class Image extends SmrtObject { }

// Explicit override
@smrt({ tableName: 'custom_images' })
class Image extends SmrtObject { }

Implementation Details:

Here's a snippet of how this decorator might look:

// packages/core/smrt/src/decorator.ts
function smrt(config?: SmrtConfig) {
  return function(target: any) {
    // target.name contains ORIGINAL class name (before minification)
    const tableName = config?.tableName || classnameToTablename(target.name);
    
    // Store permanently (survives minification)
    Object.defineProperty(target, 'SMRT_TABLE_NAME', {
      value: tableName,
      writable: false,
      enumerable: false,
      configurable: false
    });
    
    ObjectRegistry.register(target, { ...config, tableName });
    return target;
  }
}

Why this works:

Decorators execute during class definition, which happens before minification. This is crucial because it allows us to access the original class name.
The class name is still "Image" at the time the decorator runs, so we can capture it accurately.
Minification happens later in the build process, after the decorators have already executed.
We store the captured table name in a static property on the class, ensuring it survives minification. This property is non-writable, non-enumerable, and non-configurable, protecting it from accidental modification.

This approach provides a robust and elegant solution to the minification problem, ensuring our table names remain consistent in production.

2. Unified Schema Generator (Addresses #2)

Now, let's tackle the fragmented schema generation logic. Our solution here is to consolidate all schema generation into a single, dedicated class: the SchemaGenerator. This class will become the single source of truth for all schema-related operations, ensuring consistency and reducing code duplication.

// packages/core/smrt/src/schema/generator.ts
export class SchemaGenerator {
  /**
   * Generate CREATE TABLE SQL from schema definition
   * Single source of truth for all schema generation
   */
  generateCreateTable(options: GenerateTableOptions): string {
    // Single implementation of:
    // - Column type mapping
    // - DEFAULT CAST handling (DuckDB compatibility)
    // - Constraint generation
    // - Index creation
    // - Foreign key constraints
  }
  
  /**
   * Generate column definition with proper constraints
   */
  generateColumn(column: ColumnDefinition): string {
    // Unified logic for:
    // - Type conversion
    // - DEFAULT CAST wrapper
    // - NOT NULL, UNIQUE, PRIMARY KEY
    // - CHECK constraints
  }
  
  /**
   * Wrap DEFAULT values in CAST for DuckDB compatibility
   */
  private wrapDefaultInCast(
    columnType: string,
    defaultValue: any
  ): string {
    // Single implementation of CAST logic
    // Used by all schema generation paths
  }
}

This SchemaGenerator class will encapsulate all the logic for generating CREATE TABLE SQL statements, handling column type mappings, managing default value casting (especially important for DuckDB compatibility), generating constraints, creating indexes, and handling foreign key constraints. By centralizing this logic, we eliminate duplication and make it much easier to maintain and update our schema generation process.

Migration Path:

To implement this, we'll follow a structured migration path:

Extract the existing logic from the Field system (packages/core/smrt/src/fields/index.ts).
Extract the logic from utils.ts (specifically the generateSchema() function).
Extract the logic from the runtime manager (the createTable() function).
Once all the logic is extracted, we'll refactor the original code paths to delegate to the SchemaGenerator. This ensures a smooth transition and avoids breaking existing functionality.

This approach ensures a clean and maintainable codebase, with a single, authoritative source for schema generation.

3. Automatic Schema Evolution (Simple)

Finally, let's talk about schema evolution. How do we handle changes to our database schema over time? Our proposal is to implement a simple form of automatic schema evolution, where the runtime manager automatically adds new columns and indexes using a standardized ALTER TABLE API. This will make it much easier to deploy updates to our applications without manual schema migrations.

// packages/core/smrt/src/schema/runtime-manager.ts
private async updateSchemaIfNeeded(
  db: DatabaseInterface,
  schema: SchemaDefinition
): Promise<void> {
  if (!db.alterTable) {
    // Graceful fallback for adapters without alterTable support
    console.log('[schema] ALTER TABLE not supported, skipping schema evolution');
    return;
  }
  
  const current = await db.getTableSchema?.(schema.tableName);
  if (!current) return;
  
  // Auto-add new columns (safe operation)
  for (const [columnName, columnDef] of Object.entries(schema.columns)) {
    if (!current.columns[columnName]) {
      await db.alterTable.addColumn(schema.tableName, {
        name: columnName,
        ...columnDef
      });
    }
  }
  
  // Auto-add new indexes (safe operation)
  for (const index of schema.indexes) {
    if (!current.indexes.find(i => i.name === index.name)) {
      await db.alterTable.addIndex(schema.tableName, index);
    }
  }
}

This updateSchemaIfNeeded function will compare the current database schema with the desired schema and automatically add any missing columns or indexes. We'll focus on safe operations that don't risk data loss or corruption.

Safe automatic operations:

✅ Adding new columns with default values is generally safe, as it doesn't affect existing data.
✅ Adding new indexes can improve query performance without altering the data itself.

Unsafe operations require manual SQL:

❌ Dropping columns carries a significant risk of data loss and should be handled manually.
❌ Renaming columns can be ambiguous and lead to unexpected behavior.
❌ Changing column types can corrupt data if not done carefully.

For these unsafe operations, we'll require developers to use manual SQL migrations, ensuring they have full control over the process and can handle any potential issues.

Implementation Plan: Breaking it Down into Phases

To make this refactor manageable, we'll break it down into four distinct phases, each with its own set of tasks and goals.

Phase 1: Auto-Capture Table Names

This phase focuses on addressing the minification issue by implementing the decorator-based solution for capturing table names.

[ ] Update the @smrt() decorator to capture target.name before minification.
[ ] Store the captured name in a SMRT_TABLE_NAME static property on the class.
[ ] Update the tableNameFromClass() function to read from this stored property.
[ ] Add tests specifically for minified builds to ensure the solution works correctly.
[ ] Update the documentation to reflect the new decorator-based approach.

Files affected:

packages/core/smrt/src/decorator.ts
packages/core/smrt/src/utils.ts
packages/core/smrt/src/registry.ts

Phase 2: Unified Schema Generator

This phase is all about consolidating the schema generation logic into the SchemaGenerator class.

[ ] Create packages/core/smrt/src/schema/generator.ts to house the SchemaGenerator class.
[ ] Implement the SchemaGenerator class with methods for generating CREATE TABLE statements, column definitions, and handling default value casting.
[ ] Extract the existing DEFAULT CAST logic from all three code paths.
[ ] Extract the column generation logic.
[ ] Extract the table generation logic.
[ ] Migrate the Field system to use the SchemaGenerator.
[ ] Migrate utils.ts to use the SchemaGenerator.
[ ] Migrate the runtime manager to use the SchemaGenerator.
[ ] Remove any duplicate code.
[ ] Add comprehensive tests, aiming for 100% coverage, to ensure the SchemaGenerator works correctly.

Files affected:

packages/core/smrt/src/fields/index.ts
packages/core/smrt/src/utils.ts
packages/core/smrt/src/schema/runtime-manager.ts

Phase 3: SDK ALTER TABLE API (SDK repo - separate issue)

Note: This phase happens in the SDK repository and is tracked in a separate issue.

This phase focuses on standardizing the ALTER TABLE API across different database adapters in the SDK.

[ ] Define an alterTable interface in the DatabaseInterface.
[ ] Implement this interface in the SQLite adapter.
[ ] Implement it in the Postgres adapter.
[ ] Implement it in the DuckDB adapter.
[ ] Add a getTableSchema() introspection method to retrieve the current schema of a table.
[ ] Add tests for each adapter to ensure the ALTER TABLE API works correctly.

Phase 4: Automatic Schema Evolution

This phase builds on Phase 3 and implements the automatic schema evolution logic.

[ ] Update runtime-manager.updateSchemaIfNeeded() to use the standardized alterTable API.
[ ] Implement the logic for automatically adding new columns.
[ ] Implement the logic for automatically adding new indexes.
[ ] Add a configuration option, autoEvolution: boolean (default: true), to allow developers to disable automatic schema evolution if needed.
[ ] Add debug logging for schema changes to help with troubleshooting.
[ ] Update the documentation to explain how automatic schema evolution works and how to handle manual schema changes.

Files affected:

packages/core/smrt/src/schema/runtime-manager.ts

Cross-Repository Coordination: SMRT Repo and SDK Repo

This refactor involves changes in both the SMRT repository (where this issue is tracked) and the SDK repository. To ensure smooth coordination, we'll divide the work as follows:

SMRT Repo (this issue):

Phases 1, 2, and 4 will be implemented in the SMRT repository.

SDK Repo (separate issue):

Phase 3, which focuses on standardizing the ALTER TABLE methods in database adapters, will be implemented in the SDK repository.

Dependencies:

Phase 4 (Automatic Schema Evolution) depends on Phase 3 (SDK ALTER TABLE API) being completed first. This is because we need the standardized ALTER TABLE API in the SDK before we can implement automatic schema changes.
Phases 1 and 2 are independent and can proceed immediately.

Benefits: A Clear Path to a Better System

This refactor will bring a ton of benefits to our SMRT schema management system. Let's break them down:

Production-Safe: Table names will be captured before minification, eliminating the risk of broken table names in production environments. This is a huge win for stability and reliability.
Zero Configuration: The system will work automatically with sensible defaults, making it easy to get started and use. This reduces the learning curve and makes SMRT more accessible to developers.
Override Available: Developers will still be able to specify custom table names when needed, providing flexibility and control. This allows for customization when the default behavior isn't sufficient.
Maintainable: A single source of truth for schema generation will make the codebase much easier to maintain and update. This reduces code duplication and the risk of inconsistencies.
Automatic Evolution: New columns and indexes will be added automatically, simplifying schema updates and deployments. This streamlines the development process and reduces the need for manual migrations.
Database-Agnostic: A standardized API across all adapters will make our code more portable and easier to test. This allows us to switch databases more easily and reduces vendor lock-in.
No Breaking Changes: Existing code will continue to work without modification, ensuring a smooth transition. This is crucial for minimizing disruption and maintaining backward compatibility.
Consistent SQL: All code paths will generate identical SQL, ensuring consistency and predictability. This simplifies debugging and reduces the risk of subtle differences between different parts of the system.

Example: Seeing the Improvements in Action

Let's look at a few examples to illustrate how this refactor will improve our code.

Before (still works):

class Product extends SmrtObject {
  name = text();
  price = decimal();
}
// Auto-detected as "products" table

In the current system, the table name is automatically derived from the class name. This works fine, but it's vulnerable to minification issues.

After (enhanced):

@smrt() // Now captures "Product" → "products" before minification
class Product extends SmrtObject {
  name = text();
  price = decimal();
}

With the decorator in place, we capture the class name before minification, ensuring the table name remains correct in production.

With explicit override:

@smrt({ tableName: 'custom_products' })
class Product extends SmrtObject {
  name = text();
  price = decimal();
}

We can still explicitly specify a table name if needed, providing flexibility and control.

Schema evolution (automatic):

@smrt()
class Product extends SmrtObject {
  name = text();
  price = decimal();
  category = text({ default: '' }); // ← New field added
  // Runtime automatically: ALTER TABLE products ADD COLUMN category TEXT DEFAULT ''
}

When we add a new field, the runtime will automatically add the corresponding column to the database, simplifying schema updates.

Related Issues: Connecting the Dots

This refactor is closely related to several existing issues:

#1: SMRT table name generation breaks with minified class names - This issue directly motivated the need for the decorator-based solution for capturing table names.
#2: refactor(smrt): consolidate fragmented schema generation code paths - This issue highlighted the code duplication and maintainability problems with our current schema generation logic.
SDK Issue (to be created): "Standardize ALTER TABLE Methods in Database Adapters" - This issue will track the work in the SDK repository to standardize the ALTER TABLE API.

Success Criteria: Measuring Our Progress

How will we know if this refactor is successful? We'll use the following criteria:

[ ] The decorator captures table names before minification.
[ ] All @smrt() decorated classes work correctly in production builds.
[ ] There is zero code duplication for schema generation.
[ ] A single SchemaGenerator class is used by all code paths.
[ ] Automatic column/index addition works across all adapters.
[ ] All existing tests pass.
[ ] New tests for minified builds pass.
[ ] The documentation is updated with the new patterns.
[ ] There are no breaking changes to existing code.

Priority: A High-Priority Task

This refactor is a high-priority task because it addresses critical issues that are currently blocking the production deployment of minified SMRT applications. The code duplication also creates a significant maintenance burden. By addressing these issues, we'll improve the stability, maintainability, and scalability of our SMRT system.

So, there you have it, folks! A comprehensive plan for refactoring SMRT schema management. This is a big task, but by breaking it down into manageable phases and coordinating our efforts across repositories, we can create a much better system for managing our database schemas. Let's get to work! 😉