Cache Invalidation System

Overview

The OpenRegister Cache Invalidation System provides enterprise-grade cache consistency across all CRUD operations, ensuring that cached data remains synchronized with database changes. This system was designed to solve critical performance and consistency challenges while maintaining RBAC compliance and multi-tenant isolation.

Architecture Overview

graph TD
    subgraph "CRUD Operations"
        A[Individual Object CRUD]
        B[Bulk Object Operations] 
        C[Schema CRUD Operations]
    end
    
    subgraph "Cache Invalidation Service"
        D[CacheInvalidationService]
        D --> E[Object Cache Invalidation]
        D --> F[Schema Cache Invalidation]
        D --> G[Bulk Cache Coordination]
    end
    
    subgraph "Cache Layers"
        H[Search/Collection Cache]
        I[Facet Response Cache]
        J[Entity Cache]
        K[Schema Config Cache]
        L[Schema Facet Cache]
    end
    
    A --> D
    B --> D  
    C --> D
    E --> H
    E --> I
    E --> J
    F --> K
    F --> L
    F --> H
    F --> I
    G --> H
    G --> I
    G --> J
    
    style D fill:#e1f5fe
    style H fill:#c8e6c9
    style I fill:#c8e6c9
    style J fill:#c8e6c9
    style K fill:#fff3e0
    style L fill:#fff3e0

Components

CacheInvalidationService

Location: lib/Service/CacheInvalidationService.php

Central service that coordinates cache invalidation across all cache layers. Provides methods for:

• Individual object invalidation: invalidateObjectRelatedCaches()
• Schema-related invalidation: invalidateSchemaRelatedCaches()
• Bulk operation coordination: Used by ObjectService bulk methods
• RBAC-compliant cache management: User and organization-aware cache keys

Integration Points

Object CRUD Operations

SaveObject Handler (lib/Service/ObjectHandlers/SaveObject.php):

• Automatically invalidates caches after object create/update operations
• Includes register, schema, and object-specific cache clearing

DeleteObject Handler (lib/Service/ObjectHandlers/DeleteObject.php):

• Automatically invalidates caches after object deletion
• Ensures deleted objects disappear from collection queries immediately

Schema CRUD Operations

SchemasController (lib/Controller/SchemasController.php):

• Invalidates schema-related caches on schema create/update/delete
• Coordinates with dependent cache layers (collections, facets)

Bulk Operations

ObjectService (lib/Service/ObjectService.php):

• saveObjects(): Invalidates caches after bulk create/update operations
• deleteObjects(): Clears collection caches after mass deletion
• publishObjects(): Updates facet caches after bulk publication changes
• depublishObjects(): Synchronizes visibility caches after bulk depublication

Cache Types and Invalidation Strategies

Search/Collection Cache

Purpose: Caches complete API responses for collection endpoints Key Pattern: obj_search_{hash} with user and organization context Invalidation: Cleared on any object CRUD operation affecting the collection

Cache Keys:

• Include user ID for RBAC isolation
• Include organization ID for multi-tenancy
• Include query hash for request-specific caching

graph LR
    A[User Request] --> B{Cache Key Generation}
    B --> C[User ID + Org ID + Query Hash]
    C --> D[obj_search_abc123...]
    D --> E{Cache Hit?}
    E -->|Yes| F[Return Cached Response]
    E -->|No| G[Execute Query]
    G --> H[Cache Response]
    H --> I[Return Fresh Response]
    
    style E fill:#fff3e0
    style F fill:#c8e6c9
    style G fill:#ffcdd2

Entity Cache

Purpose: Caches frequently accessed schemas and registers Key Pattern: entity_{type}_{ids} Invalidation: Cleared when specific entities are modified

Facet Response Cache

Purpose: Caches facet calculation results for complex aggregations Key Pattern: Includes query filters and user context Invalidation: Cleared when underlying data changes affect facet counts

Schema-Specific Caches

Purpose:

• Schema Config Cache: Schema configuration data
• Schema Facet Cache: Pre-computed facetable field definitions

Invalidation: Cleared when schemas are created, updated, or deleted

RBAC Compliance

User-Based Cache Keys

All cache keys include user and organization context to ensure data isolation:

$keyData = [
    'query' => $query,
    'user' => $userId,           // RBAC isolation
    'org' => $orgId,             // Multi-tenancy isolation
    'version' => '1.0'           // Cache versioning
];

Anonymous User Handling

• Anonymous users get separate cache namespace
• Authenticated users have individual cache isolation
• Cross-organization access prevented through cache key separation

Performance Benefits

Before Implementation

• Stale data: New objects not visible until cache expiry (5+ minutes)
• Inconsistent facets: Facet counts don't reflect recent changes
• Cross-user data leakage: RBAC violations through shared caches
• Bulk operation invisibility: Mass imports invisible until manual cache clear

After Implementation

Metric	Before	After	Improvement
New object visibility	5+ minutes	Immediate	100% instant
Facet accuracy	Stale counts	Real-time	Always accurate
RBAC compliance	❌ Shared caches	✅ Isolated	100% secure
Bulk operation consistency	❌ Invisible	✅ Immediate	100% consistent

Performance Monitoring

Detailed logging provides visibility into cache invalidation performance:

$this->logger->info('Object cache invalidation completed', [
    'register' => $registerId,
    'schema' => $schemaId, 
    'operation' => $operation,
    'executionTime' => $executionTime . 'ms',
    'invalidatedCaches' => $invalidatedCaches
]);

Usage Examples

Individual Object Operations

Cache invalidation happens automatically:

// Create object - caches automatically invalidated
$object = $objectService->createFromArray($data, [], $register, $schema);

// Update object - caches automatically invalidated  
$object = $objectService->updateFromArray($uuid, $data, true);

// Delete object - caches automatically invalidated
$result = $objectService->deleteObject($uuid);

Bulk Operations

Cache invalidation coordinates across bulk operations:

// Bulk save - collection caches invalidated after completion
$result = $objectService->saveObjects($objectsArray, $register, $schema);

// Bulk delete - global cache invalidation
$result = $objectService->deleteObjects($uuidArray);

// Bulk publish - facet caches updated
$result = $objectService->publishObjects($uuidArray, new \DateTime());

Schema Operations

Schema changes trigger comprehensive cache invalidation:

// Schema update - affects facets, collections, and entity caches
$schema = $schemaMapper->updateFromArray($id, $schemaData);
// Cache invalidation happens automatically in SchemasController

// Schema delete - all dependent caches cleared
$schemaMapper->delete($schema);
// Comprehensive cache cleanup happens automatically

Cache Invalidation Flow

Individual Object Flow

sequenceDiagram
    participant Client
    participant Controller  
    participant SaveObject
    participant CacheService
    participant Cache
    
    Client->>Controller: POST /api/objects
    Controller->>SaveObject: Save object
    SaveObject->>SaveObject: Validate & persist
    SaveObject->>CacheService: invalidateObjectRelatedCaches()
    CacheService->>Cache: Clear search cache
    CacheService->>Cache: Clear facet cache  
    CacheService->>Cache: Clear entity cache
    SaveObject-->>Controller: Saved object
    Controller-->>Client: 201 Created
    
    Note over Cache: Next GET request sees new data immediately

Schema Change Flow

sequenceDiagram
    participant Admin
    participant SchemasController
    participant CacheService
    participant MultipleServices
    participant Cache
    
    Admin->>SchemasController: PUT /api/schemas/{id}
    SchemasController->>SchemasController: Update schema
    SchemasController->>CacheService: invalidateSchemaRelatedCaches()
    CacheService->>Cache: Clear entity cache
    CacheService->>MultipleServices: Clear schema config cache
    CacheService->>MultipleServices: Clear schema facet cache
    CacheService->>Cache: Clear collection cache
    CacheService->>Cache: Clear facet response cache
    SchemasController-->>Admin: 200 Updated
    
    Note over Cache: All schema-dependent caches updated

Bulk Operation Flow

sequenceDiagram
    participant Client
    participant ObjectService
    participant SaveObjectsHandler
    participant CacheService
    participant Cache
    
    Client->>ObjectService: saveObjects(100 objects)
    ObjectService->>SaveObjectsHandler: Bulk processing
    SaveObjectsHandler->>SaveObjectsHandler: Create/update objects
    SaveObjectsHandler-->>ObjectService: Bulk result
    ObjectService->>CacheService: invalidateObjectRelatedCaches()
    CacheService->>Cache: Clear collection caches
    CacheService->>Cache: Clear facet caches
    ObjectService-->>Client: Bulk result
    
    Note over Cache: All 100 objects immediately visible in collections

Cache Key Patterns

Collection Cache Keys

// Pattern: obj_search_{hash}
// Hash includes: query + user + organization + version
'obj_search_abc123def456...' 

Entity Cache Keys

// Specific entities
'entity_schema_42'
'entity_register_15'

// Multiple entities  
'entity_schema_md5(42,43,44)'

// All entities
'entity_schema_all'
'entity_register_all'

Facet Cache Keys

// Pattern includes user and organization context
'facet_{userId}_{orgId}_{queryHash}'

Error Handling

Non-Blocking Cache Operations

Cache invalidation failures never block the primary operation:

try {
    // Perform cache invalidation
    $this->cacheInvalidationService->invalidateObjectRelatedCaches(/*...*/);
} catch (\Exception $e) {
    // Log error but continue with primary operation
    $this->logger->warning('Cache invalidation failed', ['error' => $e->getMessage()]);
}

Graceful Degradation

• Cache unavailable: System continues without caching
• Partial invalidation failures: Logs warnings but continues
• Invalid cache data: Automatically falls back to database

Configuration

Cache TTL Settings

// Response cache (collection queries)
private const CACHE_TTL = 300; // 5 minutes

// Entity cache (schemas/registers) 
private const ENTITY_CACHE_TTL = 900; // 15 minutes

Environment Detection

The system automatically detects slow environments and adjusts caching strategies:

private function isSlowEnvironment(): bool
{
    // Checks for AC environment variables
    // Measures database response time
    // Adjusts caching aggressiveness accordingly
}

Monitoring and Debugging

Debug Logging

Enable detailed cache logging by checking Docker logs:

# View real-time cache invalidation logs
docker logs -f master-nextcloud-1 | grep -E "(cache.*invalidation|Cache.*invalidation)"

# View recent cache operations
docker logs master-nextcloud-1 --since 10m | grep 'invalidation'

Performance Metrics

The system logs detailed performance metrics:

{
  "message": "Object cache invalidation completed",
  "register": 19,
  "schema": 116,
  "operation": "create",
  "executionTime": "2.3ms",
  "invalidatedCaches": {
    "search": 15,
    "facets": 8,
    "entity": 2
  }
}

Best Practices

For Developers

1. Always use the service methods: Don't bypass cache invalidation by calling mappers directly
2. Monitor performance: Use debug logging to track cache invalidation overhead
3. Test cache scenarios: Verify that changes are immediately visible in collection calls

For Operations

1. Cache warming: Popular queries are automatically cached on first access
2. Memory monitoring: Entity caches use distributed storage (Redis/Memcached in production)
3. Cache clearing: Use clearAllCaches() method for maintenance operations

Future Enhancements

Planned Improvements

• Tagged caching: Use cache tags for more selective invalidation
• Reverse mapping: Track cache dependencies for precise invalidation
• Metrics collection: Expose cache hit/miss rates through API
• Cache warming strategies: Proactive cache population for popular queries

Extension Points

The CacheInvalidationService can be extended for additional cache types:

public function invalidateCustomCache(string $cacheType, $identifier): void
{
    // Add custom cache invalidation logic
}

External App Optimization

Cache Isolation for External Apps

Problem Solved: External apps calling ObjectService directly were experiencing cache thrashing due to shared 'anonymous' cache namespace.

Solution Implemented: Enhanced cache key generation with app-specific isolation:

graph LR
    A[External App Call] --> B{User Session?}
    B -->|Yes| C[User-based Cache Key]
    B -->|No| D{App Context Set?}
    D -->|Yes| E[external_app_myapp]
    D -->|No| F{Auto-Detection?}
    F -->|Yes| G[external_app_detected]
    F -->|No| H[anonymous_fingerprint]
    
    E --> I[Isolated Cache Namespace]
    G --> I
    H --> J[Pattern-based Cache]
    
    style I fill:#c8e6c9
    style J fill:#fff3e0

Usage for External Apps

// **RECOMMENDED**: Set explicit app context
$objectService = \OC::$server->get(\OCA\OpenRegister\Service\ObjectService::class);
$objectService->setExternalAppContext('myapp');
$results = $objectService->searchObjectsPaginated($query);

Performance Impact: External apps can see 10-30x performance improvements with proper app context.

For detailed implementation guide, see: External App Cache Optimization

---

Related Documentation:

• External App Cache Optimization
• Schema-Based Facets
• Bulk Operations Implementation
• Object Handlers