Keeping your system tables clean and performant requires routine maintenance. This guide covers the automated background workflows that manage data retention, specifically focusing on daily cleanups for Order Logs, Title Code Comparisons, and Window Management items.
By automating these processes, your system automatically purges outdated records without requiring manual intervention.
Maintenance Overview
The maintenance system relies on a two-step lifecycle for most records:
Stamping an expiration date when a new record is created.
Running scheduled sweeps to find and delete records that have reached their expiration date.
flowchart TD
A["New Record Created"] --> B["Automation calculates retention date"]
B --> C["Record updated with 'Delete on' date"]
C --> D{"Scheduled Nightly Trigger"}
D -->|"Matches 'Delete on' date"| E["Queue for deletion"]
E --> F["Loop and delete (Max 500)"]Retention Policies at a Glance
| Target Table | Retention Period | Cleanup Schedule |
|---|---|---|
| Order Logger | 14 days | Daily at 1:15 AM (EST) |
| Title Code Comparison | 14 days | Daily at 1:15 AM (EST) |
| Window Management | 1 day | Every 3 days at 2:30 AM (EST) |
All scheduled times are configured for the America/New_York (EST/EDT) timezone.
How Expiration Dates are Set
To ensure records are deleted at the right time, automations listen for new entries and immediately stamp them with a future deletion date.
Scheduled Cleanup Workflows
The actual deletion of records is handled by scheduled tasks that run during off-peak hours.
Daily 1:15 AM Cleanups
Every day (including weekends) at 1:15 AM, two separate automations run to clean up the Order Logger and Title Code Comparison tables.
Here is exactly how the daily cleanup process works:
- 1
Trigger the schedule
The automation wakes up at 1:15 AM EST.
- 2
Format the current date
The system formats today's date into a standard
MM/DD/YYYYformat. - 3
Search for expired records
The automation searches the target table for any records where the
Delete onfield exactly matches today's formatted date. - 4
Filter and verify
If no records are found, the automation stops. If expired records exist, it continues to the next step.
- 5
Batch delete
The system loops through the found records and permanently deletes them from the table.
Batch Limits: The deletion loop is capped at 500 records per run to prevent system timeouts. If you consistently generate more than 500 disposable records per day in a single table, you may need to increase the frequency of these automations.
Window Management Cleanup
The Window Management cleanup operates on a slightly different schedule and logic. Instead of relying on a pre-calculated Delete on date, it evaluates records dynamically based on when they were added.
Schedule: Runs every 3 days at 2:30 AM.
Logic: The automation takes the current date, subtracts 1 day, and searches the
[Table] HDPHD sync - update Window managementtable.Action: It finds any records where the
First Addeddate is less than or equal to that calculated date and deletes up to 500 of them.
Troubleshooting
Why are old records still showing up in my tables?
There are a few reasons this might happen:
Batch Limits: If more than 500 records expired on the same day, the automation will only delete the first 500. The rest will remain until manually cleared or until the automation is adjusted to run more frequently.
Missing 'Delete on' Dates: If the automation that stamps the 14-day expiration date was paused when the record was created, the record won't be picked up by the daily cleanup.
Can I recover deleted records?
No. Once the scheduled maintenance automation deletes a record from the table, it is permanently removed to free up system resources.