SQLModel gets corrupted when used with Celery

[superboum]

First Check

• I added a very descriptive title here.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the SQLModel documentation, with the integrated search.
• I already searched in Google "How to X in SQLModel" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to SQLModel but to Pydantic.
• I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from sqlmodel import SQLModel, Field
import time, signal, os

# === Setup ===

# https://docs.celeryq.dev/en/main/reference/celery.exceptions.html
class SoftTimeLimitExceeded(Exception):
    pass

# https://docs.python.org/3/library/signal.html#examples
def handler(signum, frame):
    raise SoftTimeLimitExceeded()

class Hero(SQLModel, table=True):
    id: int | None = Field(default=None, primary_key=True)
    name: str

    def __init__(self, **data):
        super().__init__(**data)
        time.sleep(2) # simulate a slow constructor

signal.signal(signal.SIGUSR1, handler)

# === Act ===
print(f"Reference Hero JSON: `Hero(name='foobar')` -> {Hero(name='foobar').model_dump_json()}")
print(f"Run in another terminal 'kill -SIGUSR1 {os.getpid()}' to corrupt SQLModel")
try:
    while True:
        Hero.model_validate({"name": "foobar"})
except SoftTimeLimitExceeded:
    print("Received SIGUSR1, interrupted. Hero is now corrupted.")


# === Observe ===
while True:
    print(f"Corrupted Hero JSON: `Hero(name='foobar')` -> {Hero(name='foobar').model_dump_json()}")

Description

Bug detection

This section is only for SEO, ie. help people in the same situation find this issue

We detected this issue in our codebase as we were starting to see that kind of PostgreSQL errors:

(psycopg.errors.NotNullViolation) null value in column "xxx" of relation "xxx" violates not-null constraint

Explanation

• model_validate is interrupted by an exception raised by a signal handler
  • It's how by default Celery handles soft task timeout
• exception is caught by the application, process continues its execution
• building an instance of SQLModel through its constructor now leads to an empty dict

Workaround

Note that one way to fix this error is to monkey patch partial_init:

import sqlmodel._compat as _sqlmodel_compat

@contextmanager
def _safe_partial_init() -> Generator[None, None, None]:
    token = _sqlmodel_compat.finish_init.set(False)
    try:
        yield
    finally:
        _sqlmodel_compat.finish_init.reset(token)


_sqlmodel_compat.partial_init = _safe_partial_init

However, it seems partial_init being interrupted by an exception is not the only codepath in SQLModel that can lead to corrupted models.

Operating System

Linux

Operating System Details

Reproduced in many environments (GKE cluster, local NixOS machine, macOS laptop, etc.), with different Python versions, on different SQLModel versions.

SQLModel Version

0.0.38

Python Version

3.13

Additional Context

Trace of a local execution of the reproducer:

Reference Hero JSON: `Hero(name='foobar')` -> {"name":"foobar","id":null}
Run in another terminal 'kill -SIGUSR1 27330' to corrupt SQLModel
Received SIGUSR1, interrupted. Hero is now corrupted.
Corrupted Hero JSON: `Hero(name='foobar')` -> {}
Corrupted Hero JSON: `Hero(name='foobar')` -> {}
Corrupted Hero JSON: `Hero(name='foobar')` -> {}

[MukundaKatta]

This is a real interaction between three things, not a Celery bug or an SQLModel bug in isolation:

1. Celery's SoftTimeLimitExceeded is raised from a Python signal handler (SIGUSR1 in your repro, SIGALRM in real Celery). That handler can fire between any two bytecode instructions on the currently executing thread.
2. Pydantic v2's validators mutate class-level state lazily, on first call. SQLModel.model_validate(...) may trigger the build of the class's validation schema if another thread hasn't done it yet — or enter internal caching code that's not atomic with respect to async signals.
3. A signal-raised exception in the middle of that build leaves the class's internal state half-initialized, and every subsequent model_validate or __init__ on that class sees the corrupted state. This is exactly what your repro demonstrates — the corruption survives the try/except.

The mechanism is general: any non-atomic initialization that can be interrupted by a Python-level signal handler can leave a class permanently broken. It hits SQLModel/pydantic especially hard because pydantic builds validators lazily and caches on the class.

What actually works

1. Don't raise from a signal handler in worker code.

The Celery docs themselves note that SoftTimeLimit "raises an exception in the task" — what they don't stress is that it does so from a signal context. The safer contract is:

# In your task, check periodically and exit cleanly:
from celery.exceptions import SoftTimeLimitExceeded

@app.task(soft_time_limit=30, time_limit=60)
def mytask():
    for item in items:
        if self.should_stop():  # cooperative check, not signal-driven
            raise SoftTimeLimitExceeded()
        do_work(item)

Cooperative cancellation between model-validation calls is interruption-safe. A signal-raised cancellation inside one is not.

2. If you must use soft limits, isolate them to subprocess workers.

# celeryconfig.py
worker_pool = 'prefork'          # default; each task runs in its own process
task_soft_time_limit = 30
task_time_limit = 60

With prefork, a corrupted SQLModel class lives in the child process for the duration of that task. Celery recycles the worker process on hard timeout (or via worker_max_tasks_per_child), so the corruption doesn't bleed across tasks. With threaded pools (solo/threads/gevent/eventlet) the class is shared across tasks in the same process — this is the deployment shape where your repro becomes production-fatal.

3. Pre-warm the validator cache before enabling signal handlers.

# Run once at worker start, before any soft-time-limit-bound tasks:
for model_cls in (Hero, OtherModel, ...):
    model_cls.model_validate({"name": "_"})  # or use model_rebuild()
    model_cls.model_rebuild()

Once the validator graph is fully built and cached, model_validate calls are largely signal-safe because they don't mutate class state. This is a workaround, not a fix — a sufficiently unlucky signal can still interrupt field-level validation — but it eliminates the "corrupt on first use" class of failures your repro hits.

4. Hard-limit only, or SIGKILL on timeout.

If the task is operating on external resources (DB rows, files) where abrupt termination is acceptable, skip soft limits entirely and let Celery kill the worker process on hard timeout. No signal-raised exceptions, no half-initialized objects, worker recycles cleanly.

Why the try/except doesn't save you

try:
    while True:
        Hero.model_validate({"name": "foobar"})
except SoftTimeLimitExceeded:
    print("...")

The except catches the exception — but the exception interrupted pydantic mid-way through modifying Hero.__pydantic_validator__ (or similar cached attribute). Python's exception machinery unwinds the stack, but the partially-mutated attribute stays put. The except clause is therefore catching the right exception on the right line; the state damage already happened before control returned to your frame.

Is this worth filing upstream?

Yes — but as a pydantic-core issue rather than SQLModel. The root is pydantic v2's class-level validator cache mutation not being signal-safe. A minimal repro with plain BaseModel (no SQLModel involved) is worth producing; you'll get faster engagement over there, and the fix has to land in pydantic-core if it lands at all. SQLModel inherits the problem transitively.

For now: cooperative cancellation in tasks, prefork in production, pre-warm validators at worker start. That combo has kept SQLModel + Celery stable for us across a lot of production traffic.