Skip to content

feat: declared background workers + frankenphp_get_worker_handle() #2617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nicolas-grekas wants to merge 37 commits into
base: main
Choose a base branch
from
Open

feat: declared background workers + frankenphp_get_worker_handle() #2617

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
936f791
fix: a thread that ended on its own can still be asked to shut down
nicolas-grekas Sep 21, 2026
9cff092
feat: declared background workers + frankenphp_get_worker_handle()
nicolas-grekas Sep 6, 2026
bae8218
fix: a blocking receive on the handle marks a background worker ready
nicolas-grekas Sep 11, 2026
0d81c76
fix: never inherit the reserved worker variables
nicolas-grekas Sep 11, 2026
584a512
fix: cap the restart backoff, and EOF a handle a forked child still h…
nicolas-grekas Sep 11, 2026
bd359b7
fix: resolve max_threads auto on the HTTP budget
nicolas-grekas Sep 11, 2026
bd72155
chore: simplifications and wording from the review
nicolas-grekas Sep 11, 2026
7613458
test: cover the parked run and the handle cache
nicolas-grekas Sep 11, 2026
19c4510
docs: flag frankenphp_get_worker_handle() experimental in the stub
nicolas-grekas Sep 13, 2026
d51dc26
feat: frankenphp_worker_tick(), the explicit ready point of backgroun…
nicolas-grekas Sep 13, 2026
9bac067
feat: FRANKENPHP_WORKER_BACKGROUND holds the name, HTTP workers untou…
nicolas-grekas Sep 14, 2026
c798593
feat: wake a background worker once at start
nicolas-grekas Sep 14, 2026
a90f83b
docs: event loop example for background workers
nicolas-grekas Sep 14, 2026
372ad78
refactor: one lifecycle abstraction instead of two
nicolas-grekas Sep 14, 2026
873c353
fix: never block or race on the startup failure channel
nicolas-grekas Sep 14, 2026
f326169
feat: max_execution_time bounds the bootstrap of a background worker
nicolas-grekas Sep 14, 2026
402d164
docs: stop the whole loop on drain in the Revolt example
nicolas-grekas Sep 14, 2026
a6883d6
test: only assert the bounded bootstrap where the timers are known to…
nicolas-grekas Sep 14, 2026
9f1770a
feat: num defaults to one thread for background workers
nicolas-grekas Sep 14, 2026
eea5f34
chore: review polish
nicolas-grekas Sep 14, 2026
b7d7ab3
test: the tick leaves the handle quiet
nicolas-grekas Sep 14, 2026
647f3e0
fix: keep generating unique names for workers that share a script
nicolas-grekas Sep 17, 2026
6385231
feat: report the server of a worker as its own metric label
nicolas-grekas Sep 17, 2026
0685854
fix: address the review of the background worker lifecycle
alexandre-daubois Sep 17, 2026
d8a6e31
fix: keep the declared path as the default worker name
nicolas-grekas Sep 18, 2026
31234a0
fix: pace only the background runs that end too fast
nicolas-grekas Sep 18, 2026
06ee910
refactor: declare the labels of a worker instead of widening the metrics
nicolas-grekas Sep 18, 2026
d7729c7
feat: report the server of a worker through an optional ServerMetrics…
alexandre-daubois Sep 18, 2026
12fa36a
feat: the handle of a background worker becomes an object
nicolas-grekas Sep 18, 2026
3c5c48c
refactor: keep the stream of a handle on the handle, not on the thread
nicolas-grekas Sep 20, 2026
e611798
docs: lead with the poll loop, the stream is for the libraries that t…
nicolas-grekas Sep 20, 2026
386a888
fix: a handle built behind the constructor must not reach the Go side
nicolas-grekas Sep 21, 2026
fbb0b02
fix: pace a crash loop whatever the run lasted
nicolas-grekas Sep 21, 2026
0d81bd8
chore: check the server the same way in SendRequest and SendMessage
nicolas-grekas Sep 21, 2026
2da5ffa
fix: form the Windows socket pair on loopback, and only with ourselves
nicolas-grekas Sep 21, 2026
3c66d02
feat: bound the wait for a background worker to become ready
nicolas-grekas Sep 21, 2026
6c9a8c9
chore: fewer comments, the ones left say why
nicolas-grekas Sep 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
783 changes: 783 additions & 0 deletions bgworker_test.go
View file
Open in desktop

Large diffs are not rendered by default.

62 changes: 15 additions & 47 deletions caddy/app.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import (
"github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/dunglas/frankenphp"
"github.com/dunglas/frankenphp/internal/fastabs"
)

var (
Expand Down Expand Up @@ -60,15 +59,14 @@ type FrankenPHPApp struct {
// EXPERIMENTAL: MaxRequests sets the maximum number of requests a PHP thread handles before restarting (0 = unlimited)
MaxRequests int `json:"max_requests,omitempty"`

opts []frankenphp.Option
metrics frankenphp.Metrics
ctx context.Context
logger *slog.Logger
modules []*FrankenPHPModule
usedWorkerNames map[string]bool
httpApp *caddyhttp.App
hasStarted atomic.Bool
started chan any
opts []frankenphp.Option
metrics frankenphp.Metrics
ctx context.Context
logger *slog.Logger
modules []*FrankenPHPModule
httpApp *caddyhttp.App
hasStarted atomic.Bool
started chan any
}

var errIni = errors.New(`"php_ini" must be in the format: php_ini "<key>" "<value>"`)
Expand Down Expand Up @@ -133,7 +131,6 @@ func (f *FrankenPHPApp) Start() error {
// register global workers
for _, w := range f.Workers {
w.FileName = repl.ReplaceKnown(w.FileName, "")
Comment thread
alexandre-daubois marked this conversation as resolved.
w.Name = f.createUniqueWorkerName(w, "")
opts, err := w.toWorkerOptions()
if err != nil {
return err
Expand Down Expand Up @@ -224,7 +221,6 @@ func (f *FrankenPHPApp) registerModule(repl *caddy.Replacer, module *FrankenPHPM

for _, w := range module.Workers {
w.FileName = repl.ReplaceKnown(w.FileName, "")
w.Name = f.createUniqueWorkerName(w, serverName)
workerOptions, err := w.toWorkerOptions()
if err != nil {
return err
Expand All @@ -236,37 +232,6 @@ func (f *FrankenPHPApp) registerModule(repl *caddy.Replacer, module *FrankenPHPM
return nil
}

// avoid name collisions for workers
// on collision, a name is first qualified with the server name
// ("<serverName>:<name>") before falling back to a numeric postfix
func (f *FrankenPHPApp) createUniqueWorkerName(wc workerConfig, serverName string) string {
if f.usedWorkerNames == nil {
f.usedWorkerNames = make(map[string]bool)
}

if wc.Name == "" {
wc.Name, _ = fastabs.FastAbs(wc.FileName)
}

Comment thread
nicolas-grekas marked this conversation as resolved.
name := wc.Name
suffix := 0
for {
if _, ok := f.usedWorkerNames[name]; !ok {
f.usedWorkerNames[name] = true
break
}
if serverName != "" {
name = serverName + ":" + wc.Name
serverName = ""
continue
}
suffix++
name = fmt.Sprintf("%s_%d", wc.Name, suffix)
}

return name
}

// UnmarshalCaddyfile implements caddyfile.Unmarshaler.
func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
for d.Next() {
Expand Down Expand Up @@ -381,10 +346,13 @@ func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
if len(wc.MatchPath) != 0 {
return d.Errf(`"match" can only be used in a php_server worker block, not in a global one: %q`, wc.FileName)
}
// check for duplicate workers
for _, existingWorker := range f.Workers {
if existingWorker.FileName == wc.FileName {
return d.Errf("global workers must not have duplicate filenames: %q", wc.FileName)
// check for duplicate workers; background workers are keyed
// by name, several may share a script
if !wc.Background {
for _, existingWorker := range f.Workers {
if !existingWorker.Background && existingWorker.FileName == wc.FileName {
return d.Errf("global workers must not have duplicate filenames: %q", wc.FileName)
}
}
}

Expand Down
103 changes: 71 additions & 32 deletions caddy/caddy_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -758,6 +758,45 @@ func TestMetrics(t *testing.T) {
require.NoError(t, testutil.GatherAndCompare(ctx.GetMetricsRegistry(), strings.NewReader(expectedMetrics), "frankenphp_total_threads", "frankenphp_busy_threads"))
}

// TestBackgroundWorkerFromCaddyfile starts a background worker from a
// Caddyfile and checks it runs: the sentinel its script touches appears
func TestBackgroundWorkerFromCaddyfile(t *testing.T) {
sentinel := filepath.ToSlash(filepath.Join(t.TempDir(), "bg.sentinel"))
tester := caddytest.NewTester(t)
initServer(t, tester, `
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443

frankenphp {
worker {
file ../testdata/bgworker/basic.php
num 1
name bg-caddy
background
env BG_SENTINEL `+sentinel+`
}
}
}

localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")

require.Eventually(t, func() bool {
_, err := os.Stat(sentinel)

return err == nil
}, 5*time.Second, 25*time.Millisecond, "the background worker declared in the Caddyfile did not run")
}

func TestWorkerMetrics(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
Expand Down Expand Up @@ -830,19 +869,19 @@ func TestWorkerMetrics(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="` + workerName + `"} 0
frankenphp_busy_workers{server="",worker="` + workerName + `"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="` + workerName + `"} 2
frankenphp_total_workers{server="",worker="` + workerName + `"} 2

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="` + workerName + `"} 10
frankenphp_worker_request_count{server="",worker="` + workerName + `"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="` + workerName + `"} 2
frankenphp_ready_workers{server="",worker="` + workerName + `"} 2
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -905,7 +944,7 @@ func TestPhpServerWorkerMatchPoolCount(t *testing.T) {

var pools []string
for line := range strings.SplitSeq(metrics.String(), "\n") {
if !strings.HasPrefix(line, "frankenphp_total_workers{worker=") {
if !strings.HasPrefix(line, "frankenphp_total_workers{") {
continue
}
if !strings.Contains(line, "dedup-match-worker.php") && !strings.Contains(line, "dedup-plain-worker.php") {
Expand Down Expand Up @@ -987,19 +1026,19 @@ func TestNamedWorkerMetrics(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="my_app"} 0
frankenphp_busy_workers{server="",worker="my_app"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="my_app"} 2
frankenphp_total_workers{server="",worker="my_app"} 2

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="my_app"} 10
frankenphp_worker_request_count{server="",worker="my_app"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="my_app"} 2
frankenphp_ready_workers{server="",worker="my_app"} 2
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1083,19 +1122,19 @@ func TestAutoWorkerConfig(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="` + workerName + `"} 0
frankenphp_busy_workers{server="",worker="` + workerName + `"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="` + workerName + `"} ` + workers + `
frankenphp_total_workers{server="",worker="` + workerName + `"} ` + workers + `

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="` + workerName + `"} 10
frankenphp_worker_request_count{server="",worker="` + workerName + `"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="` + workerName + `"} ` + workers + `
frankenphp_ready_workers{server="",worker="` + workerName + `"} ` + workers + `
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1356,7 +1395,7 @@ func TestMaxWaitTimeWorker(t *testing.T) {

expectedMetrics := `
# TYPE frankenphp_worker_queue_depth gauge
frankenphp_worker_queue_depth{worker="service"} 0
frankenphp_worker_queue_depth{server="",worker="service"} 0
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1457,21 +1496,21 @@ func TestMultiWorkersMetrics(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="service1"} 0
frankenphp_busy_workers{server="",worker="service1"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="service1"} 2
frankenphp_total_workers{worker="service2"} 3
frankenphp_total_workers{server="",worker="service1"} 2
frankenphp_total_workers{server="",worker="service2"} 3

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="service1"} 10
frankenphp_worker_request_count{server="",worker="service1"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="service1"} 2
frankenphp_ready_workers{worker="service2"} 3
frankenphp_ready_workers{server="",worker="service1"} 2
frankenphp_ready_workers{server="",worker="service2"} 3
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1622,12 +1661,12 @@ func TestWorkerRestart(t *testing.T) {

// Check metrics
expectedMetrics := `
# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="service"} 1
frankenphp_ready_workers{server="",worker="service"} 1
# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="service"} 1
frankenphp_total_workers{server="",worker="service"} 1
`

require.NoError(t,
Expand All @@ -1650,15 +1689,15 @@ func TestWorkerRestart(t *testing.T) {

// frankenphp_ready_workers should be back to 1 even after worker restarts
expectedMetrics = `
# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="service"} 1
frankenphp_ready_workers{server="",worker="service"} 1
# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="service"} 1
frankenphp_total_workers{server="",worker="service"} 1
# HELP frankenphp_worker_restarts Number of PHP worker restarts for this worker
# TYPE frankenphp_worker_restarts counter
frankenphp_worker_restarts{worker="service"} 3
frankenphp_worker_restarts{server="",worker="service"} 3
`

require.NoError(t,
Expand Down Expand Up @@ -2121,7 +2160,7 @@ func TestSymlinkWorkerBehavior(t *testing.T) {

// Accessing the worker script without worker configuration MUST fail
// The script checks $_SERVER['FRANKENPHP_WORKER'] and dies if not set
tester.AssertGetResponse("http://localhost:"+testPort+"/index.php", http.StatusOK, "Error: This script must be run in worker mode (FRANKENPHP_WORKER not set to '1')\n")
tester.AssertGetResponse("http://localhost:"+testPort+"/index.php", http.StatusOK, "Error: This script must be run in worker mode (FRANKENPHP_WORKER not set)\n")
})

t.Run("MultipleRequests", func(t *testing.T) {
Expand Down
Loading
Loading