[CF-4208] Add `--page-size` to on-prem Flink list commands by paras-negi-flink · Pull Request #3424 · confluentinc/cli

Paras Negi (paras-negi-flink) · 2026-07-31T15:26:15Z

Release Notes

New Features

Added a --page-size flag to the on-prem (Confluent Platform / CMF) confluent flink ... list commands, to fetch large result sets in fewer, larger API requests. Defaults to 100 — existing behavior is unchanged.

Checklist

What

Confluent Platform (CMF on-prem) only — part of CF-4208; Confluent Cloud flink commands are untouched.

On-prem Flink list commands paginate at a hardcoded size=100 and always fetch every page, so large environments cost many serial round trips (~51 requests for 5000 applications — CF-4202).

Adds --page-size to all paginated on-prem list commands (application / event / instance / catalog / catalog database / compute-pool / detached-savepoint / environment / savepoint / secret / secret-mapping / statement list). Shared addPageSizeFlag/getPageSize in command.go validate 0..MaxInt32; 0 = the default 100.
Refactors the 12 duplicated pagination loops in pkg/flink/cmf_rest_client.go into one generic listAllPages helper (net −84 lines). statement exception list is untouched (non-paginated).

Blast Radius

Additive optional flag; --page-size 0 reproduces today's output exactly. Non-breaking, easy to revert.
The shared pagination refactor is guarded by a listAllPages unit test plus multi-page integration coverage.

References

Jira: CF-4208; motivation CF-4202.
Stacked follow-up (server-side filtering): [CF-4208] Add --name/--status filtering to on-prem application list #3428.

Test & Review

Unit TestListAllPages: default / custom / large size (fewer round trips), size > total, empty, error propagation.
Integration: --page-size on application / statement / detached-savepoint list (full list fetched across multiple pages) + invalid --page-size -1; help goldens regenerated. make lint clean.

$ confluent flink application list --environment default --page-size -1
Error: `--page-size` must be between 0 and 2147483647

# small or large page size -> identical output, just more/fewer round trips
$ confluent flink application list --environment default --page-size 2
          Name          | Environment |     Job Name      | Job Status
------------------------+-------------+-------------------+--------------
  default-application-1 | default     | State machine job | RECONCILING
  default-application-2 | default     | State machine job | RECONCILING
  default-application-s | default     | State machine job | RECONCILING

confluent-cla-assistant · 2026-07-31T15:26:25Z

❌ Error getting contributor login(s).
Please ensure the email address associated with this commit is added to your Github account.

Copilot

Pull request overview

This PR closes CF-4208 by adding user-facing pagination control (--limit) across on-prem Flink “list” commands and introducing flink application list filtering (--name, --status). It also refactors the CMF REST client to centralize paging behavior (including limit short-circuiting) and updates test server behavior plus integration fixtures accordingly.

Changes:

Add a shared --limit flag (default 0 = unlimited) to on-prem Flink list commands, applying it server-side where endpoints are paginated and client-side for non-paginated exception listing.
Add --name / --status filtering to flink application list by composing CMF’s generic filter query parameter.
Refactor duplicated CMF paging loops into a generic listAllPages helper and add unit + integration coverage (with updated goldens).

Reviewed changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/test-server/flink_onprem_handler.go	Adds CMF applications `filter` query emulation in the test server.
test/flink_onprem_test.go	Adds integration coverage for `--limit` and application list filtering flags.
test/fixtures/output/flink/statement/list-limit-json.golden	New golden output for statement list with `--limit`.
test/fixtures/output/flink/statement/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/statement/list-env-missing-failure.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/statement/exception/list-help-onprem.golden	Updates help golden to include `--limit` for exception list.
test/fixtures/output/flink/secret/list-help-onprem.golden	Updates help golden to include `--limit` for secret list.
test/fixtures/output/flink/secret-mapping/list-help-onprem.golden	Updates help golden to include `--limit` for secret-mapping list.
test/fixtures/output/flink/savepoint/list-help-onprem.golden	Updates help golden to include `--limit` for savepoint list.
test/fixtures/output/flink/savepoint/list-fail-both.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/environment/list-help-onprem.golden	Updates help golden to include `--limit` for environment list.
test/fixtures/output/flink/detached-savepoint/list-limit-json.golden	New golden output for detached-savepoint list with `--limit`.
test/fixtures/output/flink/detached-savepoint/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/compute-pool/list-missing-env-flag-failure.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/compute-pool/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/catalog/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/catalog/database/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/application/list-status-no-match-json.golden	New golden output for application list status filter no-match case.
test/fixtures/output/flink/application/list-status-filter-json.golden	New golden output for application list status filter match case.
test/fixtures/output/flink/application/list-name-filter-json.golden	New golden output for application list name filter case.
test/fixtures/output/flink/application/list-limit-json.golden	New golden output for application list with `--limit`.
test/fixtures/output/flink/application/list-limit-invalid.golden	New golden output for invalid `--limit` value.
test/fixtures/output/flink/application/list-help-onprem.golden	Updates help golden to include `--name`, `--status`, and `--limit`.
test/fixtures/output/flink/application/list-env-missing.golden	Updates env-missing output to include new flags in help text.
test/fixtures/output/flink/application/instance/list-help-onprem.golden	Updates help golden to include `--limit` for instance list.
test/fixtures/output/flink/application/instance-list-env-missing.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/application/instance-list-app-missing.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/application/event/list-help-onprem.golden	Updates help golden to include `--limit` for event list.
test/fixtures/output/flink/application/event-list-missing-flags.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/application/event-list-app-missing.golden	Updates failure/help golden to include `--limit`.
pkg/flink/test/mock/cmf_client_mock.go	Updates generated CMF client mock for the new `ListStatements(..., limit)` signature.
pkg/flink/cmf_rest_client.go	Refactors paging into `listAllPages` and threads `limit` through CMF list calls.
pkg/flink/cmf_rest_client_test.go	Adds unit tests for `listAllPages` behavior across limits and paging scenarios.
internal/flink/command.go	Adds shared `--limit` flag registration and parsing helper (`getLimit`).
internal/flink/command_statement_list_onprem.go	Wires `--limit` into on-prem statement list via CMF client.
internal/flink/command_statement_exception_list_onprem.go	Adds `--limit` and truncates exceptions client-side for non-paginated endpoint.
internal/flink/command_secret_mapping_list.go	Wires `--limit` into secret-mapping list.
internal/flink/command_secret_list.go	Wires `--limit` into secret list.
internal/flink/command_savepoint_list.go	Wires `--limit` into savepoint list.
internal/flink/command_environment_list.go	Wires `--limit` into environment list.
internal/flink/command_detached_savepoint_list.go	Wires `--limit` into detached-savepoint list.
internal/flink/command_compute_pool_list_onprem.go	Wires `--limit` into compute-pool list.
internal/flink/command_catalog_list.go	Wires `--limit` into catalog list.
internal/flink/command_catalog_database_list.go	Wires `--limit` into catalog database list.
internal/flink/command_application_list.go	Adds `--name`/`--status` filtering and wires `--limit` into application list.
internal/flink/command_application_instance_list.go	Wires `--limit` into application instance list.
internal/flink/command_application_event_list.go	Wires `--limit` into application event list.

Files not reviewed (1)

pkg/flink/test/mock/cmf_client_mock.go: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The on-prem Flink (CPF/CMF) list commands loop over a hardcoded 100-item page size with no way to control it, so listing a large environment costs many serial round trips (~51 requests for 5000 applications). Add a --page-size flag to every paginated on-prem list command so callers can fetch the full list in fewer, larger requests. It defaults to 100, which preserves today's behavior and output. Collapse the 12 duplicated pagination loops in the CMF client into a single generic listAllPages helper that takes the page size. The requested size is passed through to the CMF page/size API; the loop still terminates on the first empty page. Server-side filtering for `application list` is deliberately left out and will follow in a separate PR stacked on this one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…logical pagination cases - Reword the --page-size help so it reads as a per-request batch size, not a cap on total results (regenerate the affected help/usage goldens). - Add an integration case for --page-size > MaxInt32 (3000000000), exercising the previously untested upper-bound branch of getPageSize (reuses the invalid golden). - Drop the statement/detached-savepoint --page-size cases: those mock handlers ignore the size param, so the cases only verified flag plumbing (already covered by the help goldens) rather than pagination. The applications case (mock genuinely pages) plus TestListAllPages remain the real coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Steven Gagniere (sgagniere)

Hi, just a few comments. Otherwise lgtm

Steven Gagniere (sgagniere) · 2026-08-06T20:28:43Z

 }

+func addPageSizeFlag(cmd *cobra.Command) {
+	cmd.Flags().Int("page-size", 0, "Number of results to fetch per API request while paginating; does not cap the total results returned. Defaults to 100.")


Suggested change

cmd.Flags().Int("page-size", 0, "Number of results to fetch per API request while paginating; does not cap the total results returned. Defaults to 100.")

cmd.Flags().Int("page-size", 100, "Number of results to fetch per API request while paginating; does not cap the total results returned. Defaults to 100.")

I think we can set this to 100 to match the description.

Steven Gagniere (sgagniere) · 2026-08-06T20:30:01Z

+func getPageSize(cmd *cobra.Command) (int32, error) {
+	pageSize, err := cmd.Flags().GetInt("page-size")
+	if err != nil {
+		return 0, err
+	}
+	if pageSize < 0 || pageSize > math.MaxInt32 {
+		return 0, fmt.Errorf("`--page-size` must be between 0 and %d", math.MaxInt32)
+	}
+	return int32(pageSize), nil
+}


Generally we want to avoid client side validation for things that aren't recorded in the spec, so I think we should remove these checks and treat this like a regular flag read.

Good catch, fixed in the latest commit.

…e validation Per review on #3424: - Default the flag to 100 (matching the documented behavior) instead of the 0 sentinel, and drop the redundant "Defaults to 100." text since pflag now renders "(default 100)". - Change the flag type to Int32 and read it directly, removing the hand-rolled range validation. Out-of-range input is now rejected by the flag parser itself (no client-side validation not backed by the spec), which also closes the int32-overflow concern. Non-positive values fall back to the default page size in listAllPages. Regenerated the affected help and usage goldens. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings July 31, 2026 15:26

Copilot started reviewing on behalf of Paras Negi (paras-negi-flink) July 31, 2026 15:26 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

Comment thread internal/flink/command.go Outdated

Comment thread pkg/flink/cmf_rest_client_test.go Outdated

airlock-confluentinc Bot force-pushed the cli-cf-4208-flink-list-limit-filter branch from bba9ac2 to 1ecbfcb Compare August 2, 2026 08:38

Paras Negi (paras-negi-flink) changed the title ~~[CF-4208] Add --limit pagination and application list filtering to on-prem Flink~~ [CF-4208] Add --page-size to on-prem Flink list commands Aug 2, 2026

Paras Negi (paras-negi-flink) mentioned this pull request Aug 2, 2026

[CF-4208] Add --name/--status filtering to on-prem application list #3428

Open

11 tasks

Paras Negi (paras-negi-flink) marked this pull request as ready for review August 6, 2026 18:49

Paras Negi (paras-negi-flink) requested a review from a team as a code owner August 6, 2026 18:49

Steven Gagniere (sgagniere) reviewed Aug 6, 2026

View reviewed changes

Steven Gagniere (sgagniere) approved these changes Aug 7, 2026

View reviewed changes

Navigation Menu

Uh oh!

[CF-4208] Add `--page-size` to on-prem Flink list commands - #3424

[CF-4208] Add `--page-size` to on-prem Flink list commands#3424
Paras Negi (paras-negi-flink) wants to merge 3 commits into
mainfrom
cli-cf-4208-flink-list-limit-filter

Paras Negi (paras-negi-flink) commented Jul 31, 2026 •

edited

Loading

Uh oh!

confluent-cla-assistant Bot commented Jul 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Steven Gagniere (sgagniere) left a comment

Uh oh!

Steven Gagniere (sgagniere) Aug 6, 2026

Uh oh!

Paras Negi (paras-negi-flink) Aug 7, 2026

Uh oh!

Steven Gagniere (sgagniere) Aug 6, 2026

Uh oh!

Paras Negi (paras-negi-flink) Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	cmd.Flags().Int("page-size", 0, "Number of results to fetch per API request while paginating; does not cap the total results returned. Defaults to 100.")
	cmd.Flags().Int("page-size", 100, "Number of results to fetch per API request while paginating; does not cap the total results returned. Defaults to 100.")

Uh oh!

Conversation

Paras Negi (paras-negi-flink) commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release Notes

Checklist

What

Blast Radius

References

Test & Review

Uh oh!

confluent-cla-assistant Bot commented Jul 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Steven Gagniere (sgagniere) left a comment

Choose a reason for hiding this comment

Uh oh!

Steven Gagniere (sgagniere) Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Paras Negi (paras-negi-flink) Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

Steven Gagniere (sgagniere) Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Paras Negi (paras-negi-flink) Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Paras Negi (paras-negi-flink) commented Jul 31, 2026 •

edited

Loading