Skip to content

feat: Add bulk ZIP export (#867)#222

Open
stephanbuettig wants to merge 1 commit intohttptoolkit:mainfrom
stephanbuettig:feature/zip-export
Open

feat: Add bulk ZIP export (#867)#222
stephanbuettig wants to merge 1 commit intohttptoolkit:mainfrom
stephanbuettig:feature/zip-export

Conversation

@stephanbuettig
Copy link
Copy Markdown

Adds ZIP archive export for HTTP exchanges with 37 code snippet formats via @httptoolkit/httpsnippet. Includes format picker panel, Web Worker generation, and safe filename conventions.

Features:

  • ZIP export with selectable snippet formats (37 languages/clients)
  • Format picker with category grouping and popular defaults
  • Web Worker-based generation for non-blocking UI
  • Safe filename conventions matching existing HAR export pattern

New files: snippet-formats registry, export-filenames utility, download helper, zip-metadata model, zip-download-panel component.

Unit tests for snippet-formats and export-filenames included.

Extracted from #219 as requested by @pimterry.

Adds ZIP archive export for HTTP exchanges with 37 code snippet formats
via @httptoolkit/httpsnippet. Includes format picker panel, Web Worker
generation, and safe filename conventions.

Features:
- ZIP export with selectable snippet formats (37 languages/clients)
- Format picker with category grouping and popular defaults
- Web Worker-based generation for non-blocking UI
- Safe filename conventions matching existing HAR export pattern

New files: snippet-formats registry, export-filenames utility,
download helper, zip-metadata model, zip-download-panel component.

Unit tests for snippet-formats and export-filenames included.

Extracted from httptoolkit#219 as requested by @pimterry.
@stephanbuettig
Copy link
Copy Markdown
Author

✅ Manual Test Results — 2026-04-11

Both features were tested against a fresh clone of current upstream (main, commit 23a99520) using npm start.

ZIP Export (this PR)

All runs completed with 0 snippet errors:

Scenario Formats Snippets Errors Time Size
Single request, all formats 37 37 0 87 ms 67 KB
Single large request, all formats 37 37 0 290 ms 156 KB
Single request, 7 formats 7 7 0 45 ms 8.5 KB

The ZIP download panel opens correctly, format selection persists across sessions, and the generated archives are valid and well-structured.

Batch export (PR #223, depends on this PR)

Scenario Formats Snippets Errors Time Size
13 exchanges, 7 formats 7 91 0 254 ms 152 KB
14 exchanges, all formats 37 518 0 2.1 s 795 KB

Both features are production-ready and work correctly on the current upstream codebase. Ready for review and merge.

Copy link
Copy Markdown
Member

@pimterry pimterry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't gone through everything in detail yet, especially the actual UI component code, and I haven't tested this manually either, but I think this is a good set of bits to start with from a quick review. There's a strong outline here but there's going to be a good bit of work to properly integrate this into the codebase and make it maintainable for the future.

};

// Build extended optGroups with ZIP at the top
const exportOptionsWithZip: _.Dictionary<SnippetOption[]> = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to import the whole of lodash for this one type, you can just use Record<string, SnippetOption[]>.

There are probably old examples of doing that elsewhere here - they predate Record being added to typescript, if you spot any along the way feel free to clean them up 😄.

*
* Contains ALL available HTTPSnippet targets/clients organized by language
* category. The ZIP export pipeline, format picker UI, and batch toolbar
* all consume this registry.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this here? HTTPSnippet already has a registry, we don't want to duplicate it, since the options available will change and this will get out of date very quickly.


@action.bound
setZipFormatIds(ids: ReadonlySet<string> | string[]) {
this._zipFormatIds = Array.isArray(ids) ? [...ids] : [...ids];
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ternary does nothing? Both options are the same.

metadata
} as Omit<GenerateZipRequest, 'id'>));
} catch (err) {
// postMessage can throw for unserializable data (MobX proxies, etc.)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When does this happen? We probably shouldn't catch this - we need to fix that instead. MobX proxies need to be filtered out etc. Otherwise we'll find that the export doesn't work in lots of common cases and data will silently go missing, which is a big problem.

Do you have any examples? Normally it's best to let this fail hard instead - that way it's easy to spot these issues in testing, and they'll show up in the Sentry error reports for debugging & tracking later as well.

Imo it's good to catch errors when they're due to expected issues like bad data or user input or configuration. If it can be due to implementation problems, we need to make sure that's very visible now in testing (crashing the app on purpose) and that we then fix all the problems to handle it.

};

// Safety timeout: if the worker doesn't respond within 5 minutes,
// clean up the listener to prevent memory leaks.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't really make sense, it doesn't look like handler has any references that would cause leaks here, and this doesn't actually cancel the processing, it leaves it going indefinitely. If there's a real risk it could take this long it's a problem because it will block the worker completely.

That means every other worker operation (like decoding any compressed request or response body) will wait until this is finished. If these are that slow we'll need to implement actual cancellation, look into abort controllers for how signals for that kind of thing can work.

harEntries,
formats,
metadata
} as Omit<GenerateZipRequest, 'id'>));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this use a custom wrapper instead of callApi? I'd much prefer to keep everything using the same abstraction if we can. It doesn't have progress of course, or any abort support, but both could be added there instead of here, and then that would work for all worker calls which would be great.

),
cookies: [], // Included in headers already
...(postData !== undefined ? { postData } : {})
};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we already have this same preprocessing logic in the normal export snippet generation? Is this intentionally different for some reason? I would've expected to just reuse that. Likely to give better results for users, since it guarantees the content you see in the Export card is the same thing per-request you see in the batch zip export.

* - HAR batch: "HTTPToolkit_export_{date}_{count}-requests.har"
* - ZIP archive: "HTTPToolkit_{date}_{count}-requests.zip"
* - Snippet: "{index}_{METHOD}_{STATUS}_{hostname}.{ext}"
*/
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of just referencing the other name patterns elsewhere, we could just move the logic for all of these into here.

expect(result.length).to.equal(1);
});
});

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests just assert on a selection of hardcoded data values, which doesn't seem very useful.

I think this fail at least could probably just go away (along with the hardcoded data itself).

It would be useful to have a proper end to end test of zip generation though. We should be able to do that, we do similar testing including worker API calls in https://github.com/httptoolkit/httptoolkit-ui/blob/main/test/unit/workers/worker-decoding.spec.ts.

Doesn't need to cover every possible edge cases there, we just need a basic covering of the overall key flows to make sure the structure and key behaviours work. That's probably just the success case with a couple of examples and an error case (if there are scenarios where we expect this to fail). You can then use fflate in the test to check the expected output appears. No need to test on specific code snippet contents for specific inputs or anything like that (that's covered in a lot of detail already by httpsnippet's own tests) just that the whole flow glues together correctly.

// This is never passed to httpsnippet — it's only used for dropdown rendering.
const ZIP_SNIPPET_OPTION: SnippetOption = {
target: ZIP_ALL_FORMAT_KEY as any,
client: '' as any,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These any are a bit suspicious. I think we should have some kind of better types that make this work properly, without this and witohut an extra special case functions like getExportFormatKey that just wraps getCodeSnippetFormatKey with one extra condition. We probably want something like export ExportOption = ZipExportOption | SnippetOption somewhere with some kind of discriminated union, and then to change the various references take either ExportOption or SnippetOption as appropriate, and discriminate to make all those types work correctly, without any.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants