Repository navigation
feat: Add application/x-parquet support to clarify.ModelConfig (Issue #3450) - #6068
wasim-builds wants to merge 7 commits into
Conversation
|
Hi maintainers, Submitting this update regarding feat: Add application/x-parquet support to clarify.ModelConfig (Issue #3450) for aws/sagemaker-python-sdk. Working on Amazon SageMaker Python SDK machine learning and model deployment pipelines has been a great experience, and I am very keen to continue contributing to aws/sagemaker-python-sdk. I am also open to collaborating on a contract/freelance basis, discussing full-time opportunities, or joining the organization/team. You can view my background and other open-source contributions at https://github.com/wasim-builds. |
There was a problem hiding this comment.
🤖 AI code review
The PR adds application/x-parquet as a valid content_type and accept_type for clarify.ModelConfig, both in the schema validation and the ModelConfig.__init__ runtime checks, plus docstring updates. The change is small, correct, and consistent with the existing DatasetType.PARQUET enum and DataConfig parquet support (Source: #3450). Two minor points: no unit test was added for the new accepted value, and the content_type error message was made less informative.
Reviewed commit 168202c. Automated review; verify before acting.
| "image/jpg", | ||
| "image/png", | ||
| "application/x-npy", | ||
| "application/x-parquet", |
There was a problem hiding this comment.
🟠 Medium: The validation now accepts application/x-parquet for both accept_type and content_type, but no unit test exercises this new positive path. The existing tests only cover the invalid cases (test_init_with_invalid_accept_type_raises_error, test_init_with_invalid_content_type_raises_error in tests/unit/test_clarify.py). Please add a test constructing ModelConfig(..., content_type="application/x-parquet", accept_type="application/x-parquet") and asserting it is stored in predictor_config, so the newly supported behaviour is guarded against regressions.
| raise ValueError( | ||
| f"Invalid content_type {content_type}." | ||
| f" Please choose text/csv or application/jsonlines." | ||
| f" Please choose a valid content_type." |
There was a problem hiding this comment.
🟡 Low: While the accept_type error message was improved to list all valid values, this content_type message was changed to the vague "Please choose a valid content_type." The previous (also incomplete) text at least named options. Consider listing the valid content types here as you did for accept_type (e.g. text/csv, application/jsonlines, application/json, image/jpeg, image/png, application/x-npy, application/x-parquet) so users get actionable feedback.
There was a problem hiding this comment.
🤖 AI code review
The PR adds application/x-parquet as an accepted content_type and accept_type in clarify.ModelConfig, updating both the schema validation and the runtime checks in __init__, plus the docstrings. The change is correct and consistent with the existing DatasetType.PARQUET support. Two issues: the content_type error message was regressed to a generic string, and no unit test was added for the newly accepted value.
Findings not attached to a changed line:
sagemaker-core/src/sagemaker/core/clarify/__init__.py:1186🟡 Low: This changes thecontent_typevalidation error to a generic"Please choose a valid content_type.", which no longer lists the valid options. The retrieved guidance for this change states: "Error messages for validation failures should list all valid options, not generic text." (Source: #6068). The previous message was also wrong (it only mentioned csv/jsonlines), so prefer enumerating all accepted values here, as you did foraccept_typeon line 1166-1167.
Reviewed commit 55f3e64. Automated review; verify before acting.
| f"Invalid accept_type {accept_type}." | ||
| f" Please choose text/csv or application/jsonlines." | ||
| f" Please choose text/csv, application/jsonlines, application/json, or application/x-parquet." | ||
| ) |
There was a problem hiding this comment.
🟠 Medium: No unit test was added for the newly accepted application/x-parquet value. The existing tests in tests/unit/test_clarify.py only assert that invalid types raise, so this new acceptance path is untested. The guidance for this change notes: "Add unit tests when introducing new accepted values to ModelConfig content_type and accept_type validation to prevent regressions." (Source: #6068). Please add positive tests that construct a ModelConfig with content_type="application/x-parquet" and accept_type="application/x-parquet".
…nd add unit test for parquet
Closes #3450
Summary
Add
application/x-parquetsupport toModelConfigin the Clarify module.Changes
sagemaker-core/src/sagemaker/core/clarify/__init__.py: acceptapplication/x-parquetas a validcontent_typeandaccept_type.Verification
clarify.ModelConfignow accepts parquet content types without validation errors.