Should the 2023 aswf-docker container images include minizip-ng 4.0.10?

[jfpanisset]

The 2023.3 aswf-docker container images have minizip-ng 3.0.10, where mz.h contains an invalid hex constant:

#define MZ_VERSION_BUILD                (03000a)

which breaks code in OCIOZArchive.cpp starting with OCIO 2.3.x:

#if MZ_VERSION_BUILD >= 040000

minizip-ng 4.0.10, included in the 2024 / 2025 / 2026 images fixes that:

#define MZ_VERSION_BUILD (0x04000A)

Some CI permutations in other projects want to build OCIO 2.3.x in the 2023.3 images. Would it be an issue for OCIO if 2023 images moved to minizip-ng 4.0.10?

[doug-walker]

@jfpanisset , I don't think it would be an issue for OCIO if you raise the minizip-ng version in that container.

[lgritz]

I'm seeing this issue in broken 2023 containers for OSL as well -- because OSL has to build OIIO, and OIIO has to build OCIO, and OCIO breaks with the minizip-ng in the container.

[jfpanisset]

I was hoping that a simple hack like:

sed -i 's/03000a/0x03000A/' /usr/local/include/mz.h

would be enough to allow OCIO 2.3.2 to build against minizip-ng 3.0.10, but it seems like this code in OCIOZArchive.cpp:

#if MZ_VERSION_BUILD >= 040000
    write_mem_stream = mz_stream_mem_create();
#else
    mz_stream_mem_create(&write_mem_stream);
#endif

isn't really doing what it thinks it is doing, it would need to be something like:

#if MZ_VERSION_BUILD >= 0x040000
    write_mem_stream = mz_stream_mem_create();
#else
    mz_stream_mem_create(&write_mem_stream);
#endif

The following patch kinda makes this work, but of course it requires a patched version of mz.h 3.0.10 with a correct hex constant (the sed line above):

cat << 'EOF' | patch -p1
diff --git a/src/OpenColorIO/OCIOZArchive.cpp b/src/OpenColorIO/OCIOZArchive.cpp
index d99dd79ac..bfda2778a 100644
--- a/src/OpenColorIO/OCIOZArchive.cpp
+++ b/src/OpenColorIO/OCIOZArchive.cpp
@@ -226,7 +226,7 @@
     std::string configStr = ss.str();

     // Write zip to memory stream.
-#if MZ_VERSION_BUILD >= 040000
+#if MZ_VERSION_BUILD >= 0x040000
     write_mem_stream = mz_stream_mem_create();
 #else
     mz_stream_mem_create(&write_mem_stream);
@@ -242,7 +242,7 @@
     options.compress_level  = ArchiveCompressionLevels::BEST;

     // Create the writer handle.
-#if MZ_VERSION_BUILD >= 040000
+#if MZ_VERSION_BUILD >= 0x040000
     archiver = mz_zip_writer_create();
 #else
     mz_zip_writer_create(&archiver);
@@ -341,7 +341,7 @@
     std::string outputDestination = pystring::os::path::normpath(destination);

     // Create zip reader.
-#if MZ_VERSION_BUILD >= 040000
+#if MZ_VERSION_BUILD >= 0x040000
     extracter = mz_zip_reader_create();
 #else
     mz_zip_reader_create(&extracter);
@@ -463,7 +463,7 @@
     std::vector<uint8_t> buffer;

     // Create the reader object.
-#if MZ_VERSION_BUILD >= 040000
+#if MZ_VERSION_BUILD >= 0x040000
     reader = mz_zip_reader_create();
 #else
     mz_zip_reader_create(&reader);
@@ -527,7 +527,7 @@
     void *reader = NULL;

     // Create the reader object.
-#if MZ_VERSION_BUILD >= 040000
+#if MZ_VERSION_BUILD >= 0x040000
     reader = mz_zip_reader_create();
 #else
     mz_zip_reader_create(&reader);
diff --git a/src/apps/ocioarchive/main.cpp b/src/apps/ocioarchive/main.cpp
index d99dd79ac..bfda2778a 100644
--- a/src/apps/ocioarchive/main.cpp
+++ b/src/apps/ocioarchive/main.cpp
@@ -235,7 +235,7 @@
         }

         std::string path = args[0];
-#if MZ_VERSION_BUILD >= 040000
+#if MZ_VERSION_BUILD >= 0x040000
         reader = mz_zip_reader_create();
 #else
         mz_zip_reader_create(&reader);
EOF

[jfpanisset]

In general library projects really need to think about the versioning scheme they will be using up front, and make sure it is robust enough not to have to change in the future...

[jfpanisset]

In order to build OCIO 2.2.1 against Minizip 4.0.x (in the context of VFX 2023 images, which calls for OCIO 2.2.x), I need the following patch since the Minizip API has changed between 3.x and 4.x. This is not a problem when I build my Conan packages, I can add my own custom patches (which I typically need to do for things like CMake files which don't work in Conan), but I'm not sure how this would affect OCIO's CI. Top of tree / 2.5.2 should now build in the updated 2023 ci-ocio images (once they are released) against Minizip 4, but do you see ever having to build 2.2.1 again, and if so, would applying this patch during the build be acceptable? I don't imagine you want to make a 2.2.1 release just for Minizip 4 compatibility.

diff --git a/src/OpenColorIO/OCIOZArchive.cpp b/src/OpenColorIO/OCIOZArchive.cpp
index 17e188d..91af0ec 100755
--- a/src/OpenColorIO/OCIOZArchive.cpp
+++ b/src/OpenColorIO/OCIOZArchive.cpp
@@ -24,7 +24,6 @@
 #include "mz_strm_mem.h"
 #include "mz_strm_os.h"
 #include "mz_strm_split.h"
-#include "mz_strm_zlib.h"
 #include "mz_zip.h"
 #include "mz_zip_rw.h"

@@ -225,7 +224,11 @@
     std::string configStr = ss.str();

     // Write zip to memory stream.
+#if MZ_VERSION_BUILD >= 040000
+    write_mem_stream = mz_stream_mem_create();
+#else
     mz_stream_mem_create(&write_mem_stream);
+#endif
     mz_stream_mem_set_grow_size(write_mem_stream, 128 * 1024);
     mz_stream_open(write_mem_stream, NULL, MZ_OPEN_MODE_CREATE);

@@ -237,7 +240,11 @@
     options.compress_level  = ArchiveCompressionLevels::BEST;

     // Create the writer handle.
+#if MZ_VERSION_BUILD >= 040000
+    archiver = mz_zip_writer_create();
+#else
     mz_zip_writer_create(&archiver);
+#endif

     // Archive options.
     // Compression method
@@ -332,7 +339,11 @@
     std::string outputDestination = pystring::os::path::normpath(destination);

     // Create zip reader.
+#if MZ_VERSION_BUILD >= 040000
+    extracter = mz_zip_reader_create();
+#else
     mz_zip_reader_create(&extracter);
+#endif

     MinizipNgHandlerGuard extracterGuard(extracter, false, false);

@@ -450,7 +461,11 @@
     std::vector<uint8_t> buffer;

     // Create the reader object.
+#if MZ_VERSION_BUILD >= 040000
+    reader = mz_zip_reader_create();
+#else
     mz_zip_reader_create(&reader);
+#endif

     MinizipNgHandlerGuard extracterGuard(reader, false, true);

@@ -510,7 +525,11 @@
     void *reader = NULL;

     // Create the reader object.
+#if MZ_VERSION_BUILD >= 040000
+    reader = mz_zip_reader_create();
+#else
     mz_zip_reader_create(&reader);
+#endif

     MinizipNgHandlerGuard extracterGuard(reader, false, false);

diff --git a/src/apps/ocioarchive/main.cpp b/src/apps/ocioarchive/main.cpp
index 17e188d..91af0ec 100755
--- a/src/apps/ocioarchive/main.cpp
+++ b/src/apps/ocioarchive/main.cpp
@@ -235,7 +235,11 @@
         }

         std::string path = args[0];
+#if MZ_VERSION_BUILD >= 040000
+        reader = mz_zip_reader_create();
+#else
         mz_zip_reader_create(&reader);
+#endif
         struct tm tmu_date;

         err = mz_zip_reader_open_file(reader, path.c_str());

[KevinJW]

I was about to quote our support policy, but as I can't easily find one :-) I think it is safe to say we are not going to release a 2.2.x (or probably even a 2.3.x) without a world falling down security issue.

[jfpanisset]

The following images have been released:

• 2023.4
• 2024.8
• 2025.7
• 2026.5
• 2027.0

These now include the corresponding OIIO version for that VFX Platform year:

2023.4: 2.4.17.0
2024.8: 2.5.19.1
2025.7: 3.1.6.2
2026.5: 3.1.14.1
2027.0: 3.1.14.1 (the 2027.0 images are preview / draft since for now they include the 2026 OCIO / OpenEXR / OpenVDB until the 2027 versions are out)

2023 / 2024 images have been retrofited to use minizip-ng 4.0.10, which in turn requires patching OCIO 2.2.1 to build again. See:

https://github.com/AcademySoftwareFoundation/aswf-docker/blob/main/scripts/vfx/build_ocio.sh

Also 2.3.2 and 2.4.2 need their CMake patched to find the right Python version for the VFX Platform year instead of the newer Python available in the image in /usr/bin/.

OCIO 2.5.2 builds correctly in 2023-2024-2025 images, but under those combinations you must disable OCIO_USE_OIIO_FOR_APPS since the version of OIIO included in those images will be looking for an older OCIO.

[doug-walker]

I've updated our CI to use the new container images. Everything is now passing.

I used claude to enable OCIO_USE_OIIO_FOR_APPS by downloading the required OCIO library from Artifactory.

[jfpanisset]

Does that not mean that when you are building the "latest" / 2.5.2 version of OCIO apps which require OIIO in a previous VFX platform year container, you end up linking to two separate versions of libOpenColorIO, the one you just built, and transitively the older one needed to satisfy the dependency of the older libOpenImageIO in the container image? I guess if OCIO is properly namespaced that should be fine, but I'm not sure how representative of a real use case that is.

If this is what you actually need, I could also add the "correct" legacy OCIO version to the ci-ocio images to satisfy the OIIO dependency. In the end these images are meant to be useful to the project, so whatever works best for OCIO.

[doug-walker]

Does that not mean that when you are building the "latest" / 2.5.2 version of OCIO apps which require OIIO in a previous VFX platform year container, you end up linking to two separate versions of libOpenColorIO, the one you just built, and transitively the older one needed to satisfy the dependency of the older libOpenImageIO in the container image? I guess if OCIO is properly namespaced that should be fine, but I'm not sure how representative of a real use case that is.

Yes, that is what is being done in all build years when use-oiio=ON. (Even in the current year, the CI is compiling OCIO 2.6_dev, but OIIO is linked to 2.5.2, so it's not using the same library.) OCIO's namespacing is designed to handle this and I think this actually is a real-world scenario for large apps that integrate many separate components.

If this is what you actually need, I could also add the "correct" legacy OCIO version to the ci-ocio images to satisfy the OIIO dependency. In the end these images are meant to be useful to the project, so whatever works best for OCIO.

Thanks JF, I might ask you to do that after discussing with the TSC on Monday.

[jfpanisset]

Created this issue not to forget, and will await TSC guidance.

AcademySoftwareFoundation/aswf-docker#444

[doug-walker]

JF, there was no guidance on this at the TSC meeting. Why don't we leave it as is for now and I'll reach out if we'd like any changes.

[jfpanisset]

No problem, holding off for now. Don't hesitate if you have any requests for these images, they are meant to make your life easier if at all possible.