GH-50859: [C++][Parquet] Move JsonWriter to simdjson utilities by rok · Pull Request #50990 · apache/arrow

rok · 2026-08-25T15:20:34Z

Rationale for this change

Parquet uses JsonWriter when ARROW_JSON=OFF, but its implementation was only built with Arrow JSON, causing link failures.

What changes are included in this PR?

Move JsonWriter to the simdjson utilities and update its callers and CMake/Meson builds.

Are these changes tested?

Yes. CMake shared/static and Meson Parquet builds pass with JSON disabled. Unit tests and pre-commit checks also pass.

Are there any user-facing changes?

No. This only fixes the affected build configuration.

AI disclosure - this was AI generated to test alternative approach to #50900.

GitHub Issue: [C++] parquet tools: linking error when ARROW_JSON=OFF with undefined reference to `arrow::json::JsonWriter::GetString() const' #50859

pitrou · 2026-08-25T16:06:57Z

The CI failures need fixing.

rok · 2026-08-25T16:55:01Z

@pitrou done.

Reranko05

@rok Since simdjson_internal.cc now exists, shouldn't all the non-template functions currently in simdjson_internal.h be moved into simdjson_internal.cc, leaving only declarations in the header?

rok · 2026-08-25T17:55:38Z

@rok Since simdjson_internal.cc now exists, shouldn't all the non-template functions currently in simdjson_internal.h be moved into simdjson_internal.cc, leaving only declarations in the header?

Done

Reranko05

LGTM!

rok · 2026-08-25T18:01:45Z

Again, this PR is purely AI generated. I will mark as ready for review once I review myself.

pitrou · 2026-08-27T12:27:37Z

@rok Are you willing to prioritize this?

rok · 2026-08-27T12:58:20Z

@pitrou I'll review this in about an hour and ping again

tadeja · 2026-08-27T13:35:19Z

@github-actions crossbow submit example-cpp-tutorial

github-actions · 2026-08-27T13:37:45Z

Revision: 973b4d2

Submitted crossbow builds: ursacomputing/crossbow @ actions-5765e8701c

Task	Status
example-cpp-tutorial

Parquet uses JsonWriter independently of the Arrow JSON module. Move the writer into the simdjson utilities so it is available whenever simdjson is enabled, including ARROW_JSON=OFF builds. Preserve the writer files as renames and update CMake, Meson, callers, and tests.

rok · 2026-08-27T15:25:59Z

+                 SOURCES
+                 json_writer_internal_test.cc
+                 EXTRA_LINK_LIBS
+                 simdjson::simdjson)


Should we use arrow::simdjson here instead? Both should work, but arrow alieas would hide the vendored/system simdjson.

if(SIMDJSON_VENDORED) add_library(arrow::simdjson ALIAS simdjson) else() add_library(arrow::simdjson ALIAS simdjson::simdjson) endif()

It seems better to use arrow::simdjson based on the above statements? cc @kou

Agreed, changed to arrow::simdjson.

rok · 2026-08-27T15:36:47Z

@github-actions crossbow submit example-cpp-tutorial

github-actions · 2026-08-27T15:40:36Z

Revision: 86e85bd

Submitted crossbow builds: ursacomputing/crossbow @ actions-e4578321bd

Task	Status
example-cpp-tutorial

github-actions · 2026-08-28T11:40:15Z

Revision: ab2c41a

Submitted crossbow builds: ursacomputing/crossbow @ actions-af7350541b

Task	Status
example-cpp-tutorial
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-minimal-with-formats

Copilot

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.

Copilot

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.

rok · 2026-08-28T13:22:37Z

@pitrou This is ready for review. CI failures appear unrelated and are present on main (JNI issue).

pitrou · 2026-08-31T08:11:31Z

@github-actions crossbow submit -g cpp

HuaHuaY · 2026-08-31T08:13:05Z


 Status JsonWriter::WriteValue(sj::value value) {
-  return internal::VisitJsonValue(
+  return ::arrow::internal::VisitJsonValue(


I think we don't need to add this prefix.

I think redundant namespace is now gone, see here.

github-actions · 2026-08-31T08:14:22Z

Revision: ab2c41a

Submitted crossbow builds: ursacomputing/crossbow @ actions-f7784ce147

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

pitrou · 2026-08-31T08:24:09Z

Why not put everything in simdjson_internal.h, under the same namespace?

pitrou · 2026-08-31T08:28:37Z

+       ARROW_UTIL_SRCS
+       json/object_parser.cc


If object_parser.cc is necessary even with ARROW_JSON disabled, perhaps it should be moved outside of the json directory? It seems it's Parquet-only by the way.

Moved to simdjson_internal as well.

Copilot

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

rok · 2026-08-31T12:28:54Z

Why not put everything in simdjson_internal.h, under the same namespace?

@pitrou done. Read and write utilities are now bundled together as they share simdjson helpers are internal-only.

rok · 2026-08-31T12:29:10Z

@github-actions crossbow submit -g cpp

github-actions · 2026-08-31T12:33:15Z

Revision: ea1bc35

Submitted crossbow builds: ursacomputing/crossbow @ actions-9ebf88aa33

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

pitrou · 2026-08-31T12:41:59Z

+
+/// This class is a helper to parse a JSON object from a string.
+/// It uses simdjson in the implementation.
+class ARROW_EXPORT ObjectParser {


Sorry for another nit, but arrow::internal::ObjectParser doesn't give any clue that it's about JSON anymore :-)

Should we name it JsonObjectParser perhaps?

github-actions Bot added Component: Parquet Component: C++ Component: FlightRPC awaiting committer review Awaiting committer review labels Aug 25, 2026

rok mentioned this pull request Aug 25, 2026

GH-50859: [C++] Build JsonWriter when Parquet is enabled without JSON #50900

Open

Reranko05 reviewed Aug 25, 2026

View reviewed changes

Reranko05 approved these changes Aug 25, 2026

View reviewed changes

pitrou mentioned this pull request Aug 27, 2026

GH-51013: [C++] Replace RapidJSON in JSON test utilities #51014

Open

rok force-pushed the gh-50859-parquet-without-json branch from 973b4d2 to 5a1f424 Compare August 27, 2026 14:51

rok commented Aug 27, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Aug 27, 2026

simplify meson change

86e85bd

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 27, 2026

rok marked this pull request as ready for review August 27, 2026 15:50

rok requested review from lidavidm, pitrou and wgtmac as code owners August 27, 2026 15:50

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 28, 2026

rok requested a lite review from Copilot August 28, 2026 12:55

Copilot started reviewing on behalf of rok August 28, 2026 12:56 View session

Copilot AI reviewed Aug 28, 2026

View reviewed changes

rok requested a balanced review from Copilot August 28, 2026 13:04

Copilot started reviewing on behalf of rok August 28, 2026 13:04 View session

Copilot AI reviewed Aug 28, 2026

View reviewed changes

rok requested a review from wgtmac August 28, 2026 20:12

HuaHuaY reviewed Aug 31, 2026

View reviewed changes

pitrou reviewed Aug 31, 2026

View reviewed changes

Comment thread cpp/src/arrow/util/simdjson_internal.h

pitrou reviewed Aug 31, 2026

View reviewed changes

move everything into simdjson_internal

ea1bc35

Copilot AI review requested due to automatic review settings August 31, 2026 11:57

Copilot started reviewing on behalf of rok August 31, 2026 11:58 View session

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 31, 2026

Copilot AI reviewed Aug 31, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 31, 2026

rok requested a review from pitrou August 31, 2026 12:32

pitrou reviewed Aug 31, 2026

View reviewed changes

Uh oh!

Conversation

rok commented Aug 25, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

pitrou commented Aug 25, 2026

Uh oh!

rok commented Aug 25, 2026

Uh oh!

Reranko05 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok commented Aug 25, 2026

Uh oh!

Reranko05 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pitrou commented Aug 27, 2026

Uh oh!

rok commented Aug 27, 2026

Uh oh!

tadeja commented Aug 27, 2026

Uh oh!

github-actions Bot commented Aug 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok commented Aug 27, 2026

Uh oh!

github-actions Bot commented Aug 27, 2026

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

rok commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pitrou commented Aug 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

pitrou commented Aug 31, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

rok commented Aug 31, 2026

Uh oh!

rok commented Aug 31, 2026

Uh oh!

rok commented Aug 25, 2026 •

edited by github-actions Bot

Loading

Reranko05 left a comment •

edited

Loading

Reranko05 left a comment •

edited

Loading

rok commented Aug 25, 2026 •

edited

Loading

rok commented Aug 28, 2026 •

edited

Loading