refactor: multiprocessing helper by NguyenHoangSon96 · Pull Request #222 · InfluxCommunity/influxdb3-python

NguyenHoangSon96 · 2026-07-14T18:14:56Z

Closes #

Proposed Changes

Refactor MultiprocessingWriter class:
- WriteApi will be created and used directly in the class for writing.
- MultiprocessingWriter will create processing by "spawn" method by default, but users can pass the processing creation method as a parameter to constructor.
- Tests are added for writing with multiprocessing.

Checklist

codecov · 2026-07-14T18:18:58Z

Codecov Report

❌ Patch coverage is 87.85249% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.58%. Comparing base (a022daa) to head (ab75516).

Files with missing lines	Patch %	Lines
influxdb_client_3/write_client/client/write_api.py	84.91%	43 Missing ⚠️
...nfluxdb_client_3/write_client/_sync/rest_client.py	93.49%	8 Missing ⚠️
...write_client/client/util/multiprocessing_helper.py	78.57%	3 Missing ⚠️
influxdb_client_3/__init__.py	97.22%	1 Missing ⚠️
...luxdb_client_3/write_client/client/util/helpers.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #222      +/-   ##
==========================================
+ Coverage   78.90%   88.58%   +9.68%     
==========================================
  Files          36       28       -8     
  Lines        2607     2068     -539     
==========================================
- Hits         2057     1832     -225     
+ Misses        550      236     -314

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

karel-rehor · 2026-07-15T12:39:40Z

+                bucket=self.database,
+                record=f"{measurement},tag=a value=\"number{x}\" {time.time_ns()}"
+            )
+        writer.__del__()


Please don't forget that ideally __del__() should not be called directly, and that instead a with block should be used here.

https://github.com/InfluxCommunity/influxdb3-python/pull/217/changes#r3507988624

Copilot

Pull request overview

This PR refactors the write stack by removing the generated OpenAPI-style service/client layer and moving write request construction and HTTP dispatch directly into WriteApi, while also updating tests and documentation to reflect the new behavior and configuration surface.

Changes:

Refactors WriteApi to perform request preparation + HTTP calls directly via a new low-level RestClient (removing ApiClient/WriteService/Configuration).
Updates InfluxDBClient3 initialization and tests to use the new header/rest client approach and to validate required constructor keys.
Refactors and adds integration coverage for MultiprocessingWriter start-method behavior and direct WriteApi usage.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
CHANGELOG.md	Documents breaking changes and new construction patterns (example needs a small syntax fix).
influxdb_client_3/init.py	Reworks `InfluxDBClient3` construction to create `RestClient` + `WriteApi` directly and enforce required keys.
influxdb_client_3/write_client/init.py	Removes deprecated exports and re-exports updated `WriteApi`/types.
influxdb_client_3/write_client/_sync/api_client.py	Removes legacy generated `ApiClient`.
influxdb_client_3/write_client/_sync/rest.py	Removes legacy generated REST client implementation.
influxdb_client_3/write_client/_sync/rest_client.py	Adds new `RestClient` for low-level HTTP handling + logging/redaction.
influxdb_client_3/write_client/client/init.py	Removes re-export of removed `WriteService`.
influxdb_client_3/write_client/client/_base.py	Removes legacy base client/write-api helpers tied to generated stack.
influxdb_client_3/write_client/client/influxdb_client.py	Removes legacy v2 `InfluxDBClient`.
influxdb_client_3/write_client/client/logging_handler.py	Removes `InfluxLoggingHandler` implementation.
influxdb_client_3/write_client/client/util/helpers.py	Updates exception import path.
influxdb_client_3/write_client/client/util/multiprocessing_helper.py	Refactors `MultiprocessingWriter` to create `WriteApi` directly and set default start method.
influxdb_client_3/write_client/client/write/init.py	Removes legacy re-export of removed `WriteService`.
influxdb_client_3/write_client/client/write_api.py	Major refactor: `WriteApi` becomes standalone, builds write requests and calls `RestClient` directly, retains batching pipeline.
influxdb_client_3/write_client/configuration.py	Removes legacy generated configuration object.
influxdb_client_3/write_client/service/init.py	Removes legacy service package init exporting `WriteService`.
influxdb_client_3/write_client/service/_base_service.py	Removes legacy service base class.
influxdb_client_3/write_client/service/write_service.py	Removes legacy generated `WriteService`.
influxdb_client_3/write_client/write_exceptions.py	Removes `_BaseRESTClient` logger helper (moved to `RestClient`).
tests/test_influxdb_client_3.py	Updates unit tests for new client/write header/rest behavior and required params.
tests/test_influxdb_client_3_integration.py	Adds integration tests for direct `WriteApi` usage, multiprocessing helper, and async call path.
tests/test_polars.py	Updates write tests to mock `WriteApi` call sites instead of `WriteService`.
tests/test_query.py	Fixes typo in client init parameter (`database`).
tests/test_write_api.py	Refactors tests from generated service/client usage to `InfluxDBClient3`/`WriteApi` usage.
tests/test_write_local_server.py	Updates exception import path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.

NguyenHoangSon96 · 2026-07-16T15:12:12Z

+    def __init__(self, start_method='spawn', **kwargs) -> None:
        """
        Initialize defaults.

-        For more information how to initialize the writer see the examples above.
+        For more information on how to initialize the writer, see the examples above.

        :param kwargs: arguments are passed into ``__init__`` function of ``InfluxDBClient`` and ``write_api``.
        """
+        multiprocessing.set_start_method(start_method, force=True)
        multiprocessing.Process.__init__(self)


NguyenHoangSon96 · 2026-07-16T14:53:35Z

+        for key in ["host", "token", "database"]:
+            if locals().get(key) is None:
+                raise Exception(f"The '{key}' key is required")
+


Not necessary.

NguyenHoangSon96 · 2026-07-16T15:16:33Z

@bednar @karel-rehor @alespour
Hi guys,
I made some quite big changes to our MultiprocessingWriter class.
Please take a look. I'm not very familiar with multiprocessing in Python.

NguyenHoangSon96 self-assigned this Jul 14, 2026

karel-rehor reviewed Jul 15, 2026

View reviewed changes

karel-rehor mentioned this pull request Jul 15, 2026

refactor: write api and rest client #217

Open

6 tasks

NguyenHoangSon96 force-pushed the refactor/multiprocessing-helper branch 4 times, most recently from 6fbdf4b to f227120 Compare July 16, 2026 09:25

NguyenHoangSon96 requested a review from karel-rehor July 16, 2026 11:50

NguyenHoangSon96 force-pushed the refactor/multiprocessing-helper branch 2 times, most recently from 706407e to 53ac912 Compare July 16, 2026 11:55

NguyenHoangSon96 marked this pull request as ready for review July 16, 2026 12:10

Copilot AI review requested due to automatic review settings July 16, 2026 12:10

Copilot started reviewing on behalf of NguyenHoangSon96 July 16, 2026 12:11 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread influxdb_client_3/write_client/client/util/multiprocessing_helper.py Outdated

Comment thread influxdb_client_3/write_client/client/util/multiprocessing_helper.py

NguyenHoangSon96 requested a review from Copilot July 16, 2026 12:40

Copilot started reviewing on behalf of NguyenHoangSon96 July 16, 2026 12:40 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

NguyenHoangSon96 force-pushed the refactor/multiprocessing-helper branch 3 times, most recently from 206c4a9 to f6ef07c Compare July 16, 2026 15:09

refactor: multiprocess helper class

ab75516

NguyenHoangSon96 force-pushed the refactor/multiprocessing-helper branch from f6ef07c to ab75516 Compare July 16, 2026 15:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor: multiprocessing helper#222

refactor: multiprocessing helper#222
NguyenHoangSon96 wants to merge 1 commit into
mainfrom
refactor/multiprocessing-helper

NguyenHoangSon96 commented Jul 14, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

karel-rehor Jul 15, 2026

Uh oh!

NguyenHoangSon96 Jul 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

NguyenHoangSon96 Jul 16, 2026

Uh oh!

Uh oh!

NguyenHoangSon96 Jul 16, 2026

Uh oh!

NguyenHoangSon96 commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

NguyenHoangSon96 commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Changes

Checklist

Uh oh!

codecov Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

karel-rehor Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

NguyenHoangSon96 Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

NguyenHoangSon96 Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NguyenHoangSon96 Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

NguyenHoangSon96 commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

NguyenHoangSon96 commented Jul 14, 2026 •

edited

Loading

codecov Bot commented Jul 14, 2026 •

edited

Loading