feat: add tokenizer module with sync and async support

[amourao]

• Global tokenizer and per-property
• Tests

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Secrets	0   0   0   0	View in Orca

[Copilot]

Pull request overview

Adds a new tokenize client module (sync + async) to call Weaviate’s tokenization endpoints and returns a typed TokenizeResult, with integration tests covering serialization/deserialization and both client variants.

Changes:

• Introduce weaviate.tokenize module with shared executor logic plus sync/async wrappers.
• Add TokenizeResult return type and response parsing for analyzer/stopword configs.
• Wire client.tokenize into WeaviateClient / WeaviateAsyncClient and add integration tests.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
weaviate/tokenize/types.py	Defines TokenizeResult dataclass returned by tokenization calls.
weaviate/tokenize/executor.py	Implements /tokenize and property tokenization requests + response parsing.
weaviate/tokenize/sync.py	Sync wrapper for the tokenize executor via @executor.wrap("sync").
weaviate/tokenize/async_.py	Async wrapper for the tokenize executor via @executor.wrap("async").
weaviate/tokenize/init.py	Exposes tokenize module symbols.
weaviate/client.py	Adds tokenize namespace to both sync and async clients.
weaviate/client.pyi	Updates type stubs to include tokenize attributes on clients.
weaviate/init.py	Exposes the tokenize module at the package root.
integration/test_tokenize.py	Integration coverage for sync/async tokenize calls and config handling.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 53.46939% with 114 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (dev/1.37@b9a7c69). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
integration/test_tokenize.py	46.47%	91 Missing ⚠️
weaviate/tokenization/executor.py	60.00%	10 Missing ⚠️
weaviate/tokenization/models.py	66.66%	8 Missing ⚠️
weaviate/collections/config/executor.py	44.44%	5 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             dev/1.37    #2012   +/-   ##
===========================================
  Coverage            ?   86.65%           
===========================================
  Files               ?      293           
  Lines               ?    22563           
  Branches            ?        0           
===========================================
  Hits                ?    19552           
  Misses              ?     3011           
  Partials            ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.