Skip to content

Add Netty handler to log HTTP wire traffic - #237

Merged
andsel merged 12 commits into
logstash-plugins:mainfrom
andsel:feature/log_http_wire_traffic
Aug 20, 2026
Merged

Add Netty handler to log HTTP wire traffic#237
andsel merged 12 commits into
logstash-plugins:mainfrom
andsel:feature/log_http_wire_traffic

Conversation

@andsel

@andsel andsel commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Release notes

Implements a functionality to log HTTP traffic. It's enabled by enabling debug log.

What does this PR do?

  • Created new WireLogger Netty handler to log information about channel opening/closing and dumping read and write traffic.
  • Truncate the dumped content to 4KB or can be customised by setting logstash.httpinput.wire.dump.size Java system property.
  • Updated client channel initializer to include such handler.
  • Updates Java target and source compatibility to use HttpClient provided by JDK standard library since version 11.

Why is it important/What is the impact to the user?

As a user that want's to debug his data flow I need to be able to track what's effectively sent to the HTTP input socket.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • [ ]

How to test this PR locally

Run a Logstash pipeline which includes an HTTP input filter enabling the logger in config/log4j2.properties with:

logger.httpwire.name = org.logstash.plugins.inputs.http.util.WireLogger
logger.httpwire.level = DEBUG

Related issues

Use cases

Given some issue with the messages received from an HTTP input I want to enable debug logging to check what's in transit on the TCP level, after the TLS layer.

Logs

Produced logs for gzipped content:

[2026-08-18T09:16:07,087][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] Opening connection /127.0.0.1:50455
[2026-08-18T09:16:07,091][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] << /127.0.0.1:50455 : POST / HTTP/1.1[\r][\n]Content-Type: application/json[\r][\n]Content-Encoding: gzip[\r][\n]Accept-Encoding: gzip, x-gzip, deflate[\r][\n]Host: localhost:8080[\r][\n]Content-Length: 40[\r][\n]Connection: keep-alive[\r][\n]User-Agent: Apache-HttpClient/5.4.3 (Java/21.0.10)[\r][\n][\r][\n][0x1F][0x8B][0x08][0x00][0x00][0x00][0x00][0x00][0x00][0xFF][0xAB]V[0xCA]M-.NLOU[0xB2]RP[0xCA]H[0xCD][0xC9][0xC9]W[0xAA][0x05][0x00]![0xC3][0x0E][0x9F][0x14][0x00][0x00][0x00]
[2026-08-18T09:16:07,111][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] >> /127.0.0.1:50455 : HTTP/1.1 200 OK[\r][\n]content-length: 2[\r][\n]content-type: text/plain[\r][\n][\r][\n]ok
[2026-08-18T09:16:07,115][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] Closing connection /127.0.0.1:50455

Produced logs for plain content:

[2026-08-18T09:17:05,264][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] Opening connection /127.0.0.1:50466
[2026-08-18T09:17:05,266][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] << /127.0.0.1:50466 : POST / HTTP/1.1[\r][\n]Content-Type: application/json[\r][\n]Accept-Encoding: gzip, x-gzip, deflate[\r][\n]Host: localhost:8080[\r][\n]Content-Length: 20[\r][\n]Connection: keep-alive[\r][\n]User-Agent: Apache-HttpClient/5.4.3 (Java/21.0.10)[\r][\n][\r][\n]{"message": "hello"}
[2026-08-18T09:17:05,633][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] >> /127.0.0.1:50466 : HTTP/1.1 200 OK[\r][\n]content-length: 2[\r][\n]content-type: text/plain[\r][\n][\r][\n]ok
[2026-08-18T09:17:05,641][DEBUG][org.logstash.plugins.inputs.http.util.WireLogger][main] Closing connection /127.0.0.1:50466

@andsel andsel self-assigned this Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Changelog and Version Management

No changelog or version changes detected. You can either update them manually or use a comment command to update them automatically on merge:

/pr_prepare release:<patch|minor|major> — bump version + update changelog
/pr_prepare changelog:add — changelog entry only (no version bump)

/pr_prepare release:patch
- Add the changelog entry here
  - (Additional details can be provided in multiple lines, respecting markdown formatting)

Omit the entry line to use the PR title. If multiple commands are posted, the last one wins.

@andsel andsel changed the title Feature/log http wire traffic Add Netty handler to log HTTP wire traffic Aug 18, 2026
@andsel
andsel marked this pull request as ready for review August 18, 2026 14:14

@donoghuc donoghuc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good and works as advertised. Have a couple small suggestions.

Also, will this be a new minor release?

Comment thread src/main/java/org/logstash/plugins/inputs/http/HttpInitializer.java
Comment thread docs/index.asciidoc Outdated
Comment thread docs/index.asciidoc
andsel and others added 2 commits August 19, 2026 09:35
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
@andsel
andsel requested a review from donoghuc August 19, 2026 08:27
@andsel

andsel commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

Also, will this be a new minor release?

Given that's a new feature, I think it has to be a new minor.

Comment thread docs/index.asciidoc Outdated

@donoghuc donoghuc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super small typo other than that this is g2g :shipit: in a minor release.

Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
@andsel

andsel commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

/pr_prepare release:minor

  • Implements a functionality to log HTTP traffic, disabled by default. To unable switch on debug logs.

@andsel
andsel merged commit 3585c44 into logstash-plugins:main Aug 20, 2026
11 checks passed
github-actions Bot added a commit that referenced this pull request Aug 20, 2026
andsel added a commit that referenced this pull request Aug 20, 2026
Implements a functionality to log HTTP traffic. It's enabled by enabling debug log.

As a user that want's to debug his data flow I need to be able to track what's effectively sent to the HTTP input socket.

- Created new WireLogger Netty handler to log information about channel opening/closing and dumping read and write traffic.
- Truncate the dumped content to 4KB or can be customised by setting `logstash.httpinput.wire.dump.size` Java system property.
- Updated client channel initializer to include such handler.
- Updates Java target and source compatibility to use HttpClient provided by JDK standard library since version 11.

Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>

* Bumped 3.11.0

---------

Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Log wire traffic

2 participants