SOLR-18324: Fix NPE/RTimer assertion when a V2 request fails before a SolrQueryRequest is attached - #4676
Open
epugh wants to merge 3 commits into
Open
SOLR-18324: Fix NPE/RTimer assertion when a V2 request fails before a SolrQueryRequest is attached#4676epugh wants to merge 3 commits into
epugh wants to merge 3 commits into
Conversation
…Jersey request fails before a SolrQueryRequest is attached to the request context. Discovered while investigating an intermittent CollectionsApi.GetCollectionStatus failure under basic-auth-secured clusters. When a request fails early (before V2HttpCall attaches a SolrQueryRequest to the Jersey request context), CatchAllExceptionMapper.processAndRespondToException passed a null req into RequestHandlerBase.processReceivedException, which unconditionally called req.getCore(), throwing an NPE while already handling an exception. Jersey then re-invoked the response filter chain via its "already mapped exception" path, causing RequestMetricHandling.PostRequestMetricsFilter to call stop() a second time on an already-stopped timer, triggering RTimer's assertion (visible only with assertions enabled, e.g. in test runs). Fixes: - RequestHandlerBase.processReceivedException: null-check req before req.getCore(). - CatchAllExceptionMapper.buildExceptionResponse: null-check solrQueryRequest before solrQueryRequest.getParams(). - PostRequestDecorationFilter / PostRequestLoggingFilter: null-check solrQueryRequest before dereferencing it. - RequestMetricHandling.PostRequestMetricsFilter: guard against being invoked more than once for the same request (clears the TIMER property after stopping it). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…number. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s commit only renamed the file, not its contents). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5 tasks
dsmiley
reviewed
Jul 28, 2026
dsmiley
left a comment
Contributor
There was a problem hiding this comment.
didn't look at the tests much
| final SolrQueryRequest solrQueryRequest = | ||
| (SolrQueryRequest) requestContext.getProperty(SOLR_QUERY_REQUEST); | ||
| if (solrQueryRequest == null) { | ||
| log.debug("Skipping QTime assignment because no SolrQueryRequest was attached"); |
Contributor
There was a problem hiding this comment.
TBH... I have little faith in unit testing low level plumbing like this. IMO they even have negative value as it's yet another thing to change if we change the plumbing. It's better to accomplish the high level goals (like what led you to uncover this bug). My comment applies to all your tests here. Just because you write a line of code doesn't mean it needs a direct test.
Contributor
Author
|
Thanks @dsmiley for taking a look. Yeah, I don't know that I feel comfortable with saying "yes, I found a bug" and or even "yes, this is hte fix we want". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/SOLR-18324
Summary
(Supersedes #4674, which was accidentally auto-closed when its branch got renamed outside of GitHub's branch-rename API rather than through it — same content, just corrected to reference the actual JIRA ticket number, SOLR-18324, instead of the SOLR-18322 placeholder used in the original commit message.)
See the reproduction script that is attached to the JIRA issue that can be used to demonstrate the bug on
mainand it being fixed on this branch.Discovered while investigating an intermittent
CollectionsApi.GetCollectionStatusfailure under basic-auth-secured clusters (used while migratingbin/solr deleteoff ZooKeeper). The endpoint would fail intermittently with:...cascading into Jersey re-invoking the response filter chain via its "already mapped exception" fallback path, which in turn caused
RequestMetricHandling.PostRequestMetricsFilterto call.stop()a second time on an already-stopped timer — trippingRTimer'sassert state == STARTED || state == PAUSED(only visible with assertions enabled, e.g. under-eatest runs; silently harmless in production builds where the assert is compiled out).Root cause: when a request fails before
V2HttpCallattaches aSolrQueryRequestto the Jersey request context (the actual trigger appears to be a rare race in inter-node request handling — not fully root-caused, but independent of any specific endpoint), several Jersey filters/mappers inorg.apache.solr.jerseyunconditionally dereference that possibly-nullSolrQueryRequest, causing a second, cascading exception while trying to handle the first one.Changes
RequestHandlerBase.processReceivedException: null-checkreqbefore callingreq.getCore().CatchAllExceptionMapper.buildExceptionResponse: null-checksolrQueryRequestbefore calling.getParams().PostRequestDecorationFilter/PostRequestLoggingFilter: null-checksolrQueryRequestbefore dereferencing (mirrors the existing, correct null-check already present inMediaTypeOverridingFilter, which had a// TODO Is it valid for SQRequest to be null?comment confirming this was already a known possibility).RequestMetricHandling.PostRequestMetricsFilter: guards against being invoked more than once for the same request by clearing theTIMERproperty after stopping it, so a second (re-entrant) invocation is a no-op instead of crashing.Test plan
RequestHandlerBaseTest(nullreqcases),PostRequestLoggingFilterTest,PostRequestDecorationFilterTest(new),RequestMetricHandlingTest(new) — each verified to fail without the corresponding fix and pass with itCatchAllExceptionMapperTest(SOLR-18066) still passes — no regression tohideStackTracehandlingGetCollectionStatusunder aSecurityJson.SIMPLE-secured 2-node cluster (~25-50% failure rate over multiple runs); with these fixes applied, 20+ consecutive runs passed./gradlew :solr:core:spotlessJavaCheckclean./gradlew :solr:core:compileJava :solr:core:compileTestJavaclean🤖 Generated with Claude Code