utils: do not require an IPv4 default route to find the default NIC - #14060
Open
wido wants to merge 1 commit into
Open
utils: do not require an IPv4 default route to find the default NIC#14060wido wants to merge 1 commit into
wido wants to merge 1 commit into
Conversation
A management server does not need an IPv4 default route. It can be
IPv6-first, or reach its POD network and KVM agents over IPv4 without
having a default gateway on that family at all.
getDefaultEthDevice() only looked at the IPv4 routing table and picked
column 5 of the route as the device name:
ip route show default 0.0.0.0/0 | head -1 | awk '{print $5}'
That has two problems:
* Anything the shell or 'ip' writes on stderr ends up in the captured
output (Script merges stderr into stdout), so the method can return a
string that is not a device name at all. getAllDefaultNicIps() then
passes it to NetworkInterface.getByName(), which returns null, and the
unchecked dereference throws a NullPointerException. On a host without
an IPv4 default route this aborts the boot of the management server:
ERROR [o.a.c.s.l.CloudStackExtendedLifeCycle] Error on configuring
bean RootCAProvider - Cannot invoke
"java.net.NetworkInterface.getInterfaceAddresses()" because "nic" is
null
at com.cloud.utils.net.NetUtils.getAllDefaultNicIps(NetUtils.java:298)
at o.a.c.ca.provider.RootCAProvider.loadManagementKeyStore(RootCAProvider.java:409)
ERROR [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] Failed to load
module [root-ca]
* Column 5 is only the device for routes of the form
"default via <gw> dev <name> ...". For an on-link default route such as
"default dev eth0 scope link" or "default dev eno1 proto kernel metric
256" it returns "link" or "kernel".
The device name is now taken from the token following "dev", which is
correct for every route layout including multipath routes, and the IPv6
routing table is consulted when there is no IPv4 default route. The
results of NetworkInterface.getByName() are null checked in both
getDefaultHostIp() and getAllDefaultNicIps() so an unresolvable device
name degrades to "no default NIC" instead of an exception.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #14060 +/- ##
=========================================
Coverage 19.77% 19.78%
- Complexity 19989 19997 +8
=========================================
Files 6371 6371
Lines 575899 575914 +15
Branches 70495 70499 +4
=========================================
+ Hits 113912 113953 +41
+ Misses 449563 449533 -30
- Partials 12424 12428 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A CloudStack management server does not need an IPv4 default route. It can be IPv6-first, or — as in the setup that triggered this — have IPv4 connectivity to the POD network and the KVM agents without any IPv4 default gateway. Today such a management server does not boot:
RootCAProvidercallsNetUtils.getAllDefaultNicIps()to collect the IPs that go into the management server certificate's SANs. That in turn callsNetUtils.getDefaultEthDevice(), which ran:Two problems with that:
It only looks at IPv4, and it does not always return a device name.
Scriptmerges stderr into stdout (ProcessBuilder.redirectErrorStream(true)), so anything the shell oripwrites on stderr becomes the "device name". That non-null, non-device string is handed toNetworkInterface.getByName(), which returnsnull, and the unchecked dereference on the next line throws the NPE above — which aborts the wholeroot-camodule and therefore the management server boot.Column 5 is not the device.
$5only happens to be the device for routes shaped likedefault via <gw> dev <name> .... For an on-link default route it is wrong:$5default via 10.0.0.1 dev eth0 proto static metric 100eth0eth0default dev eth0 scope linklink❌eth0default via fe80::1 dev eno1 proto ra metric 1024 expires 1798sec pref mediumeno1eno1default dev eno1 proto kernel metric 256 pref mediumkernel❌eno1default proto static+nexthop ... dev eth0 ...)eth0Fix
Small change, all in
NetUtils:dev, rather than from a fixed column. Correct for every route layout above, including multipath.ip -4 route show defaultis queried first; if there is no IPv4 default route,ip -6 route show defaultis used. An IPv6-first management server now resolves its default NIC and gets its addresses (both families) into the management server certificate, instead of getting nothing.NetworkInterface.getByName()are null-checked in bothgetAllDefaultNicIps()andgetDefaultHostIp(), so an unresolvable device name degrades to "no default NIC found" with a warning in the log instead of killing the boot.Types of changes
How Has This Been Tested?
Four unit tests added to
NetUtilsTest, covering the IPv4→IPv6 fallback, the IPv4-preferred path, an unresolvable device name, and no default route at all.The
awkexpression was checked against the route outputs in the table above.Note
NetUtilsTest#testAllIpsOfDefaultNicfails on JDK 26 withUnsupportedOperationExceptionfromCollections.reverse()ingetNetworkParams(), becauseNetworkInterface.getInterfaceAddresses()now returns an immutable list. That is pre-existing onmainand unrelated to this PR.