Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: net/netip: add conversions between IP ranges and prefixes #65909

Open
gaissmai opened this issue Feb 23, 2024 · 8 comments
Open

proposal: net/netip: add conversions between IP ranges and prefixes #65909

gaissmai opened this issue Feb 23, 2024 · 8 comments
Labels
Milestone

Comments

@gaissmai
Copy link

Proposal Details

I suggest the following additional method/functions for the net/netip package:

func (p Prefix) Range() (first, last Addr)
Range returns the inclusive range of IP addresses that p covers.

func PrefixFromRange(first, last Addr) (prefix Prefix, ok bool)
PrefixFromRange returns the IP Prefix from first to last and ok=true, if it can be presented exactly as such.

func Prefixes(first, last Addr) []Prefix
Prefixes returns the set of Prefix entries that covers the IP range from first to last.

func AppendPrefixes(dst []Prefix, first, last Addr) []Prefix
AppendPrefixes is an append version of Prefixes.

Reason:
An effective calculation of the last IP address of a netip.Prefix is only possible with the private uint128 representation in the netip package. The same applies to the inverse conversion of an IP-range into a set of prefixes.

I explicitly do not propose another type IPRange as used in netipx, but only the minimum necessary to build third party packages with IP-ranges based on net/netip.

If these functions were in the stdlib, many packages could drop the dependency on github.com/go4org/netipx or github.com/gaissmai/extnetip.

see also: #61642 #53236

@gopherbot gopherbot added this to the Proposal milestone Feb 23, 2024
@ianlancetaylor
Copy link
Contributor

CC @neild @bradfitz @josharian

@earthboundkid
Copy link
Contributor

Prefixes could be an iter.Seq instead of a slice.

@thediveo
Copy link

Prefixes could be an iter.Seq instead of a slice.

why would you want to iterate over 2001::/112? and how long can you wait for this iteration to go on?

@gaissmai
Copy link
Author

gaissmai commented Feb 24, 2024

@thediveo @earthboundkid keep in mind that max len Prefixes(start, stop) == 254
for the worst case IP-range: ::1, ffff:ffff:ffff:ffff:ffff:ffff:ffff:fffe

Prefixes(start, stop) is no sequence or iteration over an IP prefix, its just a conversion from a range to a slice of netip.Prefix with max. 254 items:

"::1/128",
"::2/127",
"::4/126",
"::8/125",
"::10/124",
"::20/123",
"::40/122",
"::80/121",
"::100/120",
"::200/119",
"::400/118",
"::800/117",
"::1000/116",
"::2000/115",
"::4000/114",
"::8000/113",
"::1:0/112",
"::2:0/111",
"::4:0/110",
"::8:0/109",
"::10:0/108",
"::20:0/107",
"::40:0/106",
"::80:0/105",
"::100:0/104",
"::200:0/103",
"::400:0/102",
"::800:0/101",
"::1000:0/100",
"::2000:0/99",
"::4000:0/98",
"::8000:0/97",
"::1:0:0/96",
"::2:0:0/95",
"::4:0:0/94",
"::8:0:0/93",
"::10:0:0/92",
"::20:0:0/91",
"::40:0:0/90",
"::80:0:0/89",
"::100:0:0/88",
"::200:0:0/87",
"::400:0:0/86",
"::800:0:0/85",
"::1000:0:0/84",
"::2000:0:0/83",
"::4000:0:0/82",
"::8000:0:0/81",
"::1:0:0:0/80",
"::2:0:0:0/79",
"::4:0:0:0/78",
"::8:0:0:0/77",
"::10:0:0:0/76",
"::20:0:0:0/75",
"::40:0:0:0/74",
"::80:0:0:0/73",
"::100:0:0:0/72",
"::200:0:0:0/71",
"::400:0:0:0/70",
"::800:0:0:0/69",
"::1000:0:0:0/68",
"::2000:0:0:0/67",
"::4000:0:0:0/66",
"::8000:0:0:0/65",
"0:0:0:1::/64",
"0:0:0:2::/63",
"0:0:0:4::/62",
"0:0:0:8::/61",
"0:0:0:10::/60",
"0:0:0:20::/59",
"0:0:0:40::/58",
"0:0:0:80::/57",
"0:0:0:100::/56",
"0:0:0:200::/55",
"0:0:0:400::/54",
"0:0:0:800::/53",
"0:0:0:1000::/52",
"0:0:0:2000::/51",
"0:0:0:4000::/50",
"0:0:0:8000::/49",
"0:0:1::/48",
"0:0:2::/47",
"0:0:4::/46",
"0:0:8::/45",
"0:0:10::/44",
"0:0:20::/43",
"0:0:40::/42",
"0:0:80::/41",
"0:0:100::/40",
"0:0:200::/39",
"0:0:400::/38",
"0:0:800::/37",
"0:0:1000::/36",
"0:0:2000::/35",
"0:0:4000::/34",
"0:0:8000::/33",
"0:1::/32",
"0:2::/31",
"0:4::/30",
"0:8::/29",
"0:10::/28",
"0:20::/27",
"0:40::/26",
"0:80::/25",
"0:100::/24",
"0:200::/23",
"0:400::/22",
"0:800::/21",
"0:1000::/20",
"0:2000::/19",
"0:4000::/18",
"0:8000::/17",
"1::/16",
"2::/15",
"4::/14",
"8::/13",
"10::/12",
"20::/11",
"40::/10",
"80::/9",
"100::/8",
"200::/7",
"400::/6",
"800::/5",
"1000::/4",
"2000::/3",
"4000::/2",
"8000::/2",
"c000::/3",
"e000::/4",
"f000::/5",
"f800::/6",
"fc00::/7",
"fe00::/8",
"ff00::/9",
"ff80::/10",
"ffc0::/11",
"ffe0::/12",
"fff0::/13",
"fff8::/14",
"fffc::/15",
"fffe::/16",
"ffff::/17",
"ffff:8000::/18",
"ffff:c000::/19",
"ffff:e000::/20",
"ffff:f000::/21",
"ffff:f800::/22",
"ffff:fc00::/23",
"ffff:fe00::/24",
"ffff:ff00::/25",
"ffff:ff80::/26",
"ffff:ffc0::/27",
"ffff:ffe0::/28",
"ffff:fff0::/29",
"ffff:fff8::/30",
"ffff:fffc::/31",
"ffff:fffe::/32",
"ffff:ffff::/33",
"ffff:ffff:8000::/34",
"ffff:ffff:c000::/35",
"ffff:ffff:e000::/36",
"ffff:ffff:f000::/37",
"ffff:ffff:f800::/38",
"ffff:ffff:fc00::/39",
"ffff:ffff:fe00::/40",
"ffff:ffff:ff00::/41",
"ffff:ffff:ff80::/42",
"ffff:ffff:ffc0::/43",
"ffff:ffff:ffe0::/44",
"ffff:ffff:fff0::/45",
"ffff:ffff:fff8::/46",
"ffff:ffff:fffc::/47",
"ffff:ffff:fffe::/48",
"ffff:ffff:ffff::/49",
"ffff:ffff:ffff:8000::/50",
"ffff:ffff:ffff:c000::/51",
"ffff:ffff:ffff:e000::/52",
"ffff:ffff:ffff:f000::/53",
"ffff:ffff:ffff:f800::/54",
"ffff:ffff:ffff:fc00::/55",
"ffff:ffff:ffff:fe00::/56",
"ffff:ffff:ffff:ff00::/57",
"ffff:ffff:ffff:ff80::/58",
"ffff:ffff:ffff:ffc0::/59",
"ffff:ffff:ffff:ffe0::/60",
"ffff:ffff:ffff:fff0::/61",
"ffff:ffff:ffff:fff8::/62",
"ffff:ffff:ffff:fffc::/63",
"ffff:ffff:ffff:fffe::/64",
"ffff:ffff:ffff:ffff::/65",
"ffff:ffff:ffff:ffff:8000::/66",
"ffff:ffff:ffff:ffff:c000::/67",
"ffff:ffff:ffff:ffff:e000::/68",
"ffff:ffff:ffff:ffff:f000::/69",
"ffff:ffff:ffff:ffff:f800::/70",
"ffff:ffff:ffff:ffff:fc00::/71",
"ffff:ffff:ffff:ffff:fe00::/72",
"ffff:ffff:ffff:ffff:ff00::/73",
"ffff:ffff:ffff:ffff:ff80::/74",
"ffff:ffff:ffff:ffff:ffc0::/75",
"ffff:ffff:ffff:ffff:ffe0::/76",
"ffff:ffff:ffff:ffff:fff0::/77",
"ffff:ffff:ffff:ffff:fff8::/78",
"ffff:ffff:ffff:ffff:fffc::/79",
"ffff:ffff:ffff:ffff:fffe::/80",
"ffff:ffff:ffff:ffff:ffff::/81",
"ffff:ffff:ffff:ffff:ffff:8000::/82",
"ffff:ffff:ffff:ffff:ffff:c000::/83",
"ffff:ffff:ffff:ffff:ffff:e000::/84",
"ffff:ffff:ffff:ffff:ffff:f000::/85",
"ffff:ffff:ffff:ffff:ffff:f800::/86",
"ffff:ffff:ffff:ffff:ffff:fc00::/87",
"ffff:ffff:ffff:ffff:ffff:fe00::/88",
"ffff:ffff:ffff:ffff:ffff:ff00::/89",
"ffff:ffff:ffff:ffff:ffff:ff80::/90",
"ffff:ffff:ffff:ffff:ffff:ffc0::/91",
"ffff:ffff:ffff:ffff:ffff:ffe0::/92",
"ffff:ffff:ffff:ffff:ffff:fff0::/93",
"ffff:ffff:ffff:ffff:ffff:fff8::/94",
"ffff:ffff:ffff:ffff:ffff:fffc::/95",
"ffff:ffff:ffff:ffff:ffff:fffe::/96",
"ffff:ffff:ffff:ffff:ffff:ffff::/97",
"ffff:ffff:ffff:ffff:ffff:ffff:8000:0/98",
"ffff:ffff:ffff:ffff:ffff:ffff:c000:0/99",
"ffff:ffff:ffff:ffff:ffff:ffff:e000:0/100",
"ffff:ffff:ffff:ffff:ffff:ffff:f000:0/101",
"ffff:ffff:ffff:ffff:ffff:ffff:f800:0/102",
"ffff:ffff:ffff:ffff:ffff:ffff:fc00:0/103",
"ffff:ffff:ffff:ffff:ffff:ffff:fe00:0/104",
"ffff:ffff:ffff:ffff:ffff:ffff:ff00:0/105",
"ffff:ffff:ffff:ffff:ffff:ffff:ff80:0/106",
"ffff:ffff:ffff:ffff:ffff:ffff:ffc0:0/107",
"ffff:ffff:ffff:ffff:ffff:ffff:ffe0:0/108",
"ffff:ffff:ffff:ffff:ffff:ffff:fff0:0/109",
"ffff:ffff:ffff:ffff:ffff:ffff:fff8:0/110",
"ffff:ffff:ffff:ffff:ffff:ffff:fffc:0/111",
"ffff:ffff:ffff:ffff:ffff:ffff:fffe:0/112",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:0/113",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:8000/114",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:c000/115",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:e000/116",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:f000/117",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:f800/118",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:fc00/119",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:fe00/120",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:ff00/121",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:ff80/122",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffc0/123",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffe0/124",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:fff0/125",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:fff8/126",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:fffc/127",
"ffff:ffff:ffff:ffff:ffff:ffff:ffff:fffe/128",

@seankhliao
Copy link
Member

what's the difference with the previously declined #53236 ?

@gaissmai
Copy link
Author

In 2022 netip was brand new in stdlib with few users and the attention for this proposal was non-existent. In the meantime, some projects have switched from net/IP to net/netip, but often still have to include netipx or extnetip for this trivial functionality.

And for 1.22 the window was opened for two other much less important compare methods, which do not bring any essential functionality but only convenience, in contrast this proposal is essential for a performant calculation between IP prefixes and IP ranges, which you need as a networker every day.

IP filter lists are often not based on prefixes but on ranges, but ranges cannot be implemented performantly in routers, for this you need prefixes (longest-common-prefix-match), so you have to be able to convert ranges into prefixes quickly and vice versa.

@seankhliao seankhliao changed the title proposal: net/netip: add a few basic functions/methods for converting IP prefixes to IP ranges and vice versa proposal: net/netip: add conversions between IP ranges and prefixes Feb 24, 2024
@josharian
Copy link
Contributor

In the meantime, some projects have switched from net/IP to net/netip, but often still have to include netipx or extnetip for this trivial functionality.

Data would help a lot here. E.g. what percent of projects that import net/netip also import netipx or extnetip? Of those that do, what APIs from netipx/extnetip do they use?

@gaissmai
Copy link
Author

@josharian the pure statistics speak against my suggestion, net/netip is imported by 3590 go4org/netipx out of 135, according to pkg.go.dev

But please keep in mind that the limit of supported functionality of net/netip is not well chosen, because also go4org/netipx only needs the whole uint128 implementation analogous to net/netip for the conversion functions from prefix to range and vice versa.

Maybe @danderson or @bradfitz as authors of netip and netipx can comment on this.

Additionally, please keep in mind that IP libraries in other languages support exactly these conversions, because this is tricky for the normal user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Incoming
Development

No branches or pull requests

7 participants