Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: x/net/ethernet: new package for ethernet sockets #15985

Closed
mdlayher opened this issue Jun 7, 2016 · 17 comments
Closed

Proposal: x/net/ethernet: new package for ethernet sockets #15985

mdlayher opened this issue Jun 7, 2016 · 17 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal Proposal-Hold
Milestone

Comments

@mdlayher
Copy link
Member

mdlayher commented Jun 7, 2016

At this time, no official x/net package is available to create and use ethernet sockets in Go. I have written a couple of packages which can be used for this purpose, but I'd like to try to create an x/net/ethernet package as an official Go x/net subproject.

My packages can be found at:

Ethernet sockets are useful for low-level networking applications like ARP, LLDP, ATA over Ethernet, and a wide variety of other networking protocols. This package would make use of Linux's raw sockets and the BSD family's BPF to enable this functionality, and I already have working examples for these operating systems in my raw package. Adding support for other operating systems should be possible over time, but I have no personal experience working with ethernet sockets outside of Linux and BSD.

The proposed API could look something like:

package ethernet

// An Ethernet frame, which the Conn and RawConn types marshal
// and unmarshal to transport data
type Frame struct {
    // Contains usual Ethernet fields
}

// net.Addr implementation which indicates source or destination
// hardware address of Ethernet frame
type Addr struct {
    HardwareAddr net.HardwareAddr
}
func (a *Addr) Network() string
func (a *Addr) String() string

// Protocol identifier used to filter traffic by EtherType
type EtherType uint16

// A net.PacketConn which wraps and unwraps bytes from ethernet
// frames seamlessly.
type Conn struct {}
func Listen(ifi *net.Interface, et EtherType) (*Conn, error)
func (c *Conn) Close() error
func (c *Conn) LocalAddr() net.Addr
func (c *Conn) ReadFrom(b []byte) (int, net.Addr, error)
func (c *Conn) SetBPF(filter []bpf.RawInstruction) error
func (c *Conn) SetDeadline(t time.Time) error
func (c *Conn) SetReadDeadline(t time.Time) error
func (c *Conn) SetWriteDeadline(t time.Time) error
func (c *Conn) WriteTo(b []byte, addr net.Addr) (int, error)

// A modified Conn which returns all Ethernet frame data on incoming
// frames and enables writing arbitrary outgoing frames.
type RawConn struct{}
func ListenRaw(ifi *net.Interface, et EtherType) (*RawConn, error)
func (c *RawConn) Close() error
func (c *RawConn) ReadFrom(b []byte) (f *Frame, p []byte, error)
func (c *RawConn) SetBPF(filter []bpf.RawInstruction) error
func (c *RawConn) SetDeadline(t time.Time) error
func (c *RawConn) SetReadDeadline(t time.Time) error
func (c *RawConn) SetWriteDeadline(t time.Time) error
func (c *RawConn) WriteTo(f *Frame, b []byte) error

I am curious if 802.11Q VLAN tags should be included in this package. In general, the operating system seamlessly handles adding and removing VLAN tags as needed, so it may not be necessary, but could also be nice to have.

In addition, since the runtime network poller is not accessible outside of package net at the moment, the SetDeadline methods may just return an error until implemented later. See my other proposal at #15984 .

If implemented, this package would fix #8432.

Thanks for your time, and please feel free to ask for clarification if needed.

@mdlayher
Copy link
Member Author

mdlayher commented Jun 7, 2016

/cc @mikioh

@mdlayher
Copy link
Member Author

Ping, @mikioh .

@quentinmit quentinmit added this to the Proposal milestone Jun 17, 2016
@mikioh
Copy link
Contributor

mikioh commented Jun 23, 2016

Thank you for your proposal. A few random thoughts.

  • Perhaps datalink might be a better package name
    • I'm not keen on providing Ethernet marshaler/unmarshaler
  • It's okay that this package focuses on a bit slow BPF/LSF and AF_PACKET socket IO
    • Having a separate package that implements Netmap/DPDK-like faster stuff sounds reasonable
  • s/Conn/PacketConn/
  • It's fine to have an internal kqueue/epoll code fragments
  • Probably we need a type that can be used to configure BPF/LSF files and AF_PACKET sockets; maybe 'type Listener struct { }`?

A simple question: What's the reason why we need to provide RawConn on Linux?

@mdlayher
Copy link
Member Author

  • Perhaps datalink might be a better package name
    • I'm not keen on providing Ethernet marshaler/unmarshaler

I'd be okay with making this package called datalink, but since I assume most folks would use it for ethernet sockets anyway, I figured I would go for the more common use case. If this package were to be a raw datalink package, should another package be created to leverage it with ethernet sockets (x/net/ethernet or maybe even x/net/datalink/ethernet)?

  • It's okay that this package focuses on a bit slow BPF/LSF and AF_PACKET socket IO
    • Having a separate package that implements Netmap/DPDK-like faster stuff sounds reasonable

I'm not familiar with Netmap or DPDK, but perhaps they could be cleanly integrated into this package at some point too.

  • Probably we need a type that can be used to configure BPF/LSF files and AF_PACKET sockets; maybe 'type Listener struct { }`?

Sure, but what type of options would need to be configured? Would this be things like the ability to set the socket to nonblocking, or change the direction of traffic to capture using BPF?

A simple question: What's the reason why we need to provide RawConn on Linux?

Maybe we don't. My intention for the RawConn type was to enable writing arbitrary ethernet frames, but perhaps it isn't needed after all. If this package only focuses on "datalink" sockets, then this functionality is totally unneeded.

@mikioh
Copy link
Contributor

mikioh commented Jun 24, 2016

I think that providing separated datalink IO and datalink wire-format/coding packages makes sense. Because the former needs to work on abstraction of various kernel system calls and data types, and the latter should take a burden of various wire-formats/codings come from IEEE wildcats. For example,

package datalink

type PacketConn struct { /* TBD */ } // implements net.PacketConn interface and a few fancy datalink IO specific methods
type Listener struct {} // or Datalink or Config or something else
func (ln Listener) Listen(ifi *net.Inteface) (PacketConn, error)

--

package ether // or ethernet

type Frame interface { Tag; Stack }
type Tag interface { Len() int; Marshal() ([]byte, error) } 
type Stack interface { Lower() Tag; Upper() Tag }

type Dot3 struct {}
  :
type Dot1ad struct {}

If this package were to be a raw datalink package, should another package be created to leverage it with ethernet sockets (x/net/ethernet or maybe even x/net/datalink/ethernet)?

For the package that provides Ethernet encoding/decoding functionality, all of us know a variety of MAC/LLC sublayers from DIX/dot3 through dot22, and dot1 higher layers including MAC bridging through congestion notification. Ah, I have no strong opinion on it. Probably we can use github.com/google/gopacket package.

Sure, but what type of options would need to be configured?

There are a few options that should be configured before calling bind on AF_PACKET sockets, I guess.

@bradfitz
Copy link
Contributor

Assigning to @mikioh to make a decision.

@bradfitz bradfitz added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Aug 22, 2016
@joshlf
Copy link

joshlf commented Oct 1, 2016

A simple question: What's the reason why we need to provide RawConn on Linux?

I just found this proposal because I was searching for a way to read and emit raw ethernet frames in Go. My intended project is a "universal auto-join" tool which is used in security testing settings to join any network, regardless of configuration (presence of DHCP, MAC filtering, etc). I need to have direct access to raw frames for this project to do things like packet sniffing, ARP spoofing, manually speaking DHCP, etc.

@mdlayher
Copy link
Member Author

mdlayher commented Oct 3, 2016

@joshlf as an interim solution, you can use my packages:

@joshlf
Copy link

joshlf commented Oct 3, 2016

@mdlayher Thanks, I will :)

While we're on the subject - if this proposal ends up getting adopted, I'd suggest two additions from your raw package:

  • Be able to create a connection which returns frames of any protocol/ethertype
  • Be able to listen for ethernet frames not destined to the current host or to the broadcast MAC address (ie, promiscuous mode)

@mdlayher
Copy link
Member Author

mdlayher commented Oct 4, 2016

Hey @mikioh :

package datalink

type PacketConn struct { /* TBD */ } // implements net.PacketConn interface and a few fancy datalink IO specific methods
type Listener struct {} // or Datalink or Config or something else
func (ln Listener) Listen(ifi *net.Inteface) (PacketConn, error)

^ I'd be happy to look into implementing your proposed API here for a x/net/datalink. Does that work for you?

Looks like it's up to you to accept the proposal, if I understand the process correctly.

We can leave ethernet and things that run on top of x/net/datalink for later.

@joshlf
Copy link

joshlf commented Oct 4, 2016

func (ln Listener) Listen(ifi *net.Inteface) (PacketConn, error)

Why have a Listener type? Why not just a Listen function at the top level?

@freddierice
Copy link

Why have a Listener type? Why not just a Listen function at the top level?

The Listener type could hold socket options to be set during the Listen call. For example, Listener could have a Promiscuous bool, where the a call to Listen would only succeed on a successful call to setsockopt(sockfd, SOL_PACKET, PACKET_ADD_MEMBERSHIP, ...) with mr_type=PACKET_MR_PROMISC (for the linux implementation).

@joshlf
Copy link

joshlf commented Oct 7, 2016

@freddierice That's a good point; I suppose if you had a top-level Listen function, it'd still have to create a listening socket under the hood (at least on Unix).

@mikioh
Copy link
Contributor

mikioh commented Oct 18, 2016

@mdlayher,

Sorry for being late. I'd like to postpone decision making to follow the consensus of #17244. Also we need to work on #15984 first.

@mikioh mikioh removed their assignment Oct 18, 2016
@rsc
Copy link
Contributor

rsc commented Dec 19, 2016

On hold for #17244.

@mdlayher
Copy link
Member Author

At this point in time, I don't think it's necessary for this functionality to live under x/net; especially since it isn't needed in the standard library or similar. I'd be okay with closing this out, unless there are strong objections to doing so.

@mikioh
Copy link
Contributor

mikioh commented Mar 20, 2019

Punch it, please!

@golang golang locked and limited conversation to collaborators Mar 19, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal Proposal-Hold
Projects
None yet
Development

No branches or pull requests

8 participants