Thanks for making this happen @daenney! If this does get accepted DTLS I would
love to join the maintainers.

IMO value is lost having it under the Pion organization. We have our own rules/culture that isn't benefiting
DTLS. The community we have right now is all around RTC, so it tends to drown out DTLS people.

I have also had a few people not interested in getting involved/contributing because it is under Pion. Maybe
people think that Pion is a company and not just an Open Source project?

CC @FiloSottile @katiehockman

cc @julieqiu

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

cc @FiloSottile @rolandshoemaker @katiehockman

cc @golang/security

Talked to the security team about this. It's a large effort, estimated to increase TLS maintenance burden by 20%, but it's also potentially important for a large fraction of the Go ecosystem. We are not staffed to take this on at the moment. But it sounds like the team behind pion/tls is entirely volunteer, which is not ideal either from a staffing perspective. The path ahead is unclear on this one.

Perhaps we can work out what the API would look like. Is it small, or is it big? What would it look like?

@rsc The API for pion/dtls is crypto/tls except it does a net.PacketConn instead of net.Conn. I think it would be good to do the same. crypto/tls is well designed and has done us a lot of good mimicking it.

Should I submit a proposal? @daenney and I would love to do that work if a well designed DTLS implementation has a high probability of getting merged. I am hesitant to take time away from other work if it is unlikely.

If you can just paste the API additions relative to crypto/tls into this thread, that would probably be the right next step. Thanks!

The following is a diff of go doc -all crypto/tls

This brings all the functionality that pion/dtls have today. The API isn't an exact match, in some places we made API mistakes. Below the diff I also summarized the features. @daenney @at-wat @bocajim @jkralik Anything I have missed/couldn't support your use case on?

@@ -49,6 +49,18 @@
 
     See https://www.iana.org/assignments/tls-parameters/tls-parameters.xml
 
+
+type ExtendedMasterSecretType int
+    ExtendedMasterSecretType declares the policy the client and server will
+    follow for the Extended Master Secret extension
+
+const (
+  RequestExtendedMasterSecret ExtendedMasterSecretType = iota
+  RequireExtendedMasterSecret
+  DisableExtendedMasterSecret
+)
+    ExtendedMasterSecretType enums
+
 const (
 	VersionTLS10 = 0x0301
 	VersionTLS11 = 0x0302
@@ -79,9 +91,36 @@
     must be non-nil and must include at least one certificate or else set
     GetCertificate.
 
+func Resume(state *State, conn net.PacketConn, config *Config) (*Conn, error)
+    Resume imports an already established dtls connection using a specific dtls
+    state
 
 TYPES
 
+type CipherSuite interface {
+  // String of CipherSuite, only used for logging
+  String() string
+
+  // ID of CipherSuite.
+  ID() CipherSuiteID
+
+  // What type of Certificate does this CipherSuite use
+  CertificateType() clientcertificate.Type
+
+  // What Hash function is used during verification
+  HashFunc() func() hash.Hash
+
+  // AuthenticationType controls what authentication method is using during the handshake
+  AuthenticationType() CipherSuiteAuthenticationType
+
+  // Called when keying material has been generated, should initialize the internal cipher
+  Init(masterSecret, clientRandom, serverRandom []byte, isClient bool) error
+  IsInitialized() bool
+  Encrypt(pkt *recordlayer.RecordLayer, raw []byte) ([]byte, error)
+  Decrypt(in []byte) ([]byte, error)
+}
+    CipherSuite is an interface that all DTLS CipherSuites must satisfy
+
 type Certificate struct {
 	Certificate [][]byte
 	// PrivateKey contains the private key corresponding to the public key in
@@ -246,10 +285,10 @@
 	// might be rejected if used.
 	SupportedVersions []uint16
 
-	// Conn is the underlying net.Conn for the connection. Do not read
+	// PacketConn is the underlying net.PacketConn for the connection. Do not read
 	// from, or write to, this connection; that will cause the TLS
 	// connection to fail.
-	Conn net.Conn
+	PacketConn net.PacketConn
 
 	// Has unexported fields.
 }
@@ -305,6 +344,15 @@
     sessions.
 
 type Config struct {
+  // CustomCipherSuites is a list of CipherSuites that can be
+  // provided by the user. This allow users to user Ciphers that are reserved
+  // for private usage.
+  CustomCipherSuites func() []CipherSuite
+
+  // ExtendedMasterSecret determines if the "Extended Master Secret" extension
+  // should be disabled, requested, or required (default requested).
+  ExtendedMasterSecret ExtendedMasterSecretType
+
 	// Rand provides the source of entropy for nonces and RSA blinding.
 	// If Rand is nil, TLS uses the cryptographic random reader in package
 	// crypto/rand.
@@ -504,6 +552,11 @@
 	// used for debugging.
 	KeyLogWriter io.Writer
 
+  // PSK sets the pre-shared key used by this DTLS connection
+  // If PSK is non-nil only PSK CipherSuites will be used
+  PSK             PSKCallback
+  PSKIdentityHint []byte
+
 	// Has unexported fields.
 }
     A Config structure is used to configure a TLS client or server. After one
@@ -541,10 +594,10 @@
 type Conn struct {
 	// Has unexported fields.
 }
-    A Conn represents a secured connection. It implements the net.Conn
+    A Conn represents a secured connection. It implements the net.PacketConn
     interface.
 
-func Client(conn net.Conn, config *Config) *Conn
+func Client(conn net.PacketConn, config *Config) *Conn
     Client returns a new TLS client side connection using conn as the underlying
     transport. The config cannot be nil: users must set either ServerName or
     InsecureSkipVerify in the config.
@@ -567,7 +620,7 @@
     DialWithDialer uses context.Background internally; to specify the context,
     use Dialer.DialContext with NetDialer set to the desired dialer.
 
-func Server(conn net.Conn, config *Config) *Conn
+func Server(conn net.PacketConn, config *Config) *Conn
     Server returns a new TLS server side connection using conn as the underlying
     transport. The configuration config must be non-nil and must include at
     least one certificate or else set GetCertificate.
@@ -612,6 +665,10 @@
     OCSPResponse returns the stapled OCSP response from the TLS server, if any.
     (Only valid for client connections.)
 
+func (c *Conn) SelectedSRTPProtectionProfile() (SRTPProtectionProfile, bool)
+    SelectedSRTPProtectionProfile returns the selected SRTP Protection Profile and if one
+		was succesfully negotiated.
+
 func (c *Conn) Read(b []byte) (int, error)
     Read reads data from the connection.
 
@@ -713,6 +770,12 @@
 	// RFC 7627, and https://mitls.org/pages/attacks/3SHAKE#channelbindings.
 	TLSUnique []byte
 
+	// IdentityHint which may be sent by the server for PSK
+  IdentityHint     []byte
+
+
+  SessionID        []byte
+
 	// Has unexported fields.
 }
     ConnectionState records basic TLS details about the connection.
@@ -723,6 +786,12 @@
     the seed. If the connection was set to allow renegotiation via
     Config.Renegotiation, this function will return an error.
 
+func (s *State) MarshalBinary() ([]byte, error)
+    MarshalBinary is a binary.BinaryMarshaler.MarshalBinary implementation
+
+func (s *State) UnmarshalBinary(data []byte) error
+    UnmarshalBinary is a binary.BinaryUnmarshaler.UnmarshalBinary implementation
+
 type CurveID uint16
     CurveID is the type of a TLS identifier for an elliptic curve. See
     https://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-8.
@@ -753,7 +822,7 @@
     Dialer dials TLS connections given a configuration and a Dialer for the
     underlying connection.
 
-func (d *Dialer) Dial(network, addr string) (net.Conn, error)
+func (d *Dialer) Dial(network, addr string) (net.PacketConn, error)
     Dial connects to the given network address and initiates a TLS handshake,
     returning the resulting TLS connection.
 
@@ -762,7 +831,7 @@
     Dial uses context.Background internally; to specify the context, use
     DialContext.
 
-func (d *Dialer) DialContext(ctx context.Context, network, addr string) (net.Conn, error)
+func (d *Dialer) DialContext(ctx context.Context, network, addr string) (net.PacketConn, error)
     DialContext connects to the given network address and initiates a TLS
     handshake, returning the resulting TLS connection.
 
@@ -778,11 +847,11 @@
 	// RecordHeader contains the five bytes of TLS record header that
 	// triggered the error.
 	RecordHeader [5]byte
-	// Conn provides the underlying net.Conn in the case that a client
+	// Conn provides the underlying net.PacketConn in the case that a client
 	// sent an initial handshake that didn't look like TLS.
 	// It is nil if there's already been a handshake or a TLS alert has
 	// been written to the connection.
-	Conn net.Conn
+	Conn net.PacketConn
 }
     RecordHeaderError is returned when a TLS record header is invalid.
 
@@ -845,3 +914,17 @@
 )
 func (i SignatureScheme) String() string
 
+type SRTPProtectionProfile uint16
+		SRTPProtectionProfile defines the parameters and options that are in effect
+		for the SRTP processing https://tools.ietf.org/html/rfc5764#section-4.1.2
+
+const (
+	SRTPProtectionProfile_AES128_CM_HMAC_SHA1_80 SRTPProtectionProfile = 0x0001
+	SRTPProtectionProfile_AES128_CM_HMAC_SHA1_32 SRTPProtectionProfile = 0x0002
+	SRTPProtectionProfile_AEAD_AES_128_GCM       SRTPProtectionProfile = 0x0007
+	SRTPProtectionProfile_AEAD_AES_256_GCM       SRTPProtectionProfile = 0x0008
+)
+
+type PSKCallback func([]byte) ([]byte, error)
+    PSKCallback is called once we have the remote's PSKIdentityHint. If the
+    remote provided none it will be nil

• Ability to require (or disable) extended master secret.
• PSK support
• Ability to implement Custom/Reserved Cipher Suite. Some IoT devices define their own suites/Needed for [CoAP](https://en.wikipedia.org/wiki/Constrained_Application_Protocol]
• SRTP Extension. Just finds a intersection of uint16 between client/server
• Use a net.PacketConn instead of net.Conn
• ConnectionState can be serialized/deserialized

Thanks for the API. Presumably those changed method signatures would become new methods, like PacketClient, etc.

@rolandshoemaker or anyone else, do these look like plausible API additions for crypto/tls to support DTLS?

@rolandshoemaker For a good first step I was hoping to use the existing crypto/tls implementation. I am totally happy with losing features if that means a better DTLS implementation.

We could have an additional layer that handles the DTLS Specific header for loss/reordering/fragmentation. Nothing in crypo/tls would need to change.

Here are some comments from my perspective... First, as background, I created github.com/qwerty-iot/dtls (formerly github.com/bocajim/dtls) which is used in several IoT platforms as the DTLS server. It was never intended to have any compatibility with crypto/tls, so it is not a good target for consideration, but it does identify some key requirements for the IoT community.

If I have to summarize the use case in a sentence, it would go like this: We require a solution for "millions of DTLS sessions that communicate once or twice per day that is compatible with high-availability deployments".

1. Pre-shared key support

2. Specifically the most common cipher for the standards we use is TLS_PSK_WITH_AES_128_CCM_8 (0xC0A8)

3. Ability to export/import DTLS sessions via callbacks. There are three main use cases:

• First is to age sessions from RAM to disk when not in use (think about 25M sessions used once per day).
• The second use case is exporting session information so it can be stored in a centralized cache such that devices can communicate with any server in a cluster without re-negotiating the session.
• Third is to export sessions at server shutdown and restore them on startup such that maintenance can be performed without requiring devices to re-handshake to the server.

4. Support for DTLS connection ID (https://tools.ietf.org/id/draft-ietf-tls-dtls-connection-id-02.html) (I'm not sure how other stacks are supporting this when the RFC has yet to be approved)

5. Limit use of any "per-session" go routines, again consider 25M active sessions with one packet per day, the memory requirements are too high if there are routines for each session blocked on reads.

We have two corporate sponsors for qwerty-iot/dtls, so we have funding to support a long-term solution, but it would need to align with our core requirements.

I also support improvements to crypto/tls to remove the amount of overlapping code in the current DTLS implementations.

Hey @bocajim I didn't realize pion/dtls didn't have the features you needed. Are you blocked on 5?

It might make more sense to keep things in userland if DTLS has intractable requirements by users?

@Sean-Der I can't say if my use case is mainstream or fringe, so I wouldn't want to scuttle efforts to standardize a DTLS implementation if my use case is outside of the core needs of developers. But given the high usage of DTLS within the IoT community, specifically the LPWA community, I think there will be more demand over time for a highly scalable stack.

The pion/dtls package is the most complete implementation in Go, and I tried to rewrite parts of it to maintain the same interface but scale in a less resource intensive way, but it became too much work for my team. The qwerty-iot/dtls library is missing lots of little elements of the DTLS RFC, but does check the box for high scalability, so maybe there is some work we can do to try to bring the two concepts together.

@FiloSottile This is a definite problem, once supported, general lifetime of a device could be as much as 10 years where compatibility needs to be maintained regardless of security concerns or advances. This is generally easy to manage by letting the server implementation choose which ciphers/extensions to support, so it shouldnt be a problem.

The hardest part to manage is that have somewhere between 3-5 hacks in my implementation for clients that do not adhere to the DTLS specification correctly, but the device vendors do not have the ability to update their clients, so we are stuck supporting their poor implementation on the server side. I don't see being able to do these types of hacks in the standard library.

IoT isn't the only use-case of DTLS as we've shown in the proposal. I'd like to be careful about overindexing on "IoT is turrible, we shouldn't have to deal with it in stdlib". There are many places where DTLS is used where we are not constrained by poor vendor implementations that can't be updated. Having an implementation in stdlib also doesn't preclude anyone maintaining their own with speciality hacks if their needs call for it.

Absolutely agree, just make sure we highlight the intended use cases and the expected gaps so there are no delusions about the applicability of the end result.

IoT isn't the only use-case of DTLS as we've shown in the proposal. I'd like to be careful about overindexing on "IoT is turrible, we shouldn't have to deal with it in stdlib". There are many places where DTLS is used where we are not constrained by poor vendor implementations that can't be updated. Having an implementation in stdlib also doesn't preclude anyone maintaining their own with speciality hacks if their needs call for it.

It sounds like maybe there's no clear reason (not enough usage) for the Go team to pick up and maintain this, and it sounds like people don't use pion/dtls because of concerns about maintainer responsiveness too. So it sounds like we're not going to solve that, and maybe the right answer is to keep using pion/dtls or leave space for someone else to step forward with more maintenance?

Very well. Lets leave it at that then.

This proposal has been declined as retracted.
— rsc for the proposal review group