Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: driver unexpectedly closing connections that should be returned to the pool #28495

Closed
rohancme opened this issue Oct 30, 2018 · 3 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@rohancme
Copy link

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go 1.11

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Ubuntu 16.04.5 LTS
GOARCH="amd64"

What did you do?

I have 2 go servers, one of which talks to a SQL Server RDS instance, and another that talks to a managed SQL Server instance in Azure.

I believe there is a slight difference in the way the 2 backends work - RDS has a single port (1433) on which the client authenticates and subsequently establishes the connection. Azure SQL seems to authenticate on port 1433 and then redirect the client to another service that actually handles the connections.

In both cases I've got substantial load running against the servers. At least 500 requests/s, with peaks of about 2k req/s. Each of these requests results in a Select query which returns a single row (that I call .Scan() on) with a primary key lookup - so really short lived connections to SQL. The average time per query is 50-80ms on both, with p95 in the 100-150ms range

I'm using the go database/sql driver with an MS-SQL implementation (Specifically go-mssqldb).

I've set Max Idle connections and Max Open connections to 64. I have NOT set MaxConnLifetime.

The Connection Pool was initialized using:

        db, err := sql.Open("sqlserver", connString)
	if err != nil {
		return nil, err
	}
	db.SetMaxIdleConns(64)
	db.SetMaxOpenConns(64)

What did you expect to see?

64 long running Established connections to the database that are occasionally idle but quickly reused.

What did you see instead?

Generally 64 Established connections to the database, with the number often dropping down to somewhere between 50 and 64. This also results in 200-400 connections to the database in the TIME_WAIT state at any given time.

@agnivade
Copy link
Contributor

The functions set an upper bound to the max no. of connections. It does not say anything about the minimum no. of connections. Maybe this is expected ? @kardianos

@agnivade agnivade changed the title database/sql driver unexpectedly closing connections that should be returned to the pool database/sql: driver unexpectedly closing connections that should be returned to the pool Oct 31, 2018
@katiehockman katiehockman added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 31, 2018
@kardianos
Copy link
Contributor

@rchakra3 As to why there are sometimes 50 open connections, I would imagine sometimes an error is returned and or something times out and the connection is dropped.

database/sql, unlike other pools, connects to the database lazily and eagerly reuses connection. So that part makes sense.

I'm not sure why you have 200-400 connections in the TIME_WAIT state.

@rohancme
Copy link
Author

rohancme commented Nov 6, 2018

@kardianos @agnivade thanks for the details!
After digging in further, I think this might be specific to the sql client implementation I'm using. I've created an issue in that repo to see if anyone has any ideas. If not I'll dig in further.

I'm closing this issue until I'm sure the client implementation isn't causing this behavior.

@rohancme rohancme closed this as completed Nov 6, 2018
@golang golang locked and limited conversation to collaborators Nov 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants