Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: Scanner.Scan requires bytes to be copied #24492

Closed
awreece opened this issue Mar 22, 2018 · 6 comments
Closed

database/sql: Scanner.Scan requires bytes to be copied #24492

awreece opened this issue Mar 22, 2018 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@awreece
Copy link
Contributor

awreece commented Mar 22, 2018

Its possible this is a doc change to sql.Scanner.Scan to reflect the expectations of sql.Scanner.Scan; it would be slightly nicer to copy src byte slices before passing them to sql.Scanner.Scan to avoid such surprises.

What version of Go are you using (go version)?

go version go1.10 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

package proto_test

import (
	"database/sql"
	"os"
	"reflect"
	"testing"

	_ "github.com/lib/pq"
)

func (w wrapper) Scan(src interface{}) error {
	srcBytes := src.([]byte)
	*w.dst = srcBytes
	return nil
}

type wrapper struct {
	dst *[]byte
}

func TestWrapper(t *testing.T) {
	dsn, ok := os.LookupEnv("DATABASE_URL")
	if !ok {
		t.Skip("no DATABASE_URL")
	}

	db, err := sql.Open("postgres", dsn)
	if err != nil {
		t.Fatal("database error", err)
	}

	var want = []byte{0x41, 0x42, 0x43}
	var got []byte

	var wrapped = wrapper{dst: &got}
	if err := db.QueryRow(`select $1::bytea`, &want).Scan(wrapped); err != nil {
		t.Error("select", err)
	}
	if !reflect.DeepEqual(got, want) {
		t.Errorf("got = %#v, want = %#v", got, want)
	}
}

What did you expect to see?

I expected the test to pass; from the docs on sql.Scanner, I assumed I could just assign the source to my destination. Digging a bit deeper, it looks like convertAssign makes a deep copy of dest via copyBytes when src is a primitive; but it doesn't do so when using the Scanner interface:

	if scanner, ok := dest.(Scanner); ok {
		return scanner.Scan(src)
	}

https://golang.org/src/database/sql/convert.go#L340

My guess is this violates the expectations of Rows.Scan`:

If a dest argument has type *[]byte, Scan saves in that argument a copy of the corresponding data. The copy is owned by the caller and can be modified and held indefinitely. The copy can be avoided by using an argument of type *RawBytes instead; see the documentation for RawBytes for restrictions on its use.

What did you see instead?

=== RUN   TestWrapper
--- FAIL: TestWrapper (0.01s)
	proto_test.go:41: got = []byte{0x20, 0x31, 0x0}, want = []byte{0x41, 0x42, 0x43}
FAIL
@awreece
Copy link
Contributor Author

awreece commented Mar 22, 2018

This is a potential fix, I'm happy to put up a PR if we like it.

 diff --git a/src/database/sql/convert.go b/src/database/sql/convert.go
index b79ec3f7b2..75388a3955 100644
--- a/src/database/sql/convert.go
+++ b/src/database/sql/convert.go
@@ -338,6 +338,9 @@ func convertAssign(dest, src interface{}) error {
        }
 
        if scanner, ok := dest.(Scanner); ok {
+               if b, ok := src.([]byte); ok {
+                       src = cloneBytes(src)
+               }
                return scanner.Scan(src)
        }

This adopts the approach of making the code behave the same as the later cases, e.g.:

	dv := reflect.Indirect(dpv)
	if sv.IsValid() && sv.Type().AssignableTo(dv.Type()) {
		switch b := src.(type) {
		case []byte:
			dv.Set(reflect.ValueOf(cloneBytes(b)))
		default:
			dv.Set(sv)
		}
		return nil
	}

@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 22, 2018
@dgryski
Copy link
Contributor

dgryski commented Mar 22, 2018

/cc @kardianos

@kardianos
Copy link
Contributor

@awreece I could see this as a documentation update. I'm skeptical about changing this code though. If we add clone bytes, other scan implementations would start copying it twice.

@awreece
Copy link
Contributor Author

awreece commented Mar 22, 2018

Fair enough. I personally prefer the consistency of "convertAssign will copy the src if it is a []byte in all cases except RawBytes"

@andybons andybons added this to the Unplanned milestone Mar 26, 2018
@gopherbot
Copy link

Change https://golang.org/cl/107995 mentions this issue: database/sql: add the driver.ValueScanner interface

@gopherbot
Copy link

Change https://golang.org/cl/108535 mentions this issue: database/sql: add note to Scanner that the database owns values

@golang golang locked and limited conversation to collaborators May 11, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants