Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packages under rsc.io are no longer available #40452

Closed
hencrice opened this issue Jul 28, 2020 · 18 comments
Closed

Packages under rsc.io are no longer available #40452

hencrice opened this issue Jul 28, 2020 · 18 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@hencrice
Copy link

What version of Go are you using (go version)?

Go 1.12, Go 1.14

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

I don't think this matters but here it goes. This failed both on my laptop with Go 1.14 and in GitHub Action that uses Go1.12

go env Output
$ go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/yenlinc/Library/Caches/go-build"
GOENV="/Users/yenlinc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB="
"
GOOS="darwin"
GOPATH="/Users/yenlinc/go"
GOPRIVATE="*"
GOPROXY="direct"
GOROOT="/usr/local/Cellar/go/1.14.3/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/yenlinc/workplace/yenlinc/amazon-kinesis-streams-for-fluent-bit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v4/nbxvs8tj4ms68w5ghhnzn_f1ckngqg/T/go-build594642831=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Ran go build command

What did you expect to see?

Modules should be fetched successfully and compilation should succeed.

What did you see instead?

Since this morning, 2 packages under rsc.io have been failing:

go build -buildmode c-shared -o ./bin/kinesis.so ./
go: github.com/aws/amazon-kinesis-firehose-for-fluent-bit@v1.4.1 requires
	github.com/golang/mock@v1.4.3 requires
	rsc.io/quote/v3@v3.1.0: unrecognized import path "rsc.io/quote/v3": reading https://rsc.io/quote/v3?go-get=1: 503 Service Unavailable
$ go mod download

go: finding github.com/aws/amazon-kinesis-firehose-for-fluent-bit v1.2.2-0.20200717080229-e3ffb88e79bb
go: finding github.com/aws/aws-sdk-go v1.33.13
go: finding github.com/lestrrat-go/strftime v1.0.2-0.20200618102204-1b0bc59fa4ab
go: finding github.com/stretchr/objx v0.3.0
go: finding github.com/sirupsen/logrus v1.6.0
go: finding github.com/yuin/goldmark v1.1.33
go: rsc.io/sampler@v1.99.99: unrecognized import path "rsc.io/sampler" (parse https://rsc.io/sampler?go-get=1: no go-import meta tags ())
go: finding golang.org/x/crypto v0.0.0-20200709230013-948cd5f35899
go: finding golang.org/x/text v0.3.3
go: finding github.com/google/gofuzz v1.1.0
go: finding gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776
go: finding github.com/cenkalti/backoff v2.2.1+incompatible
go: finding golang.org/x/net v0.0.0-20200707034311-ab3426394381
go: finding github.com/fluent/fluent-bit-go v0.0.0-20200707230002-2a28684e2382
go: finding github.com/stretchr/testify v1.6.1
go: finding github.com/json-iterator/go v1.1.10
go: finding gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f
go: finding golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae
go: finding golang.org/x/tools v0.0.0-20200717024301-6ddee64345a6
go: finding github.com/golang/mock v1.4.3
go: error loading module requirements
@hencrice
Copy link
Author

https://rsc.io/sampler?go-get=1 and https://rsc.io/quote/v3?go-get=1 show

Go 1.9 is no longer available. Please refer to https://goo.gl/aESk5L for more information.

Seems like the service that serve these pages are deprecated by Google Apps engine

@seankhliao
Copy link
Member

this is mitigated by using the proxy, which you appear to have disabled with GOPROXY=direct

but @rsc probably needs to update their app engine server

@hencrice
Copy link
Author

hencrice commented Jul 28, 2020

@seankhliao Thanks for the reply. I've actually tried that and it failed with the same error:

This is what I tried:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/yenlinc/Library/Caches/go-build"
GOENV="/Users/yenlinc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="*"
GONOSUMDB="*"
GOOS="darwin"
GOPATH="/Users/yenlinc/go"
GOPRIVATE="*"
GOPROXY="https://proxy.golang.org"
GOROOT="/usr/local/Cellar/go/1.14.3/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/yenlinc/workplace/yenlinc/amazon-kinesis-streams-for-fluent-bit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v4/nbxvs8tj4ms68w5ghhnzn_f1ckngqg/T/go-build312867321=/tmp/go-build -gno-record-gcc-switches -fno-common"

@seankhliao
Copy link
Member

@prattmic prattmic added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 28, 2020
@prattmic
Copy link
Member

cc @rsc @dmitshur @toothrot

While this would probably not normally be considered a Go issue, it is breaking the -longtest builders.

e.g., https://storage.googleapis.com/go-build-log/3c347861/linux-amd64-longtest_bb29e807.log

@cagedmantis cagedmantis added this to the soon milestone Jul 28, 2020
@cagedmantis cagedmantis added the Builders x/build issues (builders, bots, dashboards) label Jul 28, 2020
@AntonNguyen
Copy link

Just wanted to chime in and say that it's breaking for my builds (go 1.12) as well:

go test -coverprofile=coverage.txt -covermode=count ./...
go: finding github.com/golang/mock v1.4.3
go: finding github.com/go-kit/kit v0.9.0
go: finding github.com/prometheus/client_golang v1.3.0
go: finding github.com/golang/protobuf v1.4.2
go: finding github.com/kelseyhightower/envconfig v1.4.0
go: finding github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a
go: finding github.com/stretchr/testify v1.6.1
go: finding google.golang.org/grpc v1.30.0
go: finding github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/go-stack/stack v1.8.0
go: rsc.io/quote/v3@v3.1.0: unrecognized import path "rsc.io/quote/v3" (parse https://rsc.io/quote/v3?go-get=1: no go-import meta tags ())
go: finding github.com/json-iterator/go v1.1.8
go: finding github.com/stretchr/testify v1.3.0
go: finding github.com/prometheus/client_model v0.1.0
go: finding github.com/prometheus/procfs v0.0.8
go: finding golang.org/x/tools v0.0.0-20190425150028-36563e24a262
go: finding github.com/kr/pretty v0.1.0
go: finding go.opencensus.io v0.22.1
go: finding golang.org/x/sys v0.0.0-20191220142924-d4481acd189f
go: finding github.com/cespare/xxhash/v2 v2.1.1
go: finding gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/davecgh/go-spew v1.1.0
go: finding golang.org/x/net v0.0.0-20190628185345-da137c7871d7
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/stretchr/objx v0.1.0
go: finding github.com/prometheus/common v0.7.0
go: finding github.com/golang/mock v1.3.1
go: finding github.com/satori/go.uuid v1.2.0
go: finding cloud.google.com/go/datastore v1.1.0
go: error loading module requirements

@cagedmantis cagedmantis modified the milestones: soon, Backlog Jul 28, 2020
@cagedmantis cagedmantis added the Soon This needs to be done soon. (regressions, serious bugs, outages) label Jul 28, 2020
@dmitshur
Copy link
Contributor

dmitshur commented Jul 28, 2020

@hencrice I can't reproduce this with Go 1.14 with a default configuration:

$ go version
go version go1.14.6 darwin/amd64
$ cd $(mktemp -d)
$ go mod init example.com/m
go: creating new go.mod: module example.com/m
$ go mod download github.com/aws/amazon-kinesis-firehose-for-fluent-bit@v1.4.1
$ echo $?
0

What is your go env GOPROXY value? Is it modified from the default value of https://proxy.golang.org,direct? I'm guessing it does not use the module mirror. Edit: I see GOPROXY="direct" in the original issue report now.

Is it an option for you to start using the module mirror (https://proxy.golang.org) by reverting GOPROXY to its default value of https://proxy.golang.org,direct?

Making cmd/go tests not depend on external services is tracked in issue #28856. We should see if the current -longtest failures should be in scope of that issue. I'll see how quickly Russ expects to be able to fix rsc.io to determine the best next steps for it.

@hencrice
Copy link
Author

@dmitshur I've tried to set GOPROXY to direct, https://proxy.golang.org and https://proxy.golang.org,direct. All failed with the same error message.

@AntonNguyen
Copy link

Okay, setting GOPROXY fixed it for me:

$ export GOPROXY="https://proxy.golang.org"

Then when I directly download binaryregexp, it succeeds:

$ go get -u rsc.io/binaryregexp
go: finding github.com/go-openapi/errors v0.17.0
go: finding github.com/go-openapi/jsonreference v0.19.2
go: finding github.com/mitchellh/mapstructure v1.1.2
go: finding golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f
go: finding github.com/globalsign/mgo v0.0.0-20180905125535-1ca0a4f7cbcb
go: finding github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf
go: finding github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329
go: finding github.com/kr/pty v1.1.5
go: finding golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8
go: finding github.com/go-openapi/jsonpointer v0.19.2
go: finding rsc.io/binaryregexp v0.2.0
go: finding github.com/PuerkitoBio/purell v1.1.1
go: finding github.com/go-openapi/loads v0.19.0
go: finding github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
go: finding golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: finding github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63
go: finding github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
go: finding golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59
go: finding github.com/go-openapi/runtime v0.19.0
go: finding github.com/go-openapi/errors v0.18.0
go: finding github.com/go-openapi/jsonreference v0.18.0
go: finding golang.org/x/sys v0.0.0-20190321052220-f7bb7a8bee54
go: finding golang.org/x/net v0.0.0-20190320064053-1272bf9dcd53
go: finding golang.org/x/crypto v0.0.0-20190320223903-b7391e95e576
go: finding github.com/go-openapi/validate v0.18.0
go: finding github.com/go-openapi/jsonpointer v0.18.0
go: finding github.com/go-openapi/strfmt v0.18.0
go: finding github.com/go-openapi/spec v0.17.0
go: finding golang.org/x/net v0.0.0-20181005035420-146acd28ed58
go: finding github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe
go: finding github.com/go-openapi/swag v0.17.0
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/go-openapi/swag v0.18.0
go: finding github.com/go-openapi/jsonreference v0.17.0
go: finding github.com/go-openapi/loads v0.18.0
go: finding github.com/go-openapi/runtime v0.0.0-20180920151709-4f900dc2ade9
go: finding github.com/go-openapi/spec v0.18.0
go: finding github.com/go-openapi/analysis v0.0.0-20180825180245-b006789cd277
go: finding github.com/go-openapi/jsonpointer v0.17.0
go: finding github.com/PuerkitoBio/purell v1.1.0
go: finding github.com/go-openapi/strfmt v0.17.0
go: finding github.com/pborman/uuid v1.2.0
go: finding github.com/go-openapi/analysis v0.17.0
go: finding github.com/go-openapi/loads v0.17.0
go: finding github.com/go-openapi/analysis v0.18.0
go: finding github.com/google/uuid v1.0.0
go: downloading rsc.io/binaryregexp v0.2.0
go: extracting rsc.io/binaryregexp v0.2.0

@hencrice
Copy link
Author

@AntonNguyen Could you show your full go env? I tried setting GOPROXY but it still failed for me.

@dmitshur
Copy link
Contributor

@hencrice Please post the full output of running go version, then go env, then the exact command you ran that's still giving an error, so we have full information.

It should work okay if GOPROXY is set to use the module mirror and the value is used when you run go build or go mod download, so perhaps the problem was your configuration change wasn't picked up by the other command.

@AntonNguyen
Copy link

AntonNguyen commented Jul 28, 2020

@AntonNguyen Could you show your full go env? I tried setting GOPROXY but it still failed for me.
@hencrice

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/circleci/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY="https://proxy.golang.org"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/go/src/github.com/user/repo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build689553606=/tmp/go-build -gno-record-gcc-switches"

@hencrice
Copy link
Author

@dmitshur

go version outputs:

go version go1.14.3 darwin/amd64

go env outputs:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/yenlinc/Library/Caches/go-build"
GOENV="/Users/yenlinc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="*"
GONOSUMDB="*"
GOOS="darwin"
GOPATH="/Users/yenlinc/go"
GOPRIVATE="*"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.3/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/yenlinc/workplace/yenlinc/amazon-kinesis-firehose-for-fluent-bit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v4/nbxvs8tj4ms68w5ghhnzn_f1ckngqg/T/go-build709008692=/tmp/go-build -gno-record-gcc-switches -fno-common"

Content of go.mod:

module github.com/aws/amazon-kinesis-firehose-for-fluent-bit

go 1.12

require (
	github.com/aws/aws-sdk-go v1.33.13
	github.com/cenkalti/backoff v2.2.1+incompatible
	github.com/fluent/fluent-bit-go v0.0.0-20190614024040-c017a8579953
	github.com/golang/mock v1.4.3
	github.com/json-iterator/go v1.1.10
	github.com/lestrrat-go/strftime v1.0.2-0.20200618102204-1b0bc59fa4ab
	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
	github.com/modern-go/reflect2 v1.0.1 // indirect
	github.com/sirupsen/logrus v1.6.0
	github.com/stretchr/testify v1.6.1
)

Commands ran:

$ mkdir -p ./bin && go build -buildmode c-shared -o ./bin/firehose.so ./

go: github.com/golang/mock@v1.4.3 requires
	rsc.io/quote/v3@v3.1.0: unrecognized import path "rsc.io/quote/v3": reading https://rsc.io/quote/v3?go-get=1: 503 Service Unavailable

One thing I discovered is if I just:

  1. remove the go.mod and go.sum
  2. reinitialize the module via go mod init github.com/aws/amazon-kinesis-firehose-for-fluent-bit
  3. Change the go version in go.mod from 1.14 to 1.12
  4. Re-run go build
    Then it compiles successfully. But I suspect that's just because the indirect dependency on rsc.io/sampler and rsc.io/quote were removed

@dmitshur
Copy link
Contributor

dmitshur commented Jul 28, 2020

@hencrice The GOPRIVATE environment variable in go env output is set to *, which tells the go command to consider all modules as private and not to use the module proxy and checksum database for them.

To be able to fetch rsc.io/quote/v3 and other public modules from the mirror, that environment variable will need to be updated to something narrower. See https://golang.org/cmd/go/#hdr-Module_configuration_for_non_public_modules for more information about it.

@menghanl
Copy link

This break gRPC's tests for go 1.12 as well: https://travis-ci.org/github/grpc/grpc-go

While we try to figure the reason behind the failure, is there any intent to fix rsc.io/quote?
rsc.io/quote still shows Go 1.9 is no longer available. Please refer to https://goo.gl/aESk5L for more information.

@dmitshur
Copy link
Contributor

@menghanl Fixing rsc.io/quote is being tracked in issue rsc/quote#4.

@dmitshur
Copy link
Contributor

rsc.io is working again now (see rsc/quote#4 (comment)). Making cmd/go tests not depend on external services is tracked in issue #28856. I don't think there's anything else left to do here, so I'll close this.

@dmitshur dmitshur removed the Soon This needs to be done soon. (regressions, serious bugs, outages) label Jul 28, 2020
@benhoyt
Copy link
Contributor

benhoyt commented Jul 28, 2020

Russ's personal website (https://swtch.com/), which also hosts a lot of semi-official Go-related content like the Go versioning and Go proposal documents, is down with the same issue. I have a ton of respect for Russ (and Google, on good days), and these things can happen to anyone -- but one does have to see the funny side of this. As an acquaintance of mine put it: "heh, that is funny ... Go project lead's web site down due to using an EOL version of the language ... film at 11 :)"

@golang golang locked and limited conversation to collaborators Jul 28, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants