Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

need help with tls and io.Pipe #34722

Closed
vitaliy-kuzmich opened this issue Oct 6, 2019 · 2 comments
Closed

need help with tls and io.Pipe #34722

vitaliy-kuzmich opened this issue Oct 6, 2019 · 2 comments

Comments

@vitaliy-kuzmich
Copy link

vitaliy-kuzmich commented Oct 6, 2019

What version of Go are you using (go version)?

go version go1.13.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/.cache/go-build"
GOENV="
/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="~/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build376784943=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Trying to redirect pipe from zip tool from CMD to the http browser client. It works without TLS, but when I set tls cert and key, it sends corrupted file randomly, and shows "http2: stream closed" message.

Addition: on mipsle application crash with nil mistake at bufio, only when compiled with flags : -ldflags="-s -w". But without mentioned flags, the application does not crash, but produces corrupt zip file, only with tls enabled.

function :


func serveDownload(c *fb.Context, w http.ResponseWriter, files []string) (err error) {
  // Defines the file name.
  name := ""

  if c.File != nil {
  	name = c.File.Name
  }
  if name == "." || name == "" {
  	name = "archive.zip"
  } else {
  	name += ".zip"
  }
  w.Header().Set("Content-Disposition", "attachment; filename*=utf-8''"+url.PathEscape(name))
  pr, pw := io.Pipe()
  //j - cut path, and store only
  cmd := exec.Command("zip", append([]string{"-0rq", "-"}, files...)...)
  cmd.Stdout = pw
  cmd.Stderr = os.Stderr
  go func() {
  	defer pr.Close()
  	// copy the data written to the PipeReader via the cmd to stdout
  	if _, err := io.Copy(w, pr); err != nil {
  		log.Println(err)
  	}
  }()
  return cmd.Run()

}

This is full code:
https://github.com/browsefile/backend
This is rice box file, you do not need to compile front, just put it into src/lib
https://drive.google.com/file/d/1VFiNFv1_K2DiS2Wx-x03go58yaHUF3ac/view?usp=sharing

What did you expect to see?

zip file that opens

What did you see instead?

corrupted zip file and next message in console : http2: stream closed

@vitaliy-kuzmich vitaliy-kuzmich changed the title need help with tls and need help with tls and io.Pipe Oct 6, 2019
@odeke-em
Copy link
Member

odeke-em commented Oct 6, 2019

Hello @vitaliy-kuzmich, thank you for the question! Unfortunately this page isn't for asking questions, please ask questions on the following resources https://github.com/golang/go/wiki/Questions.

When filing this bugs, please also try to provide standalone reproductions.
Given your description and initial code snippet, I produced for you a full standalone snippet that has a TLS server running, makes an HTTP/2.0 request with the TLS request and streams back the zipped file which you can invoke with go run main.go > foo.zip and it unzips it alright https://play.golang.org/p/gSqhFSAUieG or inlined below

package main

import (
	"context"
	"io"
	"log"
	"net/http"
	"net/http/httptest"
	"net/http/httputil"
	"net/url"
	"os"
	"os/exec"

	"golang.org/x/net/http2"
)

func main() {
	cst := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		files := []string{"main.go"}
		if err := serveDownload(r.Context(), w, files); err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
	}))

	if err := http2.ConfigureServer(cst.Config, new(http2.Server)); err != nil {
		log.Fatal(err)
	}
	cst.TLS = cst.Config.TLSConfig
	cst.StartTLS()
	defer cst.Close()

	tr := &http.Transport{
		TLSClientConfig: cst.Config.TLSConfig,
	}
	tr.TLSClientConfig.InsecureSkipVerify = true
	http2.ConfigureTransport(tr)
	req, _ := http.NewRequest("GET", cst.URL, nil)

	res, err := tr.RoundTrip(req)
	if err != nil {
		log.Fatalf("Failed to make request: %v", err)
	}
	defer res.Body.Close()

	if false { // Uncomment in order to examine the entire response and ensure HTTP/2.0 being used.
		blob, _ := httputil.DumpResponse(res, true)
		println(string(blob))
	} else {
		io.Copy(os.Stdout, res.Body)
	}
}

func serveDownload(ctx context.Context, w http.ResponseWriter, files []string) (err error) {
	// Defines the file name.
	name := "archive.zip"
	w.Header().Set("Content-Disposition", "attachment; filename*=utf-8''"+url.PathEscape(name))
	pr, pw := io.Pipe()
	//j - cut path, and store only
	cmd := exec.Command("zip", append([]string{"-0rq", "-"}, files...)...)
	cmd.Stdout = pw
	cmd.Stderr = os.Stderr
	go func() {
		defer pr.Close()
		// copy the data written to the PipeReader via the cmd to stdout
		if _, err := io.Copy(w, pr); err != nil {
			panic(err)
		}
	}()
	return cmd.Run()
}

Running it

$ go run main.go > rp/foo.zip && cd rp && file foo.zip && unzip foo.zip && cd $OLDPWD
foo.zip: Zip archive data, at least v1.0 to extract
Archive:  foo.zip
replace main.go? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
 extracting: main.go 

which works alright, but it isn't using your *fb.Context, just the standard library's context.Context, so perhaps please check and see where/how that *fb.Context is reacting with TLS, and please follow up your question on the resources that I provided up above in https://github.com/golang/go/wiki/Questions.

@odeke-em odeke-em closed this as completed Oct 6, 2019
@vitaliy-kuzmich
Copy link
Author

@odeke-em thank you! Now I will know about questions page.

@golang golang locked and limited conversation to collaborators Oct 8, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants