Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/blog/content/context: update google/google.go to migrate to the new search API #34098

Closed
iridiscent opened this issue Sep 5, 2019 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@iridiscent
Copy link

What version of Go are you using (go version)?

$ go version
1.12.1

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

Mac OSX darwin amd64

go env Output
$ go env

What did you do?

https://blog.golang.org/context/google/google.go
In func Search the URL
req, err := http.NewRequest("GET", "https://ajax.googleapis.com/ajax/services/search/web?v=1.0", nil)
is no longer valid. Using it directly in my browser with a query gave me the following response:
{"responseData": null, "responseDetails": "The Google Web Search API is no longer available. Please migrate to the Google Custom Search API (https://developers.google.com/custom-search/)", "responseStatus": 403}

What did you expect to see?

Expected to see search results

What did you see instead?

Saw the response above

@gopherbot gopherbot added this to the Unreleased milestone Sep 5, 2019
@hyangah hyangah changed the title x/blog: x/blog/content/context: update google/google.go to migrate to the new search API Sep 5, 2019
@katiehockman
Copy link
Contributor

I'm not able to reproduce this. When I navigate to that URL I see the following:

https://blog.golang.org/context/google/google.go // +build OMIT

// Package google provides a function to do Google searches using the Google Web
// Search API. See https://developers.google.com/web-search/docs/
//
// This package is an example to accompany https://blog.golang.org/context.
// It is not intended for use by others.
//
// Google has since disabled its search API,
// and so this package is no longer useful.
package google

import (
"context"
"encoding/json"
"net/http"

"golang.org/x/blog/content/context/userip"

)

// Results is an ordered list of search results.
type Results []Result

// A Result contains the title and URL of a search result.
type Result struct {
Title, URL string
}

// Search sends query to Google search and returns the results.
func Search(ctx context.Context, query string) (Results, error) {
// Prepare the Google Search API request.
req, err := http.NewRequest("GET", "https://ajax.googleapis.com/ajax/services/search/web?v=1.0", nil)
if err != nil {
return nil, err
}
q := req.URL.Query()
q.Set("q", query)

// If ctx is carrying the user IP address, forward it to the server.
// Google APIs use the user IP to distinguish server-initiated requests
// from end-user requests.
if userIP, ok := userip.FromContext(ctx); ok {
	q.Set("userip", userIP.String())
}
req.URL.RawQuery = q.Encode()

// Issue the HTTP request and handle the response. The httpDo function
// cancels the request if ctx.Done is closed.
var results Results
err = httpDo(ctx, req, func(resp *http.Response, err error) error {
	if err != nil {
		return err
	}
	defer resp.Body.Close()

	// Parse the JSON search result.
	// https://developers.google.com/web-search/docs/#fonje
	var data struct {
		ResponseData struct {
			Results []struct {
				TitleNoFormatting string
				URL               string
			}
		}
	}
	if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
		return err
	}
	for _, res := range data.ResponseData.Results {
		results = append(results, Result{Title: res.TitleNoFormatting, URL: res.URL})
	}
	return nil
})
// httpDo waits for the closure we provided to return, so it's safe to
// read results here.
return results, err

}

// httpDo issues the HTTP request and calls f with the response. If ctx.Done is
// closed while the request or f is running, httpDo cancels the request, waits
// for f to exit, and returns ctx.Err. Otherwise, httpDo returns f's error.
func httpDo(ctx context.Context, req *http.Request, f func(*http.Response, error) error) error {
// Run the HTTP request in a goroutine and pass the response to f.
c := make(chan error, 1)
req = req.WithContext(ctx)
go func() { c <- f(http.DefaultClient.Do(req)) }()
select {
case <-ctx.Done():
<-c // Wait for f to return.
return ctx.Err()
case err := <-c:
return err
}
}

Are you still experiencing this issue?

@katiehockman katiehockman added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed help wanted labels Sep 5, 2019
@iridiscent
Copy link
Author

iridiscent commented Sep 5, 2019 via email

@katiehockman
Copy link
Contributor

@iridiscent thanks for the additional context. I'm still not sure I completely understand the issue, but looping in @andybons who may know more.

@katiehockman katiehockman added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 5, 2019
@andybons
Copy link
Member

andybons commented Sep 6, 2019

As it says in the error message:

The Google Web Search API is no longer available. Please migrate to the Google Custom Search API (https://developers.google.com/custom-search/)

We don’t maintain that API. Just the example, which we likely won’t update as blog entries are considered somewhat immutable.

I’m sure there are other APIs you could adapt this workflow to, but we can’t do anything in this case.

Thanks.

@andybons andybons closed this as completed Sep 6, 2019
@iridiscent
Copy link
Author

iridiscent commented Sep 6, 2019 via email

@golang golang locked and limited conversation to collaborators Sep 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants