Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build/maintner: refactor out gcs and api from maintnerd #24786

Closed
broady opened this issue Apr 9, 2018 · 6 comments
Closed

x/build/maintner: refactor out gcs and api from maintnerd #24786

broady opened this issue Apr 9, 2018 · 6 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Milestone
Unreleased

Comments

@broady
Copy link
Member

broady commented Apr 9, 2018 •
edited

I want to create a customized maintnerd (one feature is being able to add tracked repos while the daemon is running).

Here's a small proposal to make that more possible:

  • move all of the GCS (gcslog.go) and API functions (api.go) to a new package: "maintnerapi". optional: move GCS functionality into its own package.
  • move the protos into a subpackage (currently named "apipb" but maybe "maintpb" is better?) nevermind, there's already a maintpb for storage.

maintnerd then becomes a 500 line program that uses the maintnerapi package, which looks like this:

PACKAGE DOCUMENTATION

package maintnerapi
    import "."


FUNCTIONS

func NewAPIService(corpus *maintner.Corpus) apipb.MaintnerServiceServer

TYPES

type GCSLog struct {
    // contains filtered or unexported fields
}

func NewGCSLog(ctx context.Context, bucketName string) (*GCSLog, error)

func (gl *GCSLog) CopyFrom(src maintner.MutationSource) error
    CopyFrom is only used for the one-time migrate from disk-to-GCS code
    path.

func (gl *GCSLog) GetMutations(ctx context.Context) <-chan maintner.MutationStreamEvent

func (gl *GCSLog) Log(m *maintpb.Mutation) error

func (gl *GCSLog) ServeJSONLogsIndex(w http.ResponseWriter, r *http.Request)

func (gl *GCSLog) ServeLogFile(w http.ResponseWriter, r *http.Request)
@gopherbot gopherbot added this to the Unreleased milestone Apr 9, 2018
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Apr 9, 2018
@broady
Copy link
Member Author

broady commented Apr 9, 2018

/cc @bradfitz @andybons @shurcooL

@gopherbot
Copy link

Change https://golang.org/cl/105935 mentions this issue: maintner: refactor API into its own package

@dmitshur
Copy link
Contributor

dmitshur commented Apr 9, 2018

Would this affect the users of golang.org/x/build/maintner/godata package?

@bradfitz
Copy link
Contributor

bradfitz commented Apr 9, 2018

I want to create a customized maintnerd (one feature is being able to add tracked repos while the daemon is running).

Curious... why?

@broady
Copy link
Member Author

broady commented Apr 9, 2018 •
edited

@shurcooL nope. the only real change is moving the "apipb" package, but it's only used by maintnerd and maintq.

@bradfitz I'm tracking dozens/hundreds of repos in the google, googleapis, GoogleCloudPlatform orgs. several repos get added every week. We don't want to restart maintnerd when a new repo gets tracked. oh, and for better or worse (could/should be a separate process), exposing another handler that serves some summarized data.

@bradfitz
Copy link
Contributor

bradfitz commented Apr 9, 2018

Okay, SGTM. And I do like the idea of moving GCSLog to its own package.

@golang golang locked and limited conversation to collaborators Jun 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@bradfitz @broady @dmitshur @gopherbot