Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: add "-integration" flag for running integration tests #46409

Closed
DmitriyVTitov opened this issue May 27, 2021 · 6 comments
Closed

testing: add "-integration" flag for running integration tests #46409

DmitriyVTitov opened this issue May 27, 2021 · 6 comments
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@DmitriyVTitov
Copy link

Go testing framework has only one pre-built flag "-test.short" which is not very clear for me. Often I write integration tests which communicate with external dependencies such as databases and API across the network. These tests should not run along with unit tests.
I would like to have built-in flag "-test.integration" which works like "-test.short" to use within integration tests like this:

	if !testing.Integration() {
		t.Skip("use '-integration' flag to run this test")
	}

English is not my native so maybe name is not the best, but I'm sure we need such flag.

@mvdan
Copy link
Member

mvdan commented May 27, 2021

Have you seen threads like https://stackoverflow.com/questions/25965584/separating-unit-tests-and-integration-tests-in-go?

You already have multiple ways to separate integration tests:

  • If they're not too tricky/slow to run, you can just make them "long" tests via -short.
  • If they are tricky or very slow, you can make them opt-in via a custom flag or an environment variable.
  • A variation on the last method is to put them in a package not covered by go test ./..., such as a nested module.

I don't think a "blessed" and built-in way to do this is a good idea - everyone's integration tests have different requirements. I imagine most people's will require extra custom steps anyway, like installing dependencies or setting up a daemon, so I don't imagine that the consistency of go test -integration will help.

@DmitriyVTitov
Copy link
Author

I use short flag in opposite meaning not to run integration tests. Really I don't get precise meaning of "-short" flag and I thought it would be super simple to add custom flag for integrations tests. I understand that I should not ask for change for every minor reason, but this is part of testing methodology.
I know that I can separate tests, but why should I use build tag (they affect whole file) or custom flag (I would need to parse flags explicitly)? Isn't it much easer to use built-in flag?
There are 1-2 levels of tests above unit testing. So we could have 1-2 flags for them.

@mvdan
Copy link
Member

mvdan commented May 27, 2021

Isn't it much easer to use built-in flag?

Adding more built-in flags is not easier, you add more edge cases that we need to take care of and document.

For example, what should go test -short -integration do? What about go test -integration -bench=xxx?

I would need to parse flags explicitly

You can add var integration = flag.String(...) in just one line. You don't even need to call flag.Parse for tests.

There are 1-2 levels of tests above unit testing. So we could have 1-2 flags for them.

go test is only concerned about running tests - it does not attempt to classify them between "unit", "integration", etc.

@DmitriyVTitov
Copy link
Author

I don't understand about edge cases. What should -short -integration do? Nothing, it's just flags.
But OK, if you think this is inappropriate - close the issue. I would use global vars for custom flags in every package.

@bcmills
Copy link
Contributor

bcmills commented May 27, 2021

See previously: #30595, #31310, #34707.

(In particular, this looks like a near-exact duplicate of #30595.)

@mknyszek mknyszek changed the title go test: add "-integration" flag for running integration tests cmd/go: add "-integration" flag for running integration tests May 27, 2021
@mknyszek mknyszek added FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels May 27, 2021
@mknyszek mknyszek added this to the Backlog milestone May 27, 2021
@mknyszek mknyszek changed the title cmd/go: add "-integration" flag for running integration tests testing: add "-integration" flag for running integration tests May 27, 2021
@ianlancetaylor
Copy link
Contributor

Closing as dup. Sorry. Please comment if you disagree.

@golang golang locked and limited conversation to collaborators May 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants