Opened 7 years ago

Closed 4 years ago

#17108 closed enhancement (invalid)

faster testing with change impact analysis

Reported by: haysmark Owned by: haysmark
Priority: low Milestone: 1.11
Component: TestFramework Version: 1.9.0
Keywords: Cc:
Blocked By: Blocking:

Description

Going forward with 1.10 and 2.0, there is a lot of discussion about various test tooling. One detail that is sometimes missed is that we need a better understanding of which test cases apply to a given change. Ideally we would have a precommit hook that would run regression tests automatically, but right now that is not an option because the tests take so long to run.

I've been prototyping a DOH test module that decides which test cases apply to a list of changed files. The decision is simply based on require/define/initRobot calls; while this info gives a very conservative estimate, it will be vastly better than what we currently have.

Change History (13)

comment:1 Changed 7 years ago by haysmark

Owner: set to haysmark
Status: newassigned

comment:2 Changed 7 years ago by haysmark

Priority: undecidedlow

comment:3 Changed 7 years ago by doughays-dojo <doughays@…>

In 7b47ddc4691bf1df2dcb33b0a0ed23b78acd26d7/util:

Error: Processor CommitTicketReference failed
Unsupported version control system "git": Can't find an appropriate component, maybe the corresponding plugin was not enabled? 

comment:4 Changed 7 years ago by Colin Snover

Feature enhancements do not belong on minor release branches. Please git revert 7b47ddc4.

comment:5 Changed 7 years ago by Kitson Kelly

When did we start committing changes to released branches introducing new features without and discussion? This needs to be reverted.

comment:6 Changed 7 years ago by haysmark

There is a precedent to allow internal testing changes without discussion; see the testing changes Bill makes periodically. Certainly it is unusual for a testing change to appear in 1.9 before trunk, but it just so happened that the graph was generated against 1.9. I would be happy to change the order in which the code is committed, but for some reason I was relegated to a non-committer status in the SVN->Git transition; where was the discussion on THAT?

Last edited 7 years ago by haysmark (previous) (diff)

comment:7 Changed 7 years ago by Kitson Kelly

As discussed in the weekly meeting, committers are being added to the GitHub? repository on an on request basis, specifically so that they are clear on how we plan to manage the workflow until we are ready, because it is clear that some people don't understand well enough how to commit as well to GitHub? as SVN.

I still say it gets reverted and just because someone else does it doesn't make it right. Changing test is one thing, adding features to published branches it not acceptable in my opinion.

comment:8 Changed 7 years ago by haysmark

Ok I see, I was in the Bay Area presenting at ICSE so I must have missed that announcement. I'll revert it when csnover/you get back to me on my committer access. This obvious Dojo process snafu aside, I'm pretty well aware of the various git workflows.

comment:10 Changed 7 years ago by Douglas Hays

I reverted the change but someone should double check since I'm still getting used to git.

comment:11 Changed 7 years ago by haysmark

It's gone...

comment:12 Changed 4 years ago by dylan

Milestone: tbd1.11
Status: assignedpending

Given that we're dropping DOH for Intern, I'm going to suggest we close this out. That said, a feature like this could be a nice enhancement for Intern perhaps?

Sorry that we never landed this in DOH itself. That said, if you want to land it in DOH for 1.11, I think it would probably be accepted.

comment:13 Changed 4 years ago by trac-o-bot

Resolution: invalid
Status: pendingclosed

Because we get so many tickets, we often need to return them to the initial reporter for more information. If that person does not reply within 14 days, the ticket will automatically be closed, and that has happened in this case. If you still are interested in pursuing this issue, feel free to add a comment with the requested information and we will be happy to reopen the ticket if it is still valid. Thanks!

Note: See TracTickets for help on using tickets.