Comments on: Being a Responsible Owner http://shawnwilsher.com/archives/145 Wed, 03 Mar 2010 21:06:45 -0800 http://wordpress.org/?v=2.9.1 hourly 1 By: Jonathan Watt http://shawnwilsher.com/archives/145/comment-page-1#comment-2756 Jonathan Watt Mon, 31 Mar 2008 09:28:28 +0000 http://shawnwilsher.com/archives/145#comment-2756 Shawn: Sure, those are all reasons for writing tests, and all these reasons should be presented to our contributors. That's orthogonal to my point though. I was saying that if a volunteer isn't happy to write tests (for whatever reason) after being given the full explanation, we shouldn't chase them away by also demanding tests or making them feel bad. We've managed to ship good software in the past without good test coverage (not sure how :-) ), but we've never managed to ship good software without volunteers. Shawn: Sure, those are all reasons for writing tests, and all these reasons should be presented to our contributors. That’s orthogonal to my point though. I was saying that if a volunteer isn’t happy to write tests (for whatever reason) after being given the full explanation, we shouldn’t chase them away by also demanding tests or making them feel bad. We’ve managed to ship good software in the past without good test coverage (not sure how :-) ), but we’ve never managed to ship good software without volunteers.

]]> By: On Unit Testing Requirements http://shawnwilsher.com/archives/145/comment-page-1#comment-2752 On Unit Testing Requirements Mon, 31 Mar 2008 05:35:33 +0000 http://shawnwilsher.com/archives/145#comment-2752 [...] policy doesn’t allow for that as an exception, so someone else needs to write the test. One commenter suggested that someone else write the test(s) for the patch submitter. From a patcher’s [...] [...] policy doesn’t allow for that as an exception, so someone else needs to write the test. One commenter suggested that someone else write the test(s) for the patch submitter. From a patcher’s [...]

]]> By: Shawn Wilsher http://shawnwilsher.com/archives/145/comment-page-1#comment-2751 Shawn Wilsher Mon, 31 Mar 2008 04:41:16 +0000 http://shawnwilsher.com/archives/145#comment-2751 @Stuart As much as I'd like to say "we can get to this test later," I'm pretty much bound by toolkit review policy [1], which states "All patches must include unit tests which are run during <code>make check</code>." I'm trying to be responsible and actually follow this policy, even though I know many submodules in toolkit don't. While it's true that we have a testing community, there are (last I knew) over one hundred tests for the download manager alone. In the past, it's taken them several weeks to find regressions to things (many of which would have been easily tested with a unit test). On top of that, there are a lot of things that simply have no test coverage in litmus or in the automated test suite. I personally found out about a lot of download manager features because people filed a bug saying that we broke it a few weeks ago. @Jonathan I don't exactly agree with you. When I first started hacking, I had a lot of fun just learning all the new techniques to do things. Writing tests is just another good technique to writing better code. It helps to prove that your fix is correct, it makes sure you don't break what you just fixed in the future, and it makes sure you have a more sensible API (usually) because you have to test it. @jag Doesn't look like it, but we do have the in-testsuite flags, which there are currently 28 download manager bugs marked as such. I get your point, but I think we need to work to make (2) the defacto choice for the good of the codebase. @Bernd But who would write the tests for the second group? If you leave that to the module owner, he or she could very easily get swamped just writing tests and doing reviews. [1] http://www.mozilla.org/projects/toolkit/review.html @Stuart

As much as I’d like to say “we can get to this test later,” I’m pretty much bound by toolkit review policy [1], which states “All patches must include unit tests which are run during make check.” I’m trying to be responsible and actually follow this policy, even though I know many submodules in toolkit don’t.

While it’s true that we have a testing community, there are (last I knew) over one hundred tests for the download manager alone. In the past, it’s taken them several weeks to find regressions to things (many of which would have been easily tested with a unit test). On top of that, there are a lot of things that simply have no test coverage in litmus or in the automated test suite. I personally found out about a lot of download manager features because people filed a bug saying that we broke it a few weeks ago.

@Jonathan

I don’t exactly agree with you. When I first started hacking, I had a lot of fun just learning all the new techniques to do things. Writing tests is just another good technique to writing better code. It helps to prove that your fix is correct, it makes sure you don’t break what you just fixed in the future, and it makes sure you have a more sensible API (usually) because you have to test it.

@jag

Doesn’t look like it, but we do have the in-testsuite flags, which there are currently 28 download manager bugs marked as such.

I get your point, but I think we need to work to make (2) the defacto choice for the good of the codebase.

@Bernd

But who would write the tests for the second group? If you leave that to the module owner, he or she could very easily get swamped just writing tests and doing reviews.

[1] http://www.mozilla.org/projects/toolkit/review.html

]]> By: Bernd http://shawnwilsher.com/archives/145/comment-page-1#comment-2750 Bernd Sun, 30 Mar 2008 15:05:54 +0000 http://shawnwilsher.com/archives/145#comment-2750 There should be a separation between contributors who have cvs write permission and volunteers who do not. There should be no excuses for the first group. There should be a separation between contributors who have cvs write permission and volunteers who do not. There should be no excuses for the first group.

]]> By: jag http://shawnwilsher.com/archives/145/comment-page-1#comment-2749 jag Sun, 30 Mar 2008 14:47:16 +0000 http://shawnwilsher.com/archives/145#comment-2749 Do keywords like testwanted and testneeded exist? How about landing certain patches with those keywords added? Also, you don't want to get into a situation where the choices are 1) fix regression by fixing an obvious typo, but now I have to write a test and *insert exc^H^H^Hreason I can't right now*, or 2) back out the change (and with that some other feature or fix) that caused regression in the first place (with the added benefit of putting the test-writing burden on the regressor). When (2) becomes more attractive than (1), guess where people will look. I know that's probably exaggerating things just a tad, but I am trying to make a point ;-) Do keywords like testwanted and testneeded exist? How about landing certain patches with those keywords added?

Also, you don’t want to get into a situation where the choices are 1) fix regression by fixing an obvious typo, but now I have to write a test and *insert exc^H^H^Hreason I can’t right now*, or 2) back out the change (and with that some other feature or fix) that caused regression in the first place (with the added benefit of putting the test-writing burden on the regressor). When (2) becomes more attractive than (1), guess where people will look.

I know that’s probably exaggerating things just a tad, but I am trying to make a point ;-)

]]> By: Jonathan Watt http://shawnwilsher.com/archives/145/comment-page-1#comment-2743 Jonathan Watt Sun, 30 Mar 2008 00:18:07 +0000 http://shawnwilsher.com/archives/145#comment-2743 I think putting pressure on volunteers to write tests when their good will doesn't extend that far is very unwise - at least in the long run. Right now, during the run up to Firefox 3, it makes some sense. We need to do everything possible to make sure fixes don't introduce worse bugs. In the long run we should be willing to accept what people are willing to contribute though. Writing tests should be strongly encouraged, and we should explain to people why they're so important. This shouldn't come at the expense of taking the fun away from volunteer contributors though. Patches and tests don't necessarily need to come from the same people. I think putting pressure on volunteers to write tests when their good will doesn’t extend that far is very unwise – at least in the long run. Right now, during the run up to Firefox 3, it makes some sense. We need to do everything possible to make sure fixes don’t introduce worse bugs. In the long run we should be willing to accept what people are willing to contribute though. Writing tests should be strongly encouraged, and we should explain to people why they’re so important. This shouldn’t come at the expense of taking the fun away from volunteer contributors though. Patches and tests don’t necessarily need to come from the same people.

]]> By: Stuart Parmenter http://shawnwilsher.com/archives/145/comment-page-1#comment-2742 Stuart Parmenter Sat, 29 Mar 2008 20:54:54 +0000 http://shawnwilsher.com/archives/145#comment-2742 separately from their patches in the sense of get tests that cover what is there currently, not that all tests for new code/changes/areas should be done separately from the bug they're fixing separately from their patches in the sense of get tests that cover what is there currently, not that all tests for new code/changes/areas should be done separately from the bug they’re fixing

]]> By: Stuart Parmenter http://shawnwilsher.com/archives/145/comment-page-1#comment-2741 Stuart Parmenter Sat, 29 Mar 2008 20:53:28 +0000 http://shawnwilsher.com/archives/145#comment-2741 Shawn: Requiring tests raises the bar of entry pretty high in many cases. I don't think you'll get much argument from anyone that tests are good, but I think it is important to balance adding tests with progress. I would suggest that you as a module owner attempt to get people to build tests separately from their patches to get the additional coverage you want. We also have an amazing testing community who while isn't quite as fast as our automated tests, is pretty close. I don't see a lot of harm in letting them help to find regressions. Certainly if you keep hitting the same problem over and over again automated tests will be valuable, but it isn't clear from your post or Edward's that this is the case. Shawn: Requiring tests raises the bar of entry pretty high in many cases. I don’t think you’ll get much argument from anyone that tests are good, but I think it is important to balance adding tests with progress. I would suggest that you as a module owner attempt to get people to build tests separately from their patches to get the additional coverage you want. We also have an amazing testing community who while isn’t quite as fast as our automated tests, is pretty close. I don’t see a lot of harm in letting them help to find regressions. Certainly if you keep hitting the same problem over and over again automated tests will be valuable, but it isn’t clear from your post or Edward’s that this is the case.

]]> By: Shawn Wilsher http://shawnwilsher.com/archives/145/comment-page-1#comment-2740 Shawn Wilsher Sat, 29 Mar 2008 17:49:03 +0000 http://shawnwilsher.com/archives/145#comment-2740 @FrnchFrgg I realize my post didn't make this clear - I'm not upset with you about the patch and lack of tests. In fact, I didn't hear you complain - you just asked for clarification on what needed to be tested. I'm upset about other people who were indicated that I was being unreasonable for asking for a test because they clearly don't get it. @FrnchFrgg

I realize my post didn’t make this clear – I’m not upset with you about the patch and lack of tests. In fact, I didn’t hear you complain – you just asked for clarification on what needed to be tested. I’m upset about other people who were indicated that I was being unreasonable for asking for a test because they clearly don’t get it.

]]> By: FrnchFrgg http://shawnwilsher.com/archives/145/comment-page-1#comment-2739 FrnchFrgg Sat, 29 Mar 2008 17:44:42 +0000 http://shawnwilsher.com/archives/145#comment-2739 The problem in my case is that I don't (or didn't) know how to write tests and what to test for; so I was completely stuck for some time when you asked for a test. Add on top of that my own crunch time because I am in the middle of the written tests for an hiring contest, and you get a non written test which blocks a patch, even if I swear I didn't think the test was too stupid for me to write it. The problem in my case is that I don’t (or didn’t) know how to write tests and what to test for; so I was completely stuck for some time when you asked for a test. Add on top of that my own crunch time because I am in the middle of the written tests for an hiring contest, and you get a non written test which blocks a patch, even if I swear I didn’t think the test was too stupid for me to write it.

]]>