Recently I read a post on the Alt.net linked-in group Would mocking help prevent these software bugs? talking about the Knight Capital Group Inc.’s disaster, asking the question "if doing more testing and using mocks would have prevented it?"
Doing unit testing with mocking is a tool, not a silver bullet. Unit tests mixed with behavior testing can be very effective at finding these kinds of bugs and Mocks allow you to create and test against foreseen conditions. While TDD can help expose bugs, unforeseen data conditions can pass though unnoticed, tests only test what they are told to.
Something else to keep in mind is behavior and unit tests are only a small part of the testing tool box that also includes: integration tests, performance tests, automated UI tests, etc. and that’s what these are tools. How effective they are depends on the user, a funny quote I once heard is “A tool runs tools, a craftsman uses them”. You can run your code coverage tool all day long and say “look I have 80% code coverage” but this doesn't mean your code is well tested, it just means it has test that run though it, and can hide untested code.
One commenter on the article said something to the effect that what they where working on requires to much performance to be testable. I would like to state this is B.S. you can make testable code that has just as much performance as not testable code. You may have to change how you test the code, or use different design or testing techniques for example Dependency Injection isn't required to make testable code, if it's not fast enough use greedy constructors or lazy load properties. If you need to make a black box class for performance, you simply do integration tests around it.
In the end, saying TDD isn’t of value because it didn’t catch bug xyz is like a carpenter blaming his hammer for hitting his thumb and not the nail. On the other side it’s just as ridiculous to say “I can build a house” simply because you own a hammer and some nails.
Tuesday, January 8, 2013
Wednesday, October 17, 2012
Keep contention out of collaboration
There is an old joke “arguing with a developer is like wrestling with a pig in mud, after a while you realizes he/she likes it”, and there are a number of reasons for this, as a group we tend to have strong options, stubbornness, know-it-all, and ego to name a few. Some of this is what makes us good developers, a lot of it is what makes us irrigating to other people.
I personally have been in my fair share of heated, angry, and down right nasty 'discussions' and in the past I may have been guilty of using the force of my will to get teams I have been on to do things 'The right way'. Even joking about being the “a-hole developer”, and as I matured as a professional, I realized even if I win like this, at what cost? I began to realize it was beginning to effect my health and that I didn't like who I was becoming.
We talk about the need to collaborate, from pair programming, sprint planning, daily stand ups, retrospectives, etc. but how often do they turn into arguments? Recently in the Javascript world there has been a lot of arguing about semicolons, and if they should be required. In all honesty this is really a non-problem, and yet people are getting brutal and down right offensive, over the need to be right? Being contentious will not change anyone’s mind, if there is a “winner” it’s because the other side has given up, and at what cost? Both sides are going to be resentful of the other, and it’s going to make working together that much harder.
This is not to say we can’t have discussions from different points of view, have different opinions, be a pushover, or not think the other side is off their rocker. Far from it, but we need to keep it civil and positive. Last I checked we (as developers) are paid to solve problems, not prove we are right. The heart of collaboration is taking different points of view and coming up with the best solution and we can’t do that if we are trying to prove we are right.
It’s ok to have strong opinions, I have more then a few, but it is also important be open minded, be willing to look at what the other side is saying, and come to an acceptable resolution on how to solve the problem. Admitting when you are wrong, or at least not completely right, can be very hard. Ultimately, be willing to work as a team, and if the team decides to do things “wrong” then that’s what the team wants. If you are working with a team that insists on doing things “wrong” and you are unable to contribute, then maybe this isn’t the team for you “If you can’t change where you work, change where you work”.
Wednesday, June 27, 2012
Running your test while you write them using NCrunch
A coworker told me about watching a presentation where the presenter used NCrunch, so I decided to take a look at it.
The basic idea is NCrunch automatically runs your tests in a parallel process giving you continuous testing that’s integrated into Visual Studio. It intelligently runs tests automatically so that you don't have to, and gives you useful information including code coverage and performance metrics while you work.
Here is an example of the inline test results
here is a screenshot of the test runner
and this what the coverage looks like
it’s not quite as detailed as I would like, but not bad.
This isn’t going to be a replacement for my copy of Resharper and DotCover but at the same time, I don’t see a problem with running both. This could defiantly save me a lot of time waiting for test to run and if I want more detailed information I can still use dotCover.
This is a free tool that so far has really impressed me, for more information go to the NCrunch home page athttp://www.ncrunch.net/
The basic idea is NCrunch automatically runs your tests in a parallel process giving you continuous testing that’s integrated into Visual Studio. It intelligently runs tests automatically so that you don't have to, and gives you useful information including code coverage and performance metrics while you work.
Here is an example of the inline test results
here is a screenshot of the test runner
and this what the coverage looks like
it’s not quite as detailed as I would like, but not bad.
This isn’t going to be a replacement for my copy of Resharper and DotCover but at the same time, I don’t see a problem with running both. This could defiantly save me a lot of time waiting for test to run and if I want more detailed information I can still use dotCover.
This is a free tool that so far has really impressed me, for more information go to the NCrunch home page athttp://www.ncrunch.net/
Monday, June 25, 2012
Testing for execution sequence
I ran into a problem the other day where I was doing some cost calculations and wasn’t getting the expected result, the problem turned out to be I was doing the calculations before everything got loaded, simple mistake.
Thought it would be interesting to show the solution I used for testing, lets say we have a simple method for updating an employee record.
fairly simple, you call AuthRequests to see if the user and update the employee and then you update the employee. Here is an example test class for it
the only problem is the tests still pass if the method looks like this
so lets add some sequence tests, basically we add a counter and a dictionary<string,int> to record what was executed and it what order using the Moq callback.
for the full source see the sample application here.
Thought it would be interesting to show the solution I used for testing, lets say we have a simple method for updating an employee record.
1: public bool UpdateEmployee(EmployeeDto updateEmployee)
2: {
3: var result = false;
4: if(AuthRequests.UserCanUpdateEmployee(User.Id,updateEmployee.Id))
5: {
6: result = EmployeeRepository.UpdateEmployee(updateEmployee);
7: }
8: return result;
9: }
1: [TestFixture]
2: public class When_Updating_Employee_Information_Test: Test_Context<EmployeeRequests>
3: {
4: public Mock<IAuthRequests> AuthRequestsMock;
5: public bool ExpectedAuth;
6: public int UserId;
7: public int EmployeeId;
8: public bool Actual;
9: public bool Expected;
10: public EmployeeDto UpdateEmployee;
11: public EmployeeDto SentEmployee;
12: public UserDto ExpectedUser;
13: public Mock<IEmployeeRepository> MockEmployeeRepository;
14:
15: public override void Context()
16: {
17: base.Context();
18: UserId = 12;
19: EmployeeId = 14;
20: Expected = true;
21: ExpectedAuth = true;
22: UpdateEmployee = new EmployeeDto{Id = EmployeeId};
23: Target.User.Id = UserId;
24: AuthRequestsMock = new Mock<IAuthRequests>();
25: MockEmployeeRepository = new Mock<IEmployeeRepository>();
26:
27: AuthRequestsMock.Setup(x => x.UserCanUpdateEmployee(UserId, EmployeeId)).Returns(ExpectedAuth);
28: MockEmployeeRepository.Setup(x => x.UpdateEmployee(It.IsAny<EmployeeDto>())).Callback<EmployeeDto>(
29: x => SentEmployee = x).Returns(Expected);
30:
31: Target.AuthRequests = AuthRequestsMock.Object;
32: Target.EmployeeRepository = MockEmployeeRepository.Object;
33: }
34:
35: public override void Because()
36: {
37: Actual = Target.UpdateEmployee(UpdateEmployee);
38: }
39:
40: [Test]
41: public void Calls_AuthRequests_UserCanUpdateEmployee_Test()
42: {
43: AuthRequestsMock.Verify(x=>x.UserCanUpdateEmployee(UserId, EmployeeId), Times.Once());
44: }
45:
46: [Test]
47: public void Calls_EmployeeRepository_UpdateEmployee_Test()
48: {
49: MockEmployeeRepository.Verify(x => x.UpdateEmployee(It.IsAny<EmployeeDto>()), Times.Once());
50: }
51:
52: [Test]
53: public void Sends_expected_Emplyee_test()
54: {
55: Assert.AreEqual(UpdateEmployee.Id, SentEmployee.Id);
56: }
57:
58: [Test]
59: public void Returns_Expected_Result_Test()
60: {
61: Assert.IsTrue(Actual);
62: }
63: }
1: public bool UpdateEmployee(EmployeeDto updateEmployee)
2: {
3: var result = EmployeeRepository.UpdateEmployee(updateEmployee);
4: AuthRequests.UserCanUpdateEmployee(User.Id,updateEmployee.Id);
5: return result;
6: }
1: [TestFixture]
2: public class When_Updating_Employee_Information_With_sequence_Test : Test_Context<EmployeeRequests>
3: {
4: public Mock<IAuthRequests> AuthRequestsMock;
5: public bool ExpectedAuth;
6: public int UserId;
7: public int EmployeeId;
8: public bool Actual;
9: public bool Expected;
10: public EmployeeDto UpdateEmployee;
11: public EmployeeDto SentEmployee;
12: public UserDto ExpectedUser;
13: public Mock<IEmployeeRepository> MockEmployeeRepository;
14: public int Counter;
15: public Dictionary<string, int> Sequence;
16:
17: public override void Context()
18: {
19: base.Context();
20: UserId = 12;
21: EmployeeId = 14;
22: Expected = true;
23: ExpectedAuth = true;
24: UpdateEmployee = new EmployeeDto { Id = EmployeeId };
25: Counter = 0;
26: Sequence = new Dictionary<string, int>();
27: Target.User.Id = UserId;
28: AuthRequestsMock = new Mock<IAuthRequests>();
29: MockEmployeeRepository = new Mock<IEmployeeRepository>();
30:
31: AuthRequestsMock.Setup(x => x.UserCanUpdateEmployee(UserId, EmployeeId))
32: .Callback(() => Sequence.Add("AuthRequests.UserCanUpdateEmployee", Counter++)).Returns(ExpectedAuth);
33: MockEmployeeRepository.Setup(x => x.UpdateEmployee(It.IsAny<EmployeeDto>()))
34: .Callback<EmployeeDto>(x =>
35: {
36: SentEmployee = x;
37: Sequence.Add("EmployeeRepository.UpdateEmployee", Counter++);
38: }).Returns(Expected);
39:
40: Target.AuthRequests = AuthRequestsMock.Object;
41: Target.EmployeeRepository = MockEmployeeRepository.Object;
42: }
43:
44: public override void Because()
45: {
46: Actual = Target.UpdateEmployee(UpdateEmployee);
47: }
48:
49: [Test]
50: public void Calls_AuthRequests_UserCanUpdateEmployee_Test()
51: {
52: AuthRequestsMock.Verify(x => x.UserCanUpdateEmployee(UserId, EmployeeId), Times.Once());
53: }
54:
55: [Test]
56: public void Calls_AuthRequests_UserCanUpdateEmployee_Inorder_Test()
57: {
58: Assert.AreEqual(0, Sequence["AuthRequests.UserCanUpdateEmployee"]);
59: }
60:
61: [Test]
62: public void Calls_EmployeeRepository_UpdateEmployee_Test()
63: {
64: MockEmployeeRepository.Verify(x => x.UpdateEmployee(It.IsAny<EmployeeDto>()), Times.Once());
65: }
66:
67: [Test]
68: public void Calls_EmployeeRepository_UpdateEmployee_Inorder_Test()
69: {
70: Assert.AreEqual(1, Sequence["EmployeeRepository.UpdateEmployee"]);
71: }
72:
73: [Test]
74: public void Sends_expected_Emplyee_test()
75: {
76: Assert.AreEqual(UpdateEmployee.Id, SentEmployee.Id);
77: }
78:
79: [Test]
80: public void Returns_Expected_Result_Test()
81: {
82: Assert.IsTrue(Actual);
83: }
84: }
Wednesday, June 20, 2012
Behavioral testing with NUnit
The traditional unit test works great for testing code functionality but another important testing aspect is testing code’s behavior and you don’t need to use a behavior testing framework like mspec to do it, with the right pattern nunit, mstest, and xunit work just fine.
A traditional TestFixture might look something like this
Having a fairly large TestFixture with a lot of tests for the different functions of the method being called.
When doing behavior testing the idea is to test a code path for a specific context vs. testing for all possibilities a method could result in, basically breaking it down into much easier to read parts.
To do this we set up a base or abstract class that has a context method for setting up your test and a because method for running it, here is an example of one using generics
next we set up the the test fixture
a couple of things you may notice is first the the name is fairly long and descriptive, the idea is you want to make it easy to read the test results to know what your testing, in this case requesting employees from Seattle, WA.
The next thing you may notice is we don’t have as many test methods, sense we are only testing for what happens when we are getting employees from Seattle, WA. that’s all we are testing for. This may produce more test textures classes but at the same the test fixtures are more targeted and give a better description of what you are testing making it easier to read/maintain later and is sometimes called executable specifications, where you can find out the intended function of a class or method simply by reading then names for the test fixture and tests. For more examples of unit tests vs. behavoir tests see the sample application in the sample code repository.
A traditional TestFixture might look something like this
1: [TestFixture]
2: public class EmployeeRequests_Tests
3: {
4: private EmployeeRequests target;
5: private Mock<IEmployeeRepository> MockEmployeeRepository;
6: [SetUp]
7: public void Setup()
8: {
9: target = new EmployeeRequests();
10: MockEmployeeRepository = new Mock<IEmployeeRepository>();
11: MockEmployeeRepository.Setup(x =>
12: x.GetEmployeeByCityState(It.IsAny<string>(), It.IsAny<string>()))
13: .Returns(new List<EmployeeDvr>
14: {
15: new EmployeeDvr {City = "Seattle", State = "Washington", EmployeeName = "bob"}
16: });
17: target.EmployeeRepository = MockEmployeeRepository.Object;
18: }
19:
20: [Test]
21: public void ReturnsErrorForEmptyCity_Test()
22: {
23: target.GetEmployeeByLocation(string.Empty, "Washington");
24: var actual = target.Errors.Where(x => x.Contains("Invalid City"));
25: Assert.AreEqual(1,actual.Count());
26: }
27:
28: [Test]
29: public void ReturnsErrorForEmptyState_Test()
30: {
31: target.GetEmployeeByLocation("Seattle", string.Empty);
32: var actual = target.Errors.Where(x => x.Contains("Invalid State"));
33: Assert.AreEqual(1, actual.Count());
34: }
35:
36: [Test]
37: public void VailidCityStateCallRepository_Test()
38: {
39: target.GetEmployeeByLocation("Seattle", "Washington");
40: MockEmployeeRepository.Verify(x => x.GetEmployeeByCityState("Seattle", "Washington"));
41: }
42: }
When doing behavior testing the idea is to test a code path for a specific context vs. testing for all possibilities a method could result in, basically breaking it down into much easier to read parts.
To do this we set up a base or abstract class that has a context method for setting up your test and a because method for running it, here is an example of one using generics
1: [TestFixture]
2: public abstract class Test_Context<T> where T : new()
3: {
4: public T Target;
5:
6:
7: [SetUp]
8: public void Setup()
9: {
10: Context();
11: Because();
12: }
13:
14: public virtual void Context()
15: {
16: Target = new T();
17: }
18:
19: public abstract void Because();
20: }
1: [TestFixture]
2: public class When_Requesting_Employees_From_Seattle_Washington_Tests : Test_Context<EmployeeRequests>
3: {
4: public Mock<IEmployeeRepository> MockEmployeeRepository;
5:
6: public override void Context()
7: {
8: base.Context();
9: MockEmployeeRepository = new Mock<IEmployeeRepository>();
10: MockEmployeeRepository.Setup(x => x.GetEmployeeByCityState("Seattle", "Washington"))
11: .Returns(new List<EmployeeDvr> { new EmployeeDvr { City = "Seattle", State = "Washington" } });
12: Target.EmployeeRepository = MockEmployeeRepository.Object;
13: }
14:
15: public override void Because()
16: {
17: Target.GetEmployeeByLocation("Seattle", "Washington");
18: }
19:
20: [Test]
21: public void HasNoErrors_Test()
22: {
23: Assert.AreEqual(0,Target.Errors.Count);
24: }
25:
26: [Test]
27: public void Calls_The_Repository()
28: {
29: MockEmployeeRepository.Verify(x=>x.GetEmployeeByCityState("Seattle","Washington"),Times.Once());
30: }
31: }
The next thing you may notice is we don’t have as many test methods, sense we are only testing for what happens when we are getting employees from Seattle, WA. that’s all we are testing for. This may produce more test textures classes but at the same the test fixtures are more targeted and give a better description of what you are testing making it easier to read/maintain later and is sometimes called executable specifications, where you can find out the intended function of a class or method simply by reading then names for the test fixture and tests. For more examples of unit tests vs. behavoir tests see the sample application in the sample code repository.
Subscribe to:
Posts (Atom)
