Workshare’s philosophy is one of an open culture that facilitates communication and learning. Our staffs are intelligent beings, allowed to manage their own activities on a day-to-day basis. This allows management to focus on the removal of barriers that may inhibit their success, as well as coaching and mentoring them in their skill developments during each project. Additionally, Workshare runs an internal professional development programme of self-assessment and staff are continually encouraged to improve their knowledge and experience in and of the work place. Workshare truly considers itself to be a learning organization that places a high value on teamwork. It is this combination of culture and value on teamwork, which sets Workshare apart from our competitors.
Workshare’s market is the innovative and highly technical document change management arena. Because Workshare has unique products and is in the fast paced and constantly changing marketplace, our design and functional specifications need to be extremely flexible. Almost as fast as they could be written, specifications became outmoded and the most recent changes existed only in the heads of few staff. Albeit, our methods were agile, they were no longer scaleable to the growing size and needs of the engineering team and suffered from several common problems:
- Development was ‘bug’ driven rather than being driven by functional requirements.
- Programming knowledge was not evenly shared, we had too many specialists with too little time to train new staff
- Although we had unit tests, these were the exception rather than the rule
- We were building frequently but putting the burden on QA to spot errors and defects within code rather than building quality in.
Workshare needed a methodology that could be used to rapidly deploy software, improve communication between all departments, and be scalable and fit into the Workshare’s culture and philosophy. Extreme Programming (XP) satisfied all of these requirements and became Workshare’s methodology of choice.
Benefits of Extreme Programming (XP)
A metaphor, according to Webster’s New World Dictionary, is a figure of speech in which one thing is spoken of as if it were another, as “all the world is a stage.” The metaphor Extreme Programming (XP) uses to describe software development is of driving a car. When driving, you do not just point the car directly at your destination and drive as fast as you can until you get there. Instead, you proceed by continually adjusting your speed and direction depending on the current conditions and road layout. Sometimes, you might even drive in the wrong direction for a while - if it will eventually get you to the destination quicker. This process approach can be described as iterative and incremental and is the metaphor for the XP approach to the delivery of quality software.
In XP, the development cycles are short with small chunks of functionality so it is always possible to know what has been done and what actually remains to be done. This is one of main benefits of XP, the visibility of the software being developed. Additionally, XP improves software projects through the use of four essential practices;
At Workshare, our programmers communicate with their customers, fellow programmers and QA testers; keep their design simple and clean by testing their software starting on day one and deliver the system to the customers as early as possible. Because of this foundation, our programmers are able to courageously and quickly respond to changing customer requirements and software technology.
There are four measurable variables in any software project. They are:
- Scope what is done and what remains to be done
- Quality correctness and other such measures
- Resources personnel, equipment, space
- Time duration of the project
If any three of these variables are chosen, we can calculate the fourth. Normally, customers define scope and quality, and what resources are given. We use planning meetings to figure out how long the project will take. It must be explained, that in XP there are two major roles, that of the programmers and that of the customer. The customer role is defined as a “… role on the team for choosing what stories the system was to satisfy, what stories are needed first and what can be deferred and for defining test to verity the correct functioning of the stories.” The customer may be either external or internal to the project. Both programmers and customers are interactive throughout the duration of the project, especially when involved in the planning meetings.
At Workshare, we have a department within Engineering dedicated to satisfying the needs of our external customers and taking the customer role in the XP process. Product Management is the centre of our software development universe. Every functional area interfaces with the Product Management group, depending on where the product is in its lifecycle. The Product Management group is responsible for:
- Product direction
- Collating stories from all sources
- Deciding which stories should go into the pool of stories for the next release
- Allocating priorities to the stories
- Each XP story is the responsibility of an individual Customer.
- Developing acceptance tests for the stories in conjunction with QA
The Customers must be available 100% of the time. If they are in a meeting when we need them, we go into the meeting and pull them out. If the Customer is not available then someone else from the Product Management department will take responsibility for answering any queries. The customer's role is strengthened in XP; they control design, new features and priorities. With advice from QA, they also decide which defects are showstoppers and determine the priority for fixes.
Under XP the Customers have rights:
- Customers have the right to an overall plan, to know what can be accomplished, when, and at what cost.
- Customers have the right to see progress in a running system, proven to work by passing repeatable tests that are specified in consultation with QA.
- Customers have the right to change their mind, to substitute functionality, and to change priorities.
- Customers have the right to be informed of schedule changes, and to choose how to reduce scope to restore the original date. The Customer can even cancel at any time and be left with a useful working system reflecting investment to date.
At a planning meeting, customers define scope with user stories. They write each story (feature) on a separate index card. The story is written plainly and briefly describes the feature that the customer wants. The card becomes a notation for the programmers and should encompass from one day to two- weeks of development time. Customers also define quality, by defining the functional tests which the system must accomplish before it can be released. These tests are known as acceptance tests and can be written on the index card of the story. In XP, the customers have complete control over the system quality.
Time or Duration, in XP projects, is known as the release-cycle. Each release contains defined areas of new business functionality. At Workshare, our release cycle lasts around three months. Of course, three months is much too long a period to be able to plan accurately in advance, so we divide each release into two-week iterations. Within each iteration, the requirements for a new functionality are divided into smaller units known as user-stories.
Workshare’s Installation of Extreme Programming
Installing the Extreme Programming methodology involved wholesale changes in work practices across Engineering here at Workshare. Being a software house, we are fortunate that though our senior management is technically experienced, they are young, willing to try new ideas and embrace change. This culture helped the transition tremendously.
We recognized that we needed outside help to get us started and began our search. We quickly identified Object Mentor as the premiere Extreme Programming Consultancy. Several of our key members of staff attended Object Mentor’s Extreme Programming Immersion, after which negotiations began.
Object Mentor’s role was as trainer and mentor for our teams during the transition to XP. Several of their experienced mentors worked side-by-side with our teams for many weeks while we moved through several XP development iterations. Mentoring was given to both the programmers and our customer so that the integration was as smooth as possible.
1. Extreme Programming Practices – On-site Customer
At Workshare, we have a department in Engineering dedicated to satisfying the needs of our external customers. Product Management liaises with clients, sales, QA and support to gather and determine requirements. Within Extreme Programming, the role of the customer is strengthened; they control design, new features, and priorities. With advice and guidance from QA, they also decide which defects are showstoppers and give priority to fixes.
2. Extreme Programming Practices – Planning Game
In the planning game Customers and Developers interact to decide what work will be completed in the next iteration. The Customers provide the requirements in the form of ‘user stories’ and developers break these stories into individual tasks, typically lasting less than a day or two. They then provide estimates for how long it will take to implement each story based on the duration of the tasks. The budget, for each iteration, is decided by how much work is completed in the previous iteration; this is called team’s velocity. The customers then decide which stories to include in the current iteration by balancing priorities and durations to fit that budget. The iteration usually lasts two weeks.
3. Pair Programming
Pair programming is a process whereby two programmers share a machine with one keyboard and monitor between them and code together. All code is therefore subject to continuous peer-review and inspection. Detractors might say that his means that we only develop half as fast as we could but truth is that a developer’s role is that of a problem solver, not a typist and problems get solved twice as fast when two heads are involved. Two pairs of eyes means less errors slip through the net. Alistair Cockburn and Laurie Williams published a research paper, “The Costs and Benefits of Pair Programming”, that claims a 70% reduction in the defect rate when programmers worked in pairs. Workshare experience is similar. Pair programming also propagates knowledge throughout the department and allows new recruits to come up to speed very rapidly.
4. Test First Design
Before we write code, we write a test for the code. Obviously, when we run this test for the code, it fails because we haven’t written the code yet. This failure proves to us that the test is working. After we’ve proved that the test works, we can write the code that makes the test pass. In this way, we know with certainty, that the code is correct, and we have only written the code needed to fulfill the requirements and there is no ‘feature creep.’ Quality is built into the code at unit level. A side effect of this is that the test code also contains the specification for the requirement. Any programmer can look at the tests and see what the code is supposed to do. If the requirements change, first we change the test, again making them fail, and only then do we change the code to make the tests pass.
Refactoring is improving the design of existing code. This is in opposition to traditional methodologies where design comes first and coding comes afterwards. With refactoring, rather than occurring upfront, design occurs continuously during development. The result is a program with a design that stays good as development progresses. Refactoring also means keeping the design as simple as possible and building clarity into the code so that the code itself conveys meaning. It is said that developer read five times as much code as they write, so the easier it is to read, the easier it is to maintain.
6. Coding Standards
Workshare coding standards mean that all code is written to the same set of rules, facilitating communication through code.
7. Collective Code Ownership
Because pair programming disseminates knowledge throughout the department, we don’t suffer from the ‘guru’ problem where one staff member becomes and expert and when he/she is not available, work stops. Additionally, we practice refactoring to make the code understandable by any developer. The unit tests on the code also allow any developer to work on the code with confidence, safe in the knowledge that if he breaks anything the tests will fail and show him where it is broken.
8. Continuous Integration
When the developer has finished writing a story it is integrated only after it has passed all the unit tests. Once they have integrated their code, it is run through our automated pre-QA regression tests. If there are any problems, the integration is rolled-back and the developers have immediate feedback on the problem. Having passed pre-QA, the code then goes into the main code base ready for the next build.
Workshare builds our software automatically twice a day and, if required, can build manually at a moment’s notice. The latest build is always available to release as a working prototype and contains only code that has passed all of the unit tests and the pre-QA screening. This allows the customer to have instant feedback as to how his changes have been implemented. Following a build, the code progresses to QA for the entire range of testing.
Under XP developers have rights:
- They have the right to know what is needed, via clear requirement stories, with clear declarations of priority.
- They have the right to say how long each story will take to implement, and to revise estimates given experience.
- They have the right to identify risky stories, to have them given higher priority and to experiment to reduce risk.
- They have the right to produce quality work at all times. They have the right to peace, fun and productive enjoyable work.
XP itself does not define any QA practices but at Workshare, we have always focused on the quality of our products and so have defined our own set of XP QA practices.
1. Automated Testing
Testers work with customers and developers to write acceptance tests for each story. Defining an acceptance test helps to focus the customer on the need for the story and the story that cannot be tested is an invalid story.
2. Testing the automated buildsDaily QA is made possible in XP since you are only testing the impact of adding that day’s integrated stories. If a bug is found which fails a story, it is reported immediately to the customer and developer. It is then referred back to the developer or added to the Daily QA Report and reviewed to ensure the same bugs are not repeated.
Since XP does not define any rights for testers, we at Workshare have defined our own.
- They have the right to see progress in a Running System, proven to work by passing repeatable tests that they run and specify in consultation with the customer and developer.
- They have the right to be consulted throughout the XP planning process, to know what is needed, with clear declarations of priority and to work to release dates, which take account of the testing process, and the additional issues found during testing.
- They have the right to produce quality work at all times and seek assistance and support from customers, developers and mangers to achieve this.
- They have the right to warn or reject a build which prevents the customer testing or ‘telling the story’, to fail a story which does not pass the customers acceptance tests and to recommend that a specific build is not released.
- They have the right to be heard, for QA reports to be read and acted upon and to seek improvements across the XP Engineering process, which aim to improve software quality.
3. Project tracking using Workshare BlueSky
Of course we don’t just rely on face-to-face communication, although that is our main channel. We have a tracking system that runs on our intranet called Blue Sky. Built in-house and tailored exactly to the needs of the Engineering department, BlueSky enables us to track the progress of each project from release planning right down to the individual tasks that make up one story. All the data is kept on file and is updated continuously and is always available for analysis.
Workshare’s market is in a fast paced, highly technical document comparison arena and therefore our design and functional specifications need to be extremely flexible. We know that even though we had been using agile methods, they were no longer scaleable and we needed a methodology that could be used to rapidly deploy software, be scaleable and fit into Workshare’s culture. Extreme Programming (XP) satisfied all of these requirements.
Workshare understood, from the beginning, that the installation of the Extreme Programming methodology would involve wholesale changes to our work practices – and it did.
1. Transition [Figure 1]
Figure 1, shows two years worth of data, from March 2000 to February 2002. Over the top of the graph, I have mapped the Virginia Satir model of change as explained in Peopleware. I chose this model because this is how the staff at Object Mentor explained the process of change that we would go through when we attended the XP Immersion in February 2001.
The data contained in the graph represents defects caught by our QA department on a weekly basis, and the stars represent the release dates of upgrades to DeltaView, an existing product. You can quite clearly see that the left half of the graph, which represents our old waterfall process, is a chaotic system where releases are signified by high QA activity and a large number of defects found.
Following our visit to the training facility in Vernon Hills, Chicago, Illinois, we brought back two Object Mentor programming coaches to train the development team and a business coach to train the customers and testers. The attendees of the immersion assisted the coaches in the transition process.
The XP transition took place over three months and, at the end that period, the figures looked fairly depressing but we had been warned to expect this. In the Satir model, this is called the chaos period where we were desperately trying to learn new techniques and procedures while, at the same time, trying to go about our primary aim of producing software. It is no wonder we were producing as many defects as ever. However, we did manage to release a brand new product, Synergy.
2. Flying Solo [Figure 2]
Following the transition, we spent a further three months ‘flying solo’, refining our new-found processes and techniques, putting them into practice, getting comfortable with them and producing a patch release for DeltaView along the way. During this period we had many meetings to establish and formalize our overall methodology. Having done that we now find that our defect rate has stabilized to between 20 and 40 defects per week, with only a slight raise for our ‘release spikes’. [Figure 2] The two dips in the defect rate during this period are due to Christmas week and another week in February that we took off of production to transition to a new development platform.
The dotted line on the graph is the linear regression (trend) line for the defect rate over the two-year period. The improvement shown for the one-year period from the start of the XP transition is much, much steeper.
The previous graph showed how, over two years, Workshare’s development department dramatically improved its defect rate. What the graph didn’t show was that, at the same time, the development department was expanding quite rapidly. From nine developers in March 2000 to 25 developers in February 2002. When the data in the previous graph is normalized to take account of the number of developers, the results are even more stunning. We can see that at the beginning of the graph, the department was averaging just fewer than six defects per developer per week, while at the end of the measured period; the department was averaging less than one defect per developer per week.
Look how steep that trend line is! That’s not all though, the development department wasn’t the only department to expand. Additions were also made to the QA department to improve their performance and keep their resources in line with those of development. In March 2000, QA at Workshare Technology consisted of three testers performing manual testing of the products. Now there are nine testers and a lot of testing is now performed automatically using scripts developed for QARun. In addition, Workshare has released Synergy, thus doubling the complexity of the code. We now support more operating systems and integrate with more document management systems, again increasing the complexity of the code. With all of these changes, you would think that QA would be finding a lot more defects but they’re not, it appears that development have stopped producing them.
What these figures represent in real terms is a massive boost in productivity. There are various figures bandied about in the industry but we estimate that, prior to XP, we were expending approximately 70% of our development effort on fixing bugs, which means only 30% on new development. If we’ve reduced our defect rate by about 80% (look at the graphs) then we are now only expending 14% of our effort on defects and 86% on new development. This represents almost 3 times the productivity. A massive boost indeed!
4. Further Analysis [Figure 3]
Of course, it could be that despite all the improvements in QA, we were simply just shipping a lower quality product and the customers were doing the testing for us. Or perhaps we were losing our customer base and hence not feeling the effects of shipping lower quality. Further analysis shows that this is not the case. The number of support calls taken per week can clearly be seen to be a function of the number of current users. In fact, the trend for support calls spookily matches the trend for the number of users. [Figure 3]
It would have been nice to have mapped this graph across the same two-year period as the previous graphs but unfortunately, due to a change in systems during this period, the earlier data is unavailable. The big dip in December is the Christmas week.
The breakdown of the support figures shows us that the main area of increase in support calls is related to integration with document management systems. We will obviously be making this a target for improvement. Other areas of concern are installation and configuration, which we will also be attempting to improve with the emphasis on usability.
So defects are down, down, down. Support calls are at an expected level and the user base is expanding.
- Is there anything else we can be smug about?
- Well what about customer satisfaction?
It is all very well producing less defects and receiving no more than an expected number of support calls but what about the customers that do have problems, how good is the service we provide to them? Again we have data we can look at, albeit only from May 2001.
5. Support Call Analysis [Figure 4]
Figure 4 shows us the average time it took to resolve a support call received in a particular week. On the left of the graph you can see that calls received the week of the 2nd Jun 2001 took an average of nearly 80 days to resolve.
A reasonable average for the beginning of the graph would be over 40 days by my reckoning. Recognizing that the data at the end of the graph may be somewhat misleading due to the existence of a greater number of unclosed calls, I think we can still put an average of less than 20 days for the end of the graph. This equates to more than a twofold improvement in the time it takes us to resolve a customer issue. A trend line drawn here would also show that this figurecontinues to improve.
Simply put, Workshare needed a methodology that could be used to rapidly deploy software, improve communication between all departments, be scalable and fit into Workshare’s culture and philosophy. Extreme Programming (XP) satisfied all of these requirements and has become Workshare’s methodology of choice.
 Kent Beck, eXtreme Programming explained – Embrace Change. Addison-Wesley, 2000, page 177