Normally, I would write this in my SCN blog – but this is not about just SAP projects, I am going to do it here. As always, these are strictly my personal opinions – not that of my employer.
Some of you have seen me – at work, at some tech conference, dog show or at an airport. I doubt if “Agile” was the word that came to your mind when you met me. I could easily lose 20 pounds and have some one ask me “hey, why are you not doing something about your weight?”. You get the idea. I have tried many a weight loss / excercise plan – and have come to the firm conclusion that the magic bullet for weight loss is to eat less of everything I like (Rice, red meat, fried food) and excercise more.
In parallel, I was going through a similar excercise at work – trying to find an optimum way of managing the projects I get to run. I have read and tried several different things over the years in a variety of projects. Over the last couple of years, I have been fascinated with Agile development and hence I have been reading, talking to others, and trying it out in teams I manage – and again, I have come to the firm conclusion that, just like the various dieting schemes – Agile also does not work for me.
This is not to say Agile does not work for others . South beach diet must have worked for others, and I am sure Agile would too. For me – no sir, I will pass. That being said, let me explain why it does not work for me.
Would you pay an Agile contractor to build your deck?
I need a deck built, and I don’t know how to build one. So I hire some one else to do that. And this dude tells me he can do it one of two ways. 1. I can discuss with him on how a basic deck has to be built, and he can give me an estimate. Or 2. I can give him a rough idea, and he will start building the deck, and every day or two – he and I can get together to see how it is going and what changes I want, and I can pay him for work that he has done every day. Of course in the second option, he cannot tell me how much the deck will cost me or how long it will take him to build me one – but I can see progress every day. I don’t know about you – but I know what option I will go for. Same thing with my clients – if the work and schedule are not predictable – it is hard for them to just pay as I go.
Global and Agile are like Oil and Water – They don’t mix.
I forgot the last time I had all the IT and Business guys and gals working in the same location. Most often – we have people working on a project from all over the globe. It is seldom possible to get teams from Japan, India, Germany and US to be on a conference call. And even if you do – without a written document explaining the problem and/or solution – it is hard to get anything done.
There are only so many super stars in this world
In any given team, the norm is to have a few super stars, several average performers and a few below average ones. This has a direct effect on pulling off delivery in an agile fashion. Not every one can go away with minimal instructions and come back with the right solution, and right questions to ask for next day.
It can be argued that Agile needs less programmers, and hence you can keep just the super stars and let go of every one else. This argument works only if the scope is small. The day still has only 24 hours – and even super stars cannot work 24X7 all the time to get everything done.
Most projects do not have dedicated business users.
The whole idea of Agile is to get something back to business users faster than in a waterfall model, and keep them informed of progress frequently. This is very good – except it won’t happen in most projects. Most companies find it hard to dedicate full-time business people to a project. More often, business users have to do project work on the side. So – even if the tech team wants constant face time , the chance of that working out is low.
A few product companies have tried out Agile successfully. However, in the couple of cases where I got a chance to talk to people from the team – it seems, the customer was almost never present in the scrum meetings. Instead, the product manager assumed the role of being the customer voice. If that is the case, I would have to wonder aloud “so what is new?”. Product manager is not the one who has to live with the product after it is out – it is the customer.
What works for you – people over process? or process over people?
This is what it boils down to – Agile manifest claims the superiority of people over process. And traditional waterfall puts process over people. For me – as a manager of big teams, with a deadline and budget that seldom cuts me any slack – I would trust a good process to compensate for the human errors most of the time. I think it is a high risk for me to trust that every one in the team will perform to the same high standards. Having a disciplined process helps me get the best out my team, despite not every one being a super star.
Would you build a mission critical solution using Agile?
I don’t know – but I keep wondering if NASA would use Agile for designing systems for their next space mission? I somehow can’t see any application that is very important and deals with life and death, or dealing with large amounts of money – like air traffic control or stock market transactions – to use Agile.
I hope every one knows about 3C. This was the big Chrysler project that was the poster child for XP (Extreme programming). Well, guess what – that didn’t quite work out. Check this out http://en.wikipedia.org/wiki/Chrysler_Comprehensive_Compensation_System or just google and you will find several interesting takes on it.
Building a car and building a software solution have similarities in design. However, there are significant differences too. Just as it is impossible to build a car without knowing what exactly needs to be built – we cannot build a good software solution without knowing what the heck we are building. Otherwise, even if you follow Toyota manufacturing Process to build a Camry – you could end up with a Chevy Malibu.
Is Agile really good for long-term stability of a solution?
Most software is built by first putting a framework in place, and then building on top of it. It is the rough equivalent of putting a good foundation for your house. If you know that you will have a house with 2 floors – you will probably put a certain amount of concrete in your foundation. Now that you have finished the house – and for some reason, you now want one more floor – would you put one more floor without also doing some additional foundation work? And if you build by constantly messing with your foundation – I am definitely not going to buy that house or rent it for living. Same thing with software – the way sprints happen from what I have seen, I doubt if it is possible to put a solid foundation in place.
It is fun, but is that good enough for customer to pay?
One thing I really like about Agile is that every one involved in it usually has more fun than in a waterfall project. This improves team morale and all the good stuff – temporarily. When you cannot get user involvement, or if a blame game starts – where there is no documentation to go back and clarify what every one agreed to, this fun does not always last. And fun for the development team, while important in a project, is not the sole reason why some one pays for a solution.
Waterfall is not such a terrible thing to do, as its opponents make it to be. And it is a big exaggeration to say the team does a very long design up front, and that user gets to see things only at the end. That is not how most projects run. Waterfall can also have users involved more frequently. Also, you can build in plenty of feedback options and test driven development in a waterfall project. Also there are very strong visualization tools that can give the users a taste of the solution very early in the process. For example – iRise is a great tool to use for that. Do try it out, and your whole perspective will change. Also, once you have a good change control process established, changing requirements can be handled very effectively.
So, when do I think Agile would work?
Despite all these points above, I am not an extremist when I think of methodologies. In my opinion, Agile is going through a hype cycle exactly like how it happened with SOA. Once the hype died down a little, we mostly figured out what is possible with SOA and what is not. SOA is great for several use cases, and terrible for others. Same is the deal with Agile. Just because Agile possibly has the issues I called out above, it should not be inferred that Agile should not be used. If you have a bunch of highly skilled people, where the team has a clear vision of the end product, and/or where time to market is not such a big deal – all these challenges will vanish. Such a team can probably come out with a great solution using Agile. But in the type of projects I am dealing with – I don’t think Agile can succeed.
Rather than take an extreme view of either/or – it is probably best that we let individual projects decide the methodology they want to follow.
And Vijay Says... 
An odd post, to say the least.
Who would ever get approval to start a project when “Of course in the second option, he cannot tell me how much the deck will cost me or how long it will take”? And how do you tie that to being agile?
For me, an experienced program manager who uses the scrum methodology, Using your deck analogy, I can guarantee you that if my customer provides a rough idea of what they want, and my developers iterate on actual working functionality with the customer, the customer has a significantly higher chance of getting the result they expected vs. spending days upon days upon days designing, discussing, making changes on paper only, etc…and then showing the customer the “completed” product ONLY after its complete. It’s typically at that point, when you think you’re done, that you find out there was a mistake in the drawing regarding the size (if using agile you would have seen this when the holes for the posts were first dug), the color wasn’t exactly what they wanted (vs. seeing that when your first opened the can of stain) and the bump-out for your grill isn’t large enough because you purchased a new, larger grill while the deck was being built.
So, at the start, I can (and am required) provide an estimate for the deck, can provide the customer any changed in the cost based on iterative feedback and he knows during the entire project if any requested changes will impact the cost and schedule.
Versus your waterfall method where its assumed you’ll hit the cost and schedule targets simply because you believe you have the requirements nailed, when in fact they’ve started changing the minute after you declared them complete.
The best quote I ever heard is “If you complete your project and have implemented exactly the requirements you set at the beginning, your project has failed. It simply means you didn’t listen to any feedback the entire time and most likely didn’t implement the product customers actually wanted.”
LikeLike
for the agile methodology itself, what I don’t like is that it neglects the importance of careful planning in advance.
the problem is that making a project work is easy, but adding sth. extra could be very difficult, if not planned well at the very beginning.
For some scenario, re-starting from scratch will be much more cost-time-energy efficient than refactoring in Agile’s way.
The “every route correct”, “every sprint correct” might not necessarily result in the “whole solution correct”.
In other words, with all the local maximum at hand doesn’t mean the global maximum can be definitely obtained.
That is the big problem of agile. Too many under-estimate and neglections are trying to find excuse under the umbrella of agile methodology.
LikeLike
Glad to see a candid, and to some degree, courageous emptying of pockets here. Agile is a valid tool. So is Waterfall. So is Adaptive Waterfall. So is Lean, etc. You get the picture. Pity the person who only has a hammer in their toolbox. To such individuals the world is a nail.
It is unfortunate when some can turn into rabid defenders of a methodology without the foundation of balance or without understanding when to use the appropriate tool to arrive at an optimal solution.
LikeLike
Thanks Randy. I couldnt have summarized it better
LikeLike
Hi,
Myself and Twan van den Broek did a 5 min discussion around why lazy developers hate agile and scrum. The video is available on youtube
http://www.youtube.com/user/sapmentors#p/u/2/09yQ2WzRHTs
Cheers
LikeLike
Wrote a short blog post explaining the background for the lazy developers hate agile and scrum video
http://elsewhat.com/2010/05/20/lazy-developers-hate-agile-and%C2%A0scrum/
LikeLike
An useful review
LikeLike
Hey there! Just wanted to say that this is one well written article! Thanks for posting this. I was looking for a site that has this kind of info and I’m glad I stumbled upon this one. Gotta love the affiliate marketing business 😀 Keep up the great articles.
LikeLike
lmao nice info bro.
LikeLike
Agile ultimately boils down to an issue of trust. If the client and developers trust that neither is trying to exploit the other, Agile can be an ideal solution, as it in turn will ensure that neither side is hard done by. I agree with Kevin’s point towards the top: your decking paradeigma is not as clear-cut as you make out, as the the second option does entail a more bespoke service for the client. The point of Agile is that the client does not end up with a product completely inadequate to their needs, and the developer does not have to rush to complete a project which turns out to take longer than expected.
Of course, there will always be less than honest people out there, but this does not detract from the basic value of Agile methodologies.
I feel it’s pretty clear on which side of the fence I sit, but you can read more about the benefits of Agile here: http://blog.bitzesty.com/what-to-expect-from-an-agile-web-development
LikeLike
hi wat is your myspace page
LikeLike
Hi Vijay,
What an amazing post! Great discussion, with well known peers from the SDN community, on applying agile to the SAP world. This topic just can’t get enough attention.
To start: agile is no silver bullet to all your project risks and problems. Neither is waterfall. The best use of a project methodology is to apply it when applicable.
My personal agile experience in a waterfall oriented company started 1,5 years ago. We started in February and we had a very strict deadline to deliver in September. With a traditional waterfall approach this was just impossible. Why?
– Business design was not finished
– Specifications had to be reviewed by 3 departments (not involved in project)
– A lots of doubts were raised by non SAP people on the SAP solution
So we decided to do it differently: scrum, extended with smart (www.smartusecase.com) from Sander Hoogendoorn. We chose to do so because:
– Intensify collaboration between business and IT delivery
– Anticipate on changing requirements
– Sprints of 2 weeks to get as many feedback moments from business as possible
– Solution which was built to change (in stead of building it to last)
– Discussions with product owner on priorities in stead of deadlines
– Eliminate all the air in the project planning by involving all stakeholders and by that preventing all the long lasting review cycles
Key success factors:
– Solid but flexible solution architecture (SOA) was defined upfront
– Product owner who took his role very seriously
– Good mix in team of consultants
– 2 sprints upfront to model the high level requirements in smart use cases, resulting in the product backlog
– 2 sprints of integration and acceptance testing at the end of the project
– Involvement of application management throughout the project
We succeeded in our first project, delivered on time and within budget. Early this month we started the fourth agile SAP project. Both business and IT are very enthusiastic on the results. But again: agile, scrum nor smart are silver bullet. Apply them when applicable and when you can fulfill all the key components from the methodology.
Kind regards
Twan
LikeLike
I think the “truth” is somewhere in the middle.
I would want a guy who works on a renovation project in my house to give me an estimate.
I would want to see progress everyday and if I don’t like something. That should be discussed and corrected.
If he uncovers something unexpected (a wiring in the wall that nobody knew existed). That should be discussed and if the cost goes up I need to know about it.
I would not want to meet with a guy everyday to discuss how he hammers nails.
I don’t want to jinx this but every renovation project in my house was a success. I only hire constructors who I trust. I follow the above “open” approach. One thing that really helps me is that I know exactly what needs to be done because I’ve done some smaller renovations myself.
I don’t think building software is any different. You just need to know what you are doing so that you are able to recognize when somebody is b.s.-ing you.
-Edward
LikeLike
@EVP Thanks for taking time to comment. The only difference I see between your case and most software projects is that, unlike you – most customers who initiate projects do not have enough experience in it to know if the work is proceeding well or not.
LikeLike
Ed, those are interesting numbers indeed. And I use RPM regularly in my gigs. For the methodology itself – we have the freedom to adopt it to a given project. We have something called Ascendant, which we use for SAP gigs for example. RUP is succesfully done in a lot of projects too that I know, but I don’t remember on top of my head if any of our SAP programs used it. I personally feel (and I think I mentioned it in a comment above to Matt) that RUP has a definite waterfall flavor in terms of having 4 distinct phases.
LikeLike
Wow… terrific discourse on this topic and great content provided by many. Before I comment further, full disclosure. I have been successful using agile (specifically SCRUM) in my old organization and I am a lean management champion at SAP (in addition to my day job in IT).
Something we have heard very little about is the benefit of delivering the very most value in the very shortest time. The agile manifesto does not say it very well, but this concept is in there somewhere. The concept is front and center in SCRUM and it is the number one principal in lean management.
Unfortunately overlaying an agile methodology on a company or project structure not designed for it is a recipe for problems. Furthermore, using an agile methodology with a software architecture and development tools that were not designed for agile will simply add to the issues.
I was only able to deliver value quickly with SCRUM after I had made the changes necessary to support it (org, tools, and architecture). I was able to determine which changes were necessary by looking at the organization from a lean management perspective. You will only find isolated success at best if you simply drop agile into an existing structure without adapting that structure.
In practice, the output of our SCRUM development was both feeding a standard product portfolio and delivering customized versions of that portfolio to customer projects. Same team, multiple tracks. However, it only worked because we had a traditional project management approach used in way that incorporated the SCRUM deliveries. The agile team was managed using the empirical tools of SCRUM and things like installation, roll out, change management, etc. were handled in a more directed fashion.
One thing I was not able to change in the organization was our contracting. We worked on a fixed price, fixed delivery basis with our customers (for the product track, our product manager was the customer). An analyst worked with the customer to prepare a fixed statement of work, which we then committed to deliver. There was a contract change control mechanism in place to handle the inevitable mistakes in the original specification. There was also the inevitable negotiation for each change one as ambiguity was often the cause.
When initiating a new project, we worked with the customers and analysts to prioritize the SOW putting high technical risk and high customer value at the top of the list. After we finished building some of the most important features the contract changes started to come in. These were often related to something in the backlog in which case we could give the customer the chance to swap for something else in the backlog at no cost.
If we had already built the feature that was now subject to change and nothing could be swapped out, it was clear to us how the schedule would be impacted. We would perform the same high level estimate for the new item that we did for every item in the backlog. We knew the team’s velocity at delivering these items, so it was pretty easy to determine if an extra sprint would be necessary and we could give the customer the option to remove low priority items from the backlog or extend the delivery date to accommodate the change.
A further benefit with this approach was to light the appropriate fire under the customer’s IT organization. Nearly every solution we delivered needed complex interfaces to the customer’s host systems and they were nearly never as prepared as they needed to be to accept our system when it was installed. As a result, when we used a traditional approach, there were always delays, blame, etc. No one was happy.
With SCRUM we had the option of beginning the test of host connections as early as we wanted in the process. These were always considered risky requirements and made it to the top of the backlog. Our project manager would help the customer to prepare various stages of interface testing to coincide with our sprint output. We were virtually guaranteed to discover that the customer’s host interfaces did not work the way thought they did (often 20-30 year old COBOL code), but now the necessary changes could easily be accommodated early on.
In the end we had a win-win story to tell, but it was only possible by adapting our organization, development tools, and product architecture to match the method. The adaptation was done using a holistic lean management approach that looked at the entire delivery problem (value stream), rather than just adding a SCRUM master and changing the way the software was built. The team had a wide variety of skills and capabilities. It was not a team of all-stars, though we always had at least one.
A couple of corollary notes:
– I would never build a deck (or house) with SCRUM, but there are great examples of home builders applying lean management methodologies to dramatically increase their throughput and quality while reducing their costs. In my view, SCRUM is most effective as a management approach when the team is building something creative where very complex human thought processes are needed and should have the minimum constraints to get the best result.
– Do not ignore lead time. A new car may come out of the manufacturing plant every few minutes, but that does not mean it takes a few minutes to make a new car. You might have to wait a few days for the first car to come of a production line. It is the same with SCRUM based development. If you are starting from nothing, or starting from something built using traditional methods, there will be some lead time necessary to start delivering value after each sprint. Do not rush it. The hype is too big about producing value after each sprint, so outside observers assume that applies to the first ones equally to the last ones. The first version of the SAP StreamWork collaboration tool released to the public was the beta “12sprints”. Can you guess what the lead time was?
LikeLike
Hey Russ
Thanks for adding such a detailed and well thought out reply.
Thanks for calling out the need to “prepare” the project for Agile. That must be one of the reasons it did not work for me.
The point about lead time is important too – this is not often emphasised by people promoting Agile.
So question for you – was your team co-located or spread across multiple locations?
Cheers
Vijay
LikeLike
The team in the story was co-located. I have run SCRUM with team members separated in three locations, up to nine hours TZ difference. We made it work, but it was not pretty.
Scott Ambler is selling IBM’s Rational products these days. He presented some nice numbers on the rate of agile project success for co-located (83%) verse near (72%) and far (60%) located teams. I have not seen any numbers on differences in velocity (my guess is fasted for the co-located teams). Of course, IBM is trying to sell their new Rational Suite products for managing non-co-located agile teams.
I think my remote experience worked because I had a critical mass in one location and the other two locations only had one person each. My preference is to co-locate the individuals on a team, but I suspect (never tried it) that a SCRUM of SCRUMS where each team was located around the world could be fairly effective.
The reality is that you need to pick a topic, component, center of excellence or something similar for a co-located team to focus on to be really effective. That is part of the organizational setup that must be done for SCRUM in my view.
I think all of this implies a fairly stable operational team model, not a project by project body assignment. That is certainly a long way from how most consulting shops staff projects. We made it work by having a team focused on one product and simultaneously delivering to multiple projects. The only person that would normally leave home base was an analyst who played a product owner role.
LikeLike
Ah got it. Ok, so after I read your experience, I am all the more convinced that this is not very easy to pull off in a consulting gig. The norm is that we pull in required experts from all over – and they have never seen each other ever before. Plus, in most projects, we have at least two locations – and some times in the big global ones – upto 20. But for a product company, I think this is a lot easier to pull off.
LikeLike
Vijay, I find the stats shown by Russ on agile project success very interesting, especially since the numbers come from someone selling IBM’s RUP products. Do you not use RUP and the Rational Suite within IBM yourselves?
LikeLike
@Russ, wow, great comment and great story! This definitely belongs as it’s own blog post (perhaps on SDN). Thanks for sharing!
Cheers,
ewH
LikeLike
lol… Good idea. I just started blogging on SCN this last week. I already have a backlog. 🙂
LikeLike
Well a passing note:-
I read something very interesting here
Click to access pwc-tech-forecast-summer-2008.pdf
LikeLike
* comments not coming from a development methodologies expert nor trained in “agile development methodologies”.
Dear Vijay,
It was fun to read through the thoughts that you have put on this blog. I had read it a week back and then may be thrice later. I was doing so trying to understand the emotions that led to so many analogies used in this post.
There are two aspects to this post. One point which I think this post shouts out is that I as a practitioner do not see value stepping out of my comfort zone as the credibility or exact area of the application of this concept is not an established practice yet.
Second, as per my understanding, and as many other participants in the discussion have pointed out, selection of the project / enterprise / stake holders is important in deciding which methodology to be used in exact or as hybrid.
I loved the way you talked about building houses the waterfall way, but those are just houses not mega structures that have lot many stake holders involved. If you see the mega structures build around the world (I love mega structures program on Discovery where they showcased the evolving blue print of Burj Khalifa, the tallest building in the world by closely engaging stake holders and technology experts and doing things right. As a matter of fact, the height of this tower was not certain until the construction came to a close), you will find that CHANGE is a major factor that drives completions of such projects. The keyword here is Adaptability.
Lets keep this point of ADAPTABILITY in our mind for a while and go back to the culprit, AGILE.
Agile for me is not a mere development methodology, its an enterprise strategy. According to me, it is either applicable to those who do not have a clear vision of future because of lurking uncertainties or those who have a very clear long term vision and have kept technology / market challenges in perspective while putting down system strategies. Being agile is being change friendly, where the cost of change and the cost of competence advantage are key in decision making. If an enterprise is agile, I would say, if the board passes a resolution, the change can be quick, robust and low on cost. The automation system should be flexible enough to make space for scalability and adaptability.
You said about customers not investing in resources to give feedback and get closely involved in IT business solutions development very close to starting of projects, I understand, are customers who do not invest in becoming agile and thus do not qualify to be engaged in such development cycles.
Everything boils down to cost of creation / ownership and change management and definitely an enterprise would expect low cost of ownership and change management as well as high competence advantage delivered with the business solution. So if the enterprise has high stake in the solution development, it will keep giving feedback for the solution, so as to get the highest competence advantage when deploying the solution.
As to the point made on lack of coordination between people resources, stars etc. is a failure in choice of selection of collaboration tools and not the strategy employed to develop solution.
I can keep on adding points here that will say, that no choice of development methodology gives highest competence advantage to the customer, but one which has low cost of embracing change and that I mean is one which is AGILE.
LikeLike
Hi Vijay,
Firstly, it’s great to hear feedback against Agile methodologies as it does have limitations in many scenarios (especially building decks) and does need to consider what it is being used for. Hence, kudos to you for being brave to go up against this new way of thinking.
That said, now the rebuttal (minimal only).
Coming from a RUP background, I’ve been doing iterative, risk driven development for many years now (on and off), and it’s proven to be incredibly valuable for high risk projects. It does take a lot of mentorship when working in teams, plus plenty of convincing of the business but it does work.
In terms of your analogy about the deck, I believe typically all agile methodologies set an expectation of what the deck will look like, including maybe the colour and basic materials, but the approach means that as you build it based on your designs, you can easily accommodate changing requirements which in a deck building world could be that you hit a rock bed that requires new machinery to dig holes. (BTW – That’s a real scenario I had when building my deck that couldn’t be planned for)
In terms of resources, the RUP way is to have more focus on design up front than on development. Prototypes are typically done which are also usually developed by your most senior developers. Basically, you develop foundation pieces very early with your senior developers so that it’s rock solid as you ramp up resources during more build focussed iterations (slightly different approach to SCRUM which is about delivering value to the business).
In this way, I believe RUP may suit you better for global build or Implementations than SCRUM. Basically, as you build requirements, you could in parallel get your company code config, HR Org Structures and other fundamentals in place. The key to iterating over implementations would be to understand the foundation pieces, build them first; while ensuring your most complex and critical pieces are being designed. Then having the foundation, you can start to divvy up the more complex and critical pieces and even use a SCRUM approach if you like (since this can be carved up regionally if you like).
Anyway, I agree that Agile methodologies and global resourcing is a difficult scenario but believe it is still possible and worth it. Large SAP implementations would also require a lot of coordination to get right but could be done. That said, if you can find a happy medium with a combination of approaches…building software iteratively, regardless of your Agile methodology, is extremely powerful as requirements are never stable but provided your foundation is solid (including stable requirements), your designs can be tweaked easily leading to happier customers.
Lots more could be said about this of course, and even my response will ensure some flaming, but it’s all good.
Cheers,
Matt
LikeLike
Hi Matt
Thanks for taking time to bring your point of view to this topic.
Since Rational has been a part of IBM for a while, and since I use RPM in my projects – I am some what familiar with RUP . However, I always felt that the 4 RUP phases
kind of give it a waterfall flavour.
As you rightly said, getting customers and developers spun up on RUP needs a bit of extra effort, and that is not always possible in projects that have tight schedule.
Help me understand this “hitting the rock bed” a bit more – what would be different if we had a waterfall model? Wouldn’t we just create a change request, and re-do the time and budget estimates if an unexpected scenario hit the project? What would Agile do in extra?
The way you have described how RUP helps with prototyping is something I often do in the gigs I manage. The only difference would be that, we use the protoypes as examples for junior developers – and never plan it to be rock solid. So once we can demonstrate something works – then individual developers use it as a reference to build their production quality work products.
I could not make Agile work in global engagements – despite giving it an honest effort. I must be missing out on something – because I always thought that if it works, it is worth the pain. Just that I could never get it to work, and I did not have unlimited time to play with it.
What does work well for me is a hybrid approach. I start with a waterfall plan with defined milestones, and I plan a few iterations in design and testing phase to use the RUP like practices. But the overarching principle has always been waterfall for me.
LikeLike
Hi Vijay,
RUP is definitely waterfall flavoured, but the difference being that you complete and end to end waterfall each iteration, with varying degrees of requirements vs build as you move along. My favourite part was the idea of putting attributes against each requirement. i.e. Looking at Architectural Significance, Complexity and stability (in terms of how likely the requirement will change).
By getting the significant requirements highlighted, and tackling the stakeholders to ensure these are stable; is a big step in setting the right direction for your project. That and focusing your project on building the foundation before getting too much into detailed requirements for the less foundation like requirements gives you a grounding before you beef up your development team in later iterations.
Note – The complex unknown requirements are also tackled with quick throw away prototypes like you say that help more junior developers build this at a later stage.
My most recent example was a little smaller than most; but involved building an external facing portal leveraging the AJAX Framework Page which is relatively new within the Portal. While the project implemented in a traditional waterfall, I ran this part of the project with an iterative RUP style approach. Although it took some convincing of the customer that they won’t see that much after the first iteration, they saw immediate value in the outcomes in the 2nd iteration. In the 2nd iteration and later, we were able to increase the development team; and also accommodate some quite significant changes in requirements. The fact was, the critical architectural significant requirements didn’t change because we spent time making sure they were stable. Anyway, prototypes were done for web services between ABAP and the Portal UME; SSO and reverse proxy was prototyped independently, and in other words; all the risky items in the project were worked through and built in the first couple of iterations which meant you pushed hard at the beginning of the project where Waterfall projects you don’t really push hard till the end when you know you’re falling behind.
Anyway, back to replying to your points:
Hitting the rock bed is possibly not the best scenario as if this was a house; the foundation is the foundation (and hence critical to get right). However, with a deck; the main bit to get right is the design from an end user perspective. The fact you needed additional equipment to dig the holes; didn’t change the materials or the overall design. So in that way, it’s kind of a requirement you don’t have to overly plan for; but be ready to adapt to. Anyway, not a great example I agree.
In terms of waterfall, this can definitely work (obviously). The real issue here is typically that the customer doesn’t really understand what they are getting till they see it. No one really likes reading 50 page requirement documents but if you can mitigate this by having an IDES type of system available; or using BPM and screen mock-ups; or many other techniques; then you can lock down the requirements early on. If the requirements are locked down – Waterfall is probably the most efficient way of doing this! That said, I still believe requirements will always change to a degree and it only takes one fundamental architectural significant requirement to change late in the day and your project schedule is blown out massively.
So how about this as a rule:
If your customer is extremely mature and heavily involved in the requirements and design stage with a grate ability to sign-off on these requirements; then go for waterfall; but if you have a potentially significant system to build that requirements are obviously unstable; go an iterative approach. SCRUM will be difficult with a global team; but start with a smaller and very senior team, potentially bringing together your global leads during the first 2-3 iterations, build the core and prove the risky items (documenting the majority of requirements by the end of the 3rd iteration); then move to a more traditional waterfall which you can then fix price if you need to!
BTW – I strongly believe that good teams (in a single location or at least timezone) with strong team members are what makes SCRUM work fantastically. It’s effectively giving strong developers the process they want to follow while allowing SCRUM Masters the ability to ensure they stay on track at regular intervals. I’m guessing it would only take a couple of poor performers in your SCRUM team to really bring the team down. Possibly you could remove them from the team but in a consulting world; you can’t just ask to get all the good consultants on your projects as you need to pick the people on the bench most of the time. Anyway, I’m not a SCRUM guy, and that’s just my guess at that.
Wow – what a conversation you’ve started. Nice work!
Cheers,
Matt
LikeLike
Thanks for the post Vijay,
Whilst I might not agree with your conclusion, I think you’ve brought up a lot of important and critical points that both customers and contractors need to be aware of when going down the Agile path.
You don’t provide much information on how you’ve tried Agile and in a future blog post it would be interesting to get some details. Hopefully this discussion will not turn into a witch-hunt;I’ve seen too many examples of people giving valid criticism of Agile being meet with the response “well then you didn’t do agile right”.
I’ll try to give my views and experience on the key points you bring up. Since Agile is such an umbrella term, I will respond with basis that Scrum as methodology/framework with no premature adjustments of the rules of the game. Afterall, the rules of Scrum are there for a reason and it’s generally not recommended to modify them before you are a mature Agile organisation (which will take years and years).
> Would you pay an Agile contractor to build your deck?
First of all, contract negotiation for any IT project is difficult and if possible I would recommend the customer to run the project themself and staff it with a combination of own personell and contractors with relevant experience. This will require significant effort from the customer in the area of IT project portfolio management and prioritization, but such a move will also be benificial in that it will move the company away from the evils of budgeting (ref. Beyond Budgeting http://www.bbrt.org/).
For many companies the above alternative will not yet be an option and they would like to buy a tailor-made solution which fits their current needs. Traditional projects will estimate the cost either based on previous experience, preferred as it minimizes risk, or by partioning the task at hand into seperate pieces that can be estimated. Scrum doesn’t say that you cannot do a similar estimation in order to make a contract with a customer. However, it states that you must expect these requirements to change during the duration of the project. Therefore, it places more value in responding to change than to fulfill a contract defined prematurely.It also states that the requirements in general should be in the form of user stories which focus on the business value, thereby improving the communication with the guy how pays the bill(typically in the form: as a , I want because ).
Customers in general want to have a quoted priced with an RFP and Agile projects are no exception. The price model is usually target price as this allows the risk to be shared with potential benefits for both customer and contractors. The major difference is that the estimation is done at a slightly higher-level and should contain requirements that can be directly into a product backlog for when the project starts. Planning poker estimation is also very useful in order to do relative estimates (can of course also be used in waterfall projects). In Norway we have a Scrum version of a recommended IT contract agreement (PS2000) http://bit.ly/albLgg (link unfortunately only in Norwegian).
To conclude, Agile doesn’t equal no estimation and need to face the realities of the business world.
> Global and Agile are like Oil and Water – They don’t mix.
From my experience and what others have quoted on empirically studies; having a co-located team is always the most effective. Anything else is sub-optimal (ie. waste) and requires mitigating actions.
If you have a Scrum product with several teams, I believe that it should be an absolute requirement that the team is co-located or at least in a similar time zone. This is essential in order to build the self-organising team, gain collective ownership and share competence. However, different teams within the same product doesn’t necessarily need to be located in the same time zone as long as they have some means of coordinating modifications that affect other teams.
Again, you have the same problem with more traditional project and Scrum is merely bringing the waste up to the surface where you can perform mitigating actions. A mitigating action might be to provide more detailed specification on the interfaces between teams and there is nothing from stopping you to do this whilst still being Agile.
> There are only so many super stars in this world
You say:” Not every one can go away with minimal instructions and come back with the right solution, and right questions to ask for next day.” However, a well-comprised and self-organizing team consisting of both juniors and maybe a super star will combined be significantly better suited than a single super start. Agile is a team effort!
It is very interesting to discuss how to best use your super stars. Scrum believes that team work is the best way of harnessing the value of your project members. By sharing a collective responsibility, knowledge is shared and you are on the way of producing high-performing teams (much of Scrum research is focused in this field where certain teams are 50 times more efficient than others). This is a pardigm change and requires a lot of focus, which is one of the main reasons you have a dedicated scrum master (who is not at all a project manager) who constantly challenges the team to reduce waste and improve. It is not unheard of for the Scrum Master to ask the team who is the most critical person on your team, and instruct them that for the next sprint he will not be allowed to code, only mentor the others. Some super stars who are used to be locked in a cage and then produce amazing results will have particular problems in doing this transformation.
> Most projects do not have dedicated business users.
How do you expect projects to succeed when the people how know the business and are going to use the solution afterwards aren’t involved sufficiently? We how work in IT need to become more professional and part of that is making clear the premisses and expectations we require in order to implement a successful project. User involvement is one of, if not the, key success criteria of IT projects.
In my experience, the minimum requirements in Scrum project should be that the business provides a product owner which is responsible for the prioritization of user requirements for each sprint based on the business value they will have. This will not be full time job. Also, send the person to a product owner scrum course.
If the customer is not present in Scrum meetings, you have been too pragmatic and most likely this will affect the project negatively. Instead quote the absolute rules of Scrum in this area and stand your ground that you cannot be Agile if the customer doesn’t do his part. It will then be visible to the organisation and steering group that the project has a high risk of failure due to lack of involvement of the customer.
> What works for you – people over process? or process over people?
You say:”I would trust a good process to compensate for the human errors most of the time.”
I say that I would rather be able to show business value to the customer for each sprint and avoid a big-bang GoLive.
> Would you build a mission critical solution using Agile?
A very good question, that I don’t really have the background to answer. I would recommend in general that customers do not choose mission critical solution as their first Agile projects as there is a lot of lessons to be made.
The C3 reference is interesting and new to me. It does however reference only Extreme Programming (XP), which in my view primarily consist of good developer principles and not a framework such as Scrum for securing progress in your projects
> Is Agile really good for long-term stability of a solution?
Solution architecture is hard to get right in any project, and perhaps even more so Agile projects.
Scrum’s approach is a focus on refactoring as you gain more information. It is common to run one or more architecture sprints at the start of the project, where the focus is analysing the right architecture and implementing a user story which requires functionality in all layers.
When evaluating of different tools from a vendor, prototyping is an invaluable method and should be encouraged instead of an endless list of requirements that all vendors say yes to anyhow.
> It is fun, but is that good enough for customer to pay?
Agree that fun for the development team is not high on the list of priority. Hopefully, the sense of ownership the individuals experience and the self-organizing team will mean that the blame-game is avoided.
Again thanks for the dicussions and looking forward to the reply from the Enterprisegeeks 🙂
LikeLike
Hi Dagfinn,
That was a well crafted, and well thought-out comment – so special thanks for that. I greatly appreciate your insights.
The post was already getting too big, which is why I resisted the temptation of writing about how we went about the projects. We did have a Scrum Master ( an absolute wonderful guy, who ended up taking a lot of heat when things didn’t go well – but still kept his cool). I will ask him if he wants to do a blog on his perspective. But if I cannot convince him – I will write something .
I have a habit of speaking my mind that usually rubs some one the wrong way, and then making that a headache for me.So I expected some witch hunting when I made this post. Thankfully, every one so far have been very civil and constructive. And I am not making any pretensions of waterfall being the best for all projects and Agile not working at all either – I just wanted to express what I experienced, and my rationale. No claims of universal applicability 🙂
About the contracting part – customers and SIs are both keen on predictability of the cost it takes to realize a benefit. If the scope and $$ is small, of course some companies might be ok being flexible. And customers will ask for back up information on how you estimated a certain amount of work. In a large project – it is not easy for me to estimate based on past experience on waterfall approaches, and then say that I will deliver using Agile. If I do that – 1. I am not sure myself if these estimates will hold, and 2. The customer will wonder why she has to pay me the same and still do a “new” methodology. Most enterprise solution sales eventually boils down to money – especially in these economic times. However hard we pitch “value” or “business benefits”, it all has to boil down to $$ – or else the customer will take the “no dice” stance. Till several smaller Agile projects get executed succesfully, and we have some quantifiable data to back up estimates – I don’t see this changing.
“Agile would like you to co-locate your teams” is not a powerful argument for me to pitch to a client. Global delivery is a fact in enterprise solutions world now, and it is simply too costly for customers to do that in a big project – even for a few months. And without co-location, it is very hard to succeed in an agile paradigm. In waterfall, this risk is minimzed because of the sign off processes and redundancies that is built into the delivery process. Requirements are done by some one close to the customer, validated by the customer and then the offshore team gets to ask questions. This process is painful if you are not used to it. But once you follow the established process – it works really well. I have been doing it for 10 years at least, and have high confidence in delivering with it – irrespective of scale. Offlate, we use a product called iRise to create great visualizations of the solution and this way – the customer sees and feels how the end solution works, without the need to code the application.
The superstar thing is kind of related to the global delivery problem. Pairing is realistic only when the team is colocated. Otherwise it is a huge pain. I am a big believer in my team working as a team and not as individuals – so I am sold on the efficieny gains you mentioned. I am all for it. Just that this colocation criteria kind of spoils it.
About the lack of dedicated business users – this is a widespread problem. Plus, these users ight be all around the globe. The person who eventually gets assigned migh not even know the “real”requirements. So a specification, and ideally a visualization – prototype, or iRise model, gets sent around and then every one gets to sign off after giving their feedback. This is how we overcome that in my projects. Most customers know that it is not ideal to not involve usrs completely – but they also expect the consulting company to show them how to mitigate the risk. It is an interesting idea to ask for customers to send some key people in their business community for scrum training. I have not attempted it – mostly because we usually use up all the training budget in sending people for SAP training which is quite expensive.
Showing business value in each sprint, and big bang are not exactly mutually exclusive in my opinion. There are many succesful big bang projects that have shown business value to the customer. And in integrated processes like most SAP ones, it is hard to do these incremental things because of the tonnes of dependencies. Check Nathan’s comment below – I think he explains it better than me. I think incremental business value, and using Agile for it, is a very plausible way to handle small to medium enhancements though.
On the 3C thing – it only got the popularity because they chose a not-so-succesful project as a poster child for XP. You are right – they did not have a framework like scrum though. That is a fair point.
On the long term stability of architecture when using Agile – this is something I want to ask Thomas Jung,after listening to his podcast on doing parts of Netweaver 7.2 using scrum. I have my reservations on whether Agile can deliver a robust architecture foundation due to the idea of constant refactoring. Let me send him a tweet. May be I am wrong, and it does work.
Thanks again, Dagfinn – your comments really helped clarify a few things about Agile in my mind.
Cheers
Vijay
LikeLike
A bit shorter reply this time 🙂
The contract-part is tough, but several companies gained competitive advantages by bringing agile elements to it. Amongst others Systematic and Trifork have reported that they won contracts based on the following main rules:
– Changes to functionality not started is free of charge
– Changes in priority of functionality is free
– New functionality can be added as the project proceeds, as long as similar sized functionality is taken out
– The customer can cancel the contract at any time, in this case 20% of remaining cost goes to vendor (in Norway 4-6% is recommended)
– It is required that the customer prioritises all functionality
Especially the part about cancelling the contract when the functionality delivered so far is sufficient adds flexibility to the customer. (Systematic also feature in a paper about combining CMMI level 5 and Scrum, worth a read http://bit.ly/bveqrD)
Agile projects also fail, but they embraces the concept of fail fast, fail cheap and bringing the cause of the failure up to the surface. There are too many over-analyzed traditional IT projects suggestions that the business has used significant time on, but are not realizable within the giving conditions.
The debate about Global delivery debate is huge and I know you have much more experience with it than me. However, it is my impression that within the scandinavian region the offshoring buzz/hype has somewhat receded. I believe this has a lot to do with the complexity and inefficiencies that occur when you are not co-located.
With regards to business users, I believe the key point is that business/customer must decide which functionality provides the most business value and therefore should be prioritized. This is part of the product owner role, and at least this person should be trained.
You say: “I have my reservations on whether Agile can deliver a robust architecture foundation due to the idea of constant refactoring” . It is a valid question, but it is important to know a bit about why we refactor within Agile. It is estimate that 80% of the software cost comes during the maintenance pahse. Traditionally, a lot of the cost comes due to inherited technical debt from the initial project. In a tradtional project the pressure of being on-time and on-budget means that revising the architecture and code is not done as new knowledge is gained. In Agile we embrace change and when new knowledge is gained questioning our earlier work, we in theory do a cost-benefit analysis for the entire lifespan of the solution to see if we should refactor. When refactoring is combined with automated tests, the cost of changes is not as high as you would expect.It is also not the case that in Agile we start coding on day 1 and whenever there is an issue refactor. But we do know that the best way of validating an architecture hypothesis is to produce working software using it.
Thank you for initiating this debate; a shame we won’t meet at sapphire.
LikeLike
Oh Vijay, where do I even start? I guess I can start with the points that I agree with which are:
1) Agile is no silver bullet
2) Agile doesn’t work for everyone
3) It’s best to avoid being extreme one way or the other and to pick the most appropriate tools and methodologies for each job at hand.
There are so many other arguments in here that are debatable, that I will probably have to save a full length response for one of our upcoming podcast episodes on the Enterprise Geeks. IT projects are not the same as building houses or decks, and they are certainly not running life and death missions to the moon. If they were, there would be a lot more crumbling, half-built houses around and nobody would dare step foot on a space shuttle.
On our previous egeeks podcast on the topic of agile, I mentioned that the answer for Enterprise IT projects most likely falls somewhere in the middle of these two approaches. In it’s most simple form, agile is about embracing change through continuous feedback and prioritization. Agile does not and should not have to be the unorganized mayhem that you portray in this post.
@Michael K, I do not see in Vijay’s post where he mentions ERP “implementations” specifically. However, while it’s hard to picture an ERP implementation following the extreme processes of agile, they would be equally likely to fail following a strict waterfall approach. Your blog is solely dedicated to project failures, but you point out one failure that used an Agile approach. Am I then to assume that all the rest of the project failures are due to following a traditional, waterfall approach? I would guess not, because regardless of the methodology used, there are always going to be projects with poor decisions, bad management, company politics, and big time failures.
Cheers,
ewH
LikeLike
Plenty to debate,Ed – I agree.
I am glad that we both kind of agree on a middle of the road approach, letting projects decide their methodology rather than take an extreme view.
Ok – so about the comparison to building a deck. Let me explain what I had in mind. Have you ever seen a house or deck being built in an agile fashion? I have not.And I bet there are more succesful deck/house building projects than there are succesful IT projects. And house building is a perfect waterfall project, with a lot of QA processes at each stage. And while the average IT project does not deal with life/death – there are several that do. Think of software for NASDAQ, or for scanning equipment etc. They cannot afford an unstable system. And – as far as I know, they follow a strict waterfall approach.
All kinds of projects fail – and due to all kinds of reasons, as you rightly said. However, waterfall takes a lot of blame for failures only because the vast majority of them use waterfall. What is usually glossed over is the fact that most of the succesful projects also did waterfall.
Now, where I mostly differ from your point of view is your statement “agile is about embracing change through continuous feedback and prioritization”. I am yet to see a project that was succesful after taking on continuous change on priorities. Continious change is the big enemy of a good,stable, high quality system in my eyes. You have an important role to play in IT in your company – would you encourage your users to change their mind all the time through a project, agile or otherwise?
All this said, I am a big fan of enterprise geeks, and I am looking forward to listening to your future podcast on the subject.
LikeLike
Vijay,
If we truly agree that one should use the right tools for the right job, the use of specific examples, such as NASA and NASDAQ, to strengthen the argument of why waterfall is the overall better approach, does not apply.
If these projects are better suited for a waterfall approach, then so be it. If building decks and houses are better suited for a waterfall approach, then so be it. I don’t agree with the premise of comparing building houses/decks to software projects, so any analogy as such is irrelevant. I can argue that raising a child successfully takes an agile approach with constant change and adaptation based on the individual. Does this mean someone should use this same methodology for all IT projects? No, because the analogy of raising children to implementing a software project is fundamentally flawed.
“I am yet to see a project that was successful after taking on continuous change on priorities.” – Requirements and priorities change regardless of your project methodology. Agile just allows you to find out, embrace, and adapt to them earlier.
Re:enterprise geeks – I know you are definitely a loyal member of the beloved eghead nation, and we are overjoyed to have such intelligent listeners. If you would like to continue this friendly debate further, perhaps you would like to join in on the episode? We gave Dagfinn and the Agile approach it’s time, so we would be more than happy to bring you on and give Waterfall it’s appropriate due.
Thanks for sparking this debate; it’s been thought provoking and very fun.
Cheers,
ewH
LikeLike
Ed, since we are in the debate – lets push this a little further.
I would contest that raising a child is not a project, but building a house or deck is a project.
A project is a temporary thing – where there is a defined start and end date, aftr which that team disperses, and a tangible output.
Raising a child therefore,does not qualify as a project. But there might be other specific examples that might suit Agile.
As I refine my thinking on Agile – fully admitting that I am not a big expert other than trying it 3 times in last 2 years – I think the major gripe I have is on its scalability.
In my job, the projects are generally big (at least $10 Million or so, and have at least 50 people, and at least working out of 3 timezones). So – when I say Agile does not work for me – it is in this context that my opinions were formed.
About a podcast with enterprise geeks – that would be a great honor, and I will gladly do that. Thanks for the offer.
LikeLike
Vijay, I won’t further the discussion regarding child rearing, since my only point was that I would NOT use it as an argument towards agile since it doesn’t relate.
To paraphrase the main point of my previous comments, there are different approaches for different types of projects and Agile flavored methodologies belong in the mix with the big boys like waterfall. It’s easy to write it off as unorganized mayhem with no plan, but that portrayal is inaccurate.
As for scalability, that will be a good topic for your upcoming visit on egeeks. I am looking forward to it!
Cheers,
ewH
LikeLike
Ed, fair enough. I did not expect to make any converts – we all have our points of view, and it is just fun comparing notes. We can leave it at that.
Let me know the plans for the podcast – maybe we can do it in Orlando during SAPPHIRE too if that works.
LikeLike
Vijay, I didn’t realize that you would be at SAPPHIRE Orlando; that is an excellent idea! If others that will be there would like to participate, just let me know.
Cheers,
ewH
LikeLike
Sounds like a plan, Ed – looking forward to it. I should be in the IBM booth most of the time – but since we both have mentor shirts, it shouldn’t be hard to find each other.
LikeLike
I was on a new ERP implementation with one of the Big 6 that tried this. They didn’t call it Agile… this was circa 2002 so I’m not sure how mainstream the concept of Agile Development was back then. Either way… the focus was to make a traditional ERP implementation highly iterative. Deliver a solution (let’s say, FICO Org Structure) in a week, move on to another task such as sales order entry, repeat until go-live.
The reason it didn’t work well is that traditional ERP covers process areas that are quite and heavily integrated large. It’s probably for that reason, because it’s a common complaint of ERP, that they thought they could break it into smaller chunks so that it could be more easily digested. But ERP processes such as order-to-cash are quite long and have numerous variables. You can’t just tackle Order Entry without regard to how payment, cash application, and a holistic reporting solution on order-to-cash will be impacted. It’s shortsighted and wreaks of academic/management approaches designed by folks that haven’t done this before. That’s my rant.
To circle back, the project eventually resorted to a traditional implementation approach but this wasn’t formalized. Heaven forbid that a mistake be acknowledged!!! Instead, the last 60 days were a cram session, scenario by scenario into SAP… which is how most large ERP projects are run (focused on scenarios and implemented by phases) only they do it over 9-12 months, not 60 days.
Cheers,
-nathan
LikeLike
Hey Nathan, thanks for chiming in.
I tried Agile in SAP and non-SAP projects, and the only time I did not have to abandon it completely was when I had a captive customer who was in the same building as my developmer and architect, and the scope was a set of screen enhancements.
So, like I just mentioned in my reply to Ed – I doubt it if Agile scales to larger and more complex projects
LikeLike
Vijay,
Very good post, I enjoyed your analysis and examples that you articulated here. Like you said there is a time and place for everything. I often recommend Agile when dealing with improving user experiences, consumer tools and shopping experiences etc.. however when dealing with systems that are heavily regulated or mission critical (tax, payroll, inventory) I would stay far away!
LikeLike
Thanks Scott
LikeLike
Vijay,
Very compelling post. I have yet to hear of ERP implementations being performed on an Agile basis, although I’m sure many people would like to try.
Arizona State University tried a kind of Agile-like approach on one major project, with terrible results. Here’s a post on that ill-fated project:
http://blogs.zdnet.com/projectfailures/?p=412
LikeLike
@mike Pretty scary that it was a conscious decision that ASU took to go down this path.
LikeLike
I disagree with some of your points… for example-
Building a deck- does your understanding evolve throughout the project, are there any problems discovered along the way… if not, then go waterfall. If so, the estimate will change, your requirements will change, and agile is about adapting to this as it occurs instead of fighting over the fact that the original estimate is no longer reasonable.
Global- my first experience with Agile was at Siemens Medical. It was initially very successful and included teams in the US, Bangalore, and Romania on the same project. The eventual slip of Agile was due to unrelated reasons from the global distribution. That being said, I don’t like doing agile in distributed setting.
Superstars- I don’t even understand this point. Pairing in Agile is about helping your superstars lift up your team.
Dedicated business owners- customers aren’t present in waterfall or agile… they cause problems for both when they suddenly show up at the end and have different requirements or different budget in hand. Agile attempts to bring this problem to light and deal with it along the way.
I’m stopping here… I don’t think Agile is the silver bullet, but I can say that I’ve seen more waterfall project problems than agile project problems.
LikeLike
@Kevin – apparently the clients you work with are a lot different than the ones I am used to. I am yet to see one who said – lets use agile, and I am cool with estimates changing each time some one in business changes her mind.
Glad we agree on agile being a relatively bad idea in a distributed setting. Even with 2 teams in two different locations- and with 3 experienced scrum masters in the team – we had to pull teeth to get things done.
Here is the superstar explanation – since there is very little formal documentation, you need people who can take minimal information and work something out and come back with questions etc for next iteration. This is not something most average techies can do – paired or otherwise. Again – just my observation from teams I have worked with that tried this.
Waterfall also can VERY efficiently deal with changing customer requirements. I, in fact would argue, that a formal change management process – with change control, re-estimation and approval, is a more robust way than letting a project team who is too closely involved (and emotionally invested) in the project to take a call by themselves.
There is no surprise that you and I have seen more waterfall problems than Agile problems – it is simply because most projects in the world follow waterfall. It is not an apples-to-apples comparison.
My general view at the moment is that waterfall is a more appropriate tool for more types of projects than Agile. But till we gather more information and reach concrete conclusions – it is probably best that individual projects choose what they like best.
LikeLike
Vijay, I largely agree with this post, in particular, the spirit of trying to take the hype out of a particular approach and have it be accountable to what actually works rather than give it a pass because it is trendy or sexy. This last point, “Rather than take an extreme view of either/or – it is probably best that we let individual projects decide the methodology they want to follow.” I think it’s hard to argue with. One reason I am interested in agile is that unlike you, I think many in the SAP world don’t know much about it and should spend more time learning about it. Not because it is perfect, but because it brings in new tactics and ideas. There is a customer panel on Agile at Sapphire this year and I’m definitely interested to hear these stories of Agile in action.
My only criticism of your post is that it seems there may be a bit of a “rush to judgement”. I’m not sure I see the point of saying “Agile will be great for some things and terrible for others.” Unlike SOA, I”m not sure we are clear yet on where Agile can work and where it can’t. SOA being around longer, I think we can make more concrete judgments on that. But I realize that you always keep an open mind and in a few years, as this unfolds, which should have a much clearer sense of what the appropriate use cases for Agile are. In the meantime, a hype-free discussion that puts new ideas to the practical test is the best route.
LikeLike
@Jon thanks for your thoughtful comments. I think Agile got hyped before it had a chance to prove its worth one way or the other. About the SOA comparison – Agile like concepts – like XP, has been around for a good while now, and I would think it has more tenure than SOA 🙂
Despite me holding on to the SOA comparison as valid, The rush to judge – yeah, that is probably true :).
LikeLike
Vijay
Timing of this blog can’t be better. Matter of fact, my colleague and me were discussing about this the other day. We are in a big ERP transformation project and are in the design phase and were discussing agile development in this context.
Agile Development/Scrum is based on the fact that requirements constantly change. So split the overall solution to smaller solution sets and deliver them fast. The problem with that is, there is a good chance that the functionality provided in first sprint is already obsolete because of newly identified/changed requirement in the third sprint. Now, because I followed agile approach, I ramped up my team earlier than I would have in the waterfall approach and still had to re-do the stuff.
I still think agile development will work for software product companies because innovations in technology can be provided in smaller solution sets by small teams but I don’t see how this can work for projects like SAP transformation. In most cases, the business case for ERP transformations is based on an integrated solution and splitting it in to smaller solution sets may not be possible. Matter of fact, I have seen big bang implementation projects have been more successful than projects that are run in phases. Same would be the case for agile development.
As always, I enjoyed reading your blog. Thanks for bringing this up. Great read!!
KK
LikeLike
@KK Looks like we have had very similar experience, and kind of think along the same lines on agile.
It does probably work well for product development. But the one thing I find hard to understand is that the customer is seldom involved in the process. If a product manager is taking the role of a customer, is it really catering to customer requirement sin an “agile way”? and if not, why not waterfall?
LikeLike
very good insights of challenges we are facing by using latest methodologies..
LikeLike