How Useful is Methodology?

I often wonder how useful methodology is. Often there you have companies that spend a great deal of resources in a methodology and then find that their staff barely implement it. Then suddenly they run to a light methodology worked around Agile (some latest fashion) and it seems that this takes away the whole systems engineering approach out of it.

For me, all this conflict over methodology is decadent. I agree with Ivar Jacobson and feel that the source of the problem stems from the so-called conflict between software engineering approaches as one approach, compared with Agile methods. Jacobson writes in his blog that this conflict is unnecessary and that the ideas are in fact complementary. I wish him good luck to explaining that to all the consulting gurus and book writing gurus.

Definitely, his blog is one that I have been reading for sometime now and it brings out whole hosts of perspective and insight about the issues. Have UML and RUP really lost their momentum? In the East Coast financial domain it certainly has not! But we are laggards so perhaps the trend will be thus here as well over the next 2 years or so. But we definitely need far more documentation than the lightweight Agile!

I must confess I don't instinctively grasp it as well as I grasp RUP, or didn't and am only beginning to understand it. The aspect I was having trouble with is understanding what Jacobson calls the social engineering part of Agile. For me this takes us into management theory and away from the systems engineering core methodology. But then, maybe they are complements. Maybe like Scientific Management was replaced by Fordism, an increasingly social aspect is coming into software development methodologies. Which brings us to the question of whether this development is a pendulum swing and essentially circular, or if its linear. I personally feel that the essence of RUP will always remain, as will the essence of Agile. If we see the development of management theory over the years, the beginnings of Scientific Management were very "systems engineering" and the evolution over time moved towards increasingly behavioral methodologies.

Can this be the fate of our paradigm? I believe it is possible, and I give as proof the profusion of gurus in both fields!