What is a Methodology?

>> Saturday, December 12, 2009

Since the beginning of my IT career some 25 years ago, I have entertained a sound reserve towards software development methodologies. I have always found it ironic that in our industry we label "methodology" a checklist of documents and a set of document templates that must be produced or filled in at different stages of a project. Such "methodologies" are very good at specifying what "deliverable" must be produced and when they should be deposited but usually very silent on how exactly do we produce the ultimate deliverable, i.e. the software system itself. It is true, that traditional methodologies also describe a series of software development phases and how should be doing what at each phase. But again, how exactly do we work within each phase was/is often left to the imagination.

For this reason, I was always more attracted by the literature that describes and discusses how to effectively and efficiently design software and how to write and manage code and systems. Strangely, this literature tended to shy away from the term methodology and preferred to use expressions such as "software engineering".

It was therefore refreshing for me when the Agile movement came into light. For once, I was hearing and reading about a "methodology" that corresponds to my own experience and inclinations. One that I could relate to. One that talked about practices, techniques, systems and tools. That puts the emphasize on the delivery of software instead of the delivery of documentation. An approach that prescribes the use of tests, of continuous refactoring, that talks about team organization, work habits, and human interactions. Those were the early days of the Agile movement with a lot of talk about extreme programming and unit testing.

However, methodologists at heart were not to be left on the sidelines. They woke up and smelled the coffee and had the intuition that there might be something for them in this new trend. Since it was difficult to sell the idea of a list of documents for each of the different stages of an agile project (agile is lean and iterative remember?) they instead invented a new religion with priests (scrum masters) and ceremonies (dailys, retros, demos etc). I am not arguing that all this is B.S. but I do see a dubious trend when people consider they are agile because they are holding SCRUM ceremonies. I may be naive, but to me, ceremonies are meant to be (or should be) a means to help teams and organizations increase their agility. If ceremonies become an obstacle to agility, then we should get rid of them and invent something else. 

On the same basis, we now find "agile" tools on the market that help organizations follow the SCRUM way of doing things. Considering yourself agile because you use such a tools is completely fallacious. I personally consider it much more important, for the sake of agility, to use automated testing tools and build systems then to use a SCRUM compliant project management tool.

So long,

Laurent

Read more...

Accumulating Technical Debt: A Costly Proposition

I just the passed couple of weeks doing some very technical work with Maven on a large (above 50K person-day) project. Essentially, our dedicated team of three engineers spent a total of approx 30 days to realign the build configuration and process for this  project. We already had a pretty good idea of what was wrong and how to fix it but what we found amazing is the extent and severity of the problems and anomalies that this project had been dragging for many many months and how little control over the whole build process the teams had. I won't get into the technical details here (maybe in a follow-up article), but imagine a project where daily builds take a few hours, where the result of the build is extremely messy on top of being somewhat unpredictable, where the deployed applications are plagued by failures due to conflicting jar versions and by different compile and runtime dependencies. In addition, the build process, although based on Maven, relied heavily on homegrown Maven plugins to do all sorts of "magic" (albeit unnecessary) tricks. I just described a recipe for problems and problems they had. The comforting news is that it was possible to turn the project around and realign it so we now have a controlled build and release configuration that relies almost exclusively on generic Maven plugins.

The real question is how do projects get into such a predicament and how come they tolerate the negative consequences for such a long time period? In this particular program, external audits were conducted on at least two occasions and each time, strong and concrete recommendations were made and warnings were issued but things continued as is. The direct result of this is that a lot of fuel was wasted on attempting to alleviate and compensate for the negative effects of the fundamentally flawed build and release process and configuration. I like to compare the situation to traveling in a luxury sedan with half deflated tires and a wobbly steering. You are burning too much fuel and rubber and you are a catastrophe in waiting but instead of taking the time to stop and fix (or replace) your slow leaking tires and steering you prefer to just add air from time to time and put up with the steering problems because your are in a hurry to complete your 5000 km journey. I mentioned a luxury sedan because the tool set for this project is top notch: Maven, Nexus, Hudson, Eclipse, Accurev. All the right tools, but most of them not being used properly.

Officially, this particular initiative is using the "Agile Methodology". In reality, it is applying the SCRUM approach to managing teams and projects. In many ways, the project is not compliant with the agile software development principles. Dragging for months and months a technical debt and paying interest at every detour is not, in my book, an agile strategy. In fact, the SCRUM discourse of focusing on client functionality and of managing a backlog of "User Stories" that are prioritized by the product owner and that must fit within the sprint can have  perverse effects that we must be aware of.  Organizations that adopt this approach should put some mechanism in places so that unjustified technical debt does not accumulate and reach crippling proportions.

Many methodologists like to present the so-called "Toyota approach" as a model to follow. I do not consider myself an expert in the subtleties of one method versus another but my understanding of the Toyota way is that if a problem in the assembly line jeopardizes product quality, we stop the assembly line immediately and fix the problem before we pursue. Even if that means that 20 or 30 people might be idle for a few minutes or a few hours. Software development is not an automobile assembly line but when the process is broken, taking the time to fix it always pays off.

Following our intervention and the extensive refactoring of build configurations, the builds are easier to manage, the build process is faster, lighter, transparent, predictable and flexible, they can be performed by different individuals (not just by one "specialist" know-it-all) the deployments are more stable and overall, teams are more productive. I am convinced that refusing to make this change earlier did not allow the project to deliver more functionality. Quite the contrary, it has delayed and slowed down the whole development process, it has poisoned the work atmosphere, it has distracted everybody from their core tasks and has entertained the impression of a semi-controlled chaos which most people and organizations do not appreciate.

So drive carefully and keep your vehicle in good repair,

Until next time,

Laurent

Read more...

WAS 6.1 Profile Creation

>> Monday, November 2, 2009

This article is very technical and describes how to create a Websphere Application Server (WAS) 6.x profile that can be reused on different developer workstations. After searching the net for the recipe and not finding a complete one, we had to devise our own. The following has worked for us but may need adaptation depending on your own needs and situation.

In the following procedure, it is assumed that WAS 6.1. is installed on the workstation. The name WAS_HOME refers to the path where WAS is installed (ex: C:\IBM\WAS6.1\AppServer).

We also assume that the following folders were created:
   C:\
     WAS6.1 
       jdbcDrivers
       logs
       profiles
       sharedLibs

1) Launch the profile management tool from
              [WAS_HOME]\bin\ProfileManagement\pmt.bat

2) In the Profile Management Tool wizard:
  • Select the Advanced profile creationoption
  • Select Deploy the administrative console & Deploy the default application
  • Set the profile name and directory to WASDev[0n] and C:\WAS6.1\profiles\WASDev[0n] where [0n] is the profile number
  • Select Create the server using the development template
  • Unselect Make this profile the default
  • Specify the following Node name: localhostDevNode[0n]
  • Specify the following Host name: localhost
  • DO NOT enable administrative security
  • Specify Default Port Values
  • Uncheck Run the application server process as a Windows service
  • DO NOT Create a web server definition
  • Wait until the Profile Management Tool wizard has completed
  • Do not run the First Steps Console

3) Verify the content of C:\WAS6.1\profiles\WASDev[0n]\logs\AboutThisProfile.txt. It should contain something similar to this:

Application server environment to create: Application server
Location: C:\WAS6.1\profiles\WASDev02
Disk space required: 200 MB
Profile name: WASDev02
Make this profile the default: False
Node name: localhostDevNode02
Host name: localhost
Enable administrative security (recommended): False
Administrative console port: 9060
Administrative console secure port: 9043
HTTP transport port: 9080
HTTPS transport port: 9443
Bootstrap port: 2809
SOAP connector port: 8880
Run application server as a service: False
Create a Web server definition: False

4) Edit the file C:\ibm\was6.1\profiles\profiles\WASDev[0n]\config\cells\[host]Node[0n]Cell\nodes\localhostWASNode\variables.xml and substitute the following:
<entries xmi:id="VariableSubstitutionEntry_9" symbolicName="JAVA_HOME" 
  value="C:/PATH_TO_WAS_INSTALLATION/java" />
With:
<entries xmi:id="VariableSubstitutionEntry_9" symbolicName="JAVA_HOME" 
  value="${WAS_INSTALL_ROOT}/java" />

In the same file, set the value of ORACLE_JDBC_DRIVER_PATH and other JDBC drivers paths if necessary to C:\WAS6.1\jdbcDrivers

5) Add the following files from an existing development profile to the root directory of the new profile:

readme.html
startWAS61.bat
stopWAS61.bat
WAS 6.1 Console

6) Edit the content of the file bin\setupCmdLine.bat to replace absolute references to WAS_HOME with the %WAS_HOME% expression. Alternatively, copy the bin\setupCmdLine.bat files from an existing development profile. As a reference, the bin\setupCmdLine.bat should contain something close to the following:

SET WAS_USER_SCRIPT=C:\WAS6.1\profiles\TDADev02\bin\setupCmdLine.bat
SET USER_INSTALL_ROOT=C:\WAS6.1\profiles\TDADev02
SET JAVA_HOME=%WAS_HOME%\java
SET WAS_CELL=qcdev584Node05Cell
SET WAS_NODE=localhostDevNode02

REM Set the default OSGI -X and -D java ARGS.
SET OSGI_INSTALL=-Dosgi.install.area="%WAS_HOME%"
SET OSGI_CFG=-Dosgi.configuration.area="%USER_INSTALL_ROOT%"\configuration

REM SET ITP_LOC=%WAS_HOME%\deploytool\itp
REM SET CONFIG_ROOT=%WAS_HOME%\config
SET CONFIG_ROOT=%USER_INSTALL_ROOT%\config
SET CLIENTSAS=-Dcom.ibm.CORBA.ConfigURL=file:%USER_INSTALL_ROOT%/properties/sas.client.props
SET CLIENTSOAP=-Dcom.ibm.SOAP.ConfigURL=file:%USER_INSTALL_ROOT%/properties/soap.client.props
SET CLIENTSSL=-Dcom.ibm.SSL.ConfigURL=file:%USER_INSTALL_ROOT%/properties/ssl.client.props
SET JAASSOAP=-Djava.security.auth.login.config=%USER_INSTALL_ROOT%/properties/wsjaas_client.conf
SET WAS_EXT_DIRS=%JAVA_HOME%\lib;%WAS_HOME%\classes;%WAS_HOME%\lib;%WAS_HOME%\installedChannels;%WAS_HOME%\lib\ext;%WAS_HOME%\web\help;%ITP_LOC%\plugins\com.ibm.etools.ejbdeploy\runtime
REM SET WAS_BOOTCLASSPATH=
REM SET CLIENT_CONNECTOR_INSTALL_ROOT=%WAS_HOME%\installedConnectors
SET CLIENT_CONNECTOR_INSTALL_ROOT=%USER_INSTALL_ROOT%\installedConnectors

SET WAS_CLASSPATH=%WAS_HOME%\properties;%WAS_HOME%\lib\startup.jar;%WAS_HOME%\lib\bootstrap.jar;%WAS_HOME%/lib/j2ee.jar;%WAS_HOME%/lib/lmproxy.jar;%WAS_HOME%/lib/urlprotocols.jar;%JAVA_HOME%\lib\tools.jar
SET WAS_PATH=%WAS_HOME%\bin;%JAVA_HOME%\bin;%JAVA_HOME%\jre\bin;%PATH%

SET WAS_LOGGING=-Djava.util.logging.manager=com.ibm.ws.bootstrap.WsLogManager -Djava.util.logging.configureByServer=true

SET QUALIFYNAMES=

7) Replace all occurrences of absolute paths to WAS_HOME appearing in C:\WAS6.1\*.bat files by the following variable %WAS_HOME%. For example, the file startServer.bat should contain the following:

@echo off
SETLOCAL
set WAS_USER_SCRIPT=C:\WAS6.1\profiles\TDADev02\bin\setupCmdLine.bat
call "%WAS_HOME%\bin\startServer.bat" %*
ENDLOCAL & set MYERRORLEVEL=%ERRORLEVEL%
if defined PROFILE_CONFIG_ACTION (exit %MYERRORLEVEL%) else exit /b %MYERRORLEVEL%


Alternatively, copy the bin\*.bat files from an existing development profile.

8) Make sure the profile is functional by going through the following steps:
  • Start the WAS server by launching startWAS61.bat
  • Open the WAS console by launching "WAS 6.1 Console"

9) Copy the required JDBC drivers (jar files) to C:\WAS6.1\jdbcDrivers

10) OPTIONAL: Configure the profile by defining the required resources (Data Sources, Mail Session, Message Queues, JAAS J2C Authentication aliases etc). Alternatively, it is possible to copy a resources.xml file from an existing WAS profile to the following folder:

C:\WAS6.1\profiles\TDADev[0n]\config\cells\[host]Node[0n]Cell\nodes\localhostDevNode[0n]\servers\server1

For Data sources, it is important to perform a "Test connection" on each defined data source.

11) OPTIONAL: Configure (define) shared libraries for apps that point to a subfolder of C:\WAS6.1\sharedLibs (ex: C:\WAS6.1\sharedLibs\mysharedlib)

12) Test your configuration by deploying the desired target applications.

OPTIONAL

13) If the above tests are conclusive, uninstall the applications and delete the files from C:\WAS6.1\profiles\WASDev02\logs

14) Create a ZIP file using the following procedure:
- Make sure the WAS instance is stopped
- Zip the contents of the C:\WAS6.1\profiles\TDADev[0n] folder

Have fun,

Laurent

Read more...

Commercial or Open Source: Which One is More Agile?

>> Sunday, November 1, 2009

Is there a link between open source software and an organization's agility?

To me, that's a good question. At first, one would be tempted to say no, the type of license under which a given software artifact or system is distributed has no impact on the degree of agility of the organization that uses it. But anybody who has worked in an organization where one or the other (or both) types of licenses are used will tell you, or I expect them to tell you, that commercial software licenses tend to limit your options, your flexibility, your adaptability. Now, the lack of such qualities, in my book, tend to reduce agility, certainly not to increase it. To make my point, I tried to list the most important "rigidifying" (sic) effects of the use of commercial software licenses in an organization. Note that licensed software, for the purpose of this discussion, is any piece of software whether it is a productivity or management tool, a runtime engine, a coding or building tool, a data management system, a library or a framework incorporated in a greater ensemble.

  1. The decision making and bureaucratic process leading to the adoption of any significant piece of software is usually time consuming and arduous. When, on top of the technical and political (I was tempted by the word "religious") issues there are important budgetary and legal issues then it becomes an odyssey. In an agile organization, the decision making process should be swift and focused on the needs of the business. When it can be done without the intervention of accountants, attorneys and other peripheral stakeholders, it fits in more readily with the agile philosophy.
  2. Commercial licenses usually come with constraints. Constraints on the number of users,on the number of CPUs, they may have an expiration date, a service contract for updates and support etc. This leads sometimes to absurd (but very real) situations where a piece of software acquired for a high price is not used by all who need it because there are not enough "seats" and acquiring additional ones was not budgeted and/or approved. I was actually exposed to a situation where a bug tracking system was used by the testers but not by the developers because the number of available licenses was insufficient. Talk about something really useful and agile. In such a situation, the relative quality and feature set of a piece of software becomes irrelevant.
  3. With commercial licenses, it is often the case that the installation and distribution of copies of the software must be "managed". The number of copies installed must be monitored and the software must often be unlocked. I find that not only is commercial software more costly to acquire, it is also usually much more costly to use post-acquisition. This all adds complexity and overhead without real business value. Things that are at odds with the precepts of agility.
  4. Inertia. An agile organization should minimize inertia. However, did you ever have to convince a manager who obtained approval for a large investment in a piece of software that the software is useless or obsolete? Not an easy assignment. Often, the best decision, from a business and technical standpoint, is simply to abandon a piece of software. If there was no initial, up-front, financial commitment, the decision is often easier. Strange enough, tell a manager that he should drop a costly piece of software and use a free one instead because the free one is better, more powerful and more cost effective and you often get lot of resistance. Not a very rational stance but I guess that is human nature. There still is a strong belief that if you pay for something you will obtain more value, more service, more support. This is often true but not always. In software, strangely enough, this is not always the case and it is less and less the case. But that is possibly the subject for another article.
So long,

Laurent

Read more...

Technical Leadership in Scrum Teams and Projects

>> Tuesday, October 27, 2009

I have been working for the past several months in an organization that has a number of teams that apply the Scrum methodology [1]. In fact, I am working within a Scrum team myself. The teams and team members are expected to apply very rigorously a series (if not all of) of the scrum mantras. These include the Scrum vocabulary, concepts and ceremonies. There is a lot to say about this experience but today I essentially want to share my views on team and program technical leadership in this particular scrum program.

What is somewhat striking in the way Scrum is applied in this program is that leadership in general is somewhat downplayed or neglected. Or let me put it this way: governance and leadership remain somewhat fuzzy and the advocated rules and practices encourage this. It is as if there was a reticence to recognize the need for leadership and the need to empower individuals with the responsibility to lead.

I admit that I am somewhat allergic to autocratic hierarchies and that I value collegiality, collaboration, democracy, consensual decision making and all. But I also value efficiency, good design, clear directions and a clear delimitation of roles and responsibilities. An ex-colleague of mine liked to repeat the same riddles over and over (don't we all have such a colleague?) and one of these was: What is a camel? Answer: A horse designed by a committee. I have nothing against camels but, to a North-American at least, they look somewhat awkward. In other words, I strongly believe that without strong and clear leadership in the various aspects of a software development initiative (since this is what we are concerned with), it is very difficult to produce solid, clean and effective software in an efficient manner. It appears, from my experience, that some (if not all) Scrum gurus (or at least practitioners) believe otherwise. At least they act and talk as if they believed otherwise.

As an example, we spent 45 minutes today in a team meeting (we had more than 15 people in the room) called a Sprint retrospective, trying to come up with a solution to a problem we had collectively identified in the preceding 10 minutes. To me, this is a completely futile exercise. I am willing to agree that a sprint retrospective can be useful and that if we collectively recognize that something needs to be taken car of we can fruitfully spend 10 or 15 minutes discussing the problem, gathering different views on the subject and on what should be done. However, trying, on the spot, to elaborate and decide upon a solution to this problem with 15 people around the table is sheer nonsense. Most real problems require analysis, design and implementation planning and this cannot be improvised in 20 minutes by a group of 10 or more individuals. It is best done by two or three people with one of them assuming the lead. In other contexts, the reflex in the above situation would generally be to delegate the responsibility to work on the solution to the most obvious people/person for the job and, if required, to present the proposed solution to the assembly for feedback. Is this approach passé? Is it anti-scrum? I really wonder sometimes.

To illustrate my point further, this particular program includes 6 scrum development teams that produce code on a daily basis and a good part of the software artifacts produced by any given team is used by or uses other team's artifacts. Yet, from the start, technical leadership in this program has been, and still is, fuzzy. The organization has not "officially" (or informally) recognized the need for technical leadership that spans the various teams.

This philosophy (if I can call it as such) is also present within the scrum teams. In fact, when I joined the program and was presented with it's modus operandi, one of my first questions was: Who assumes technical leadership within a Scrum team that develops software? The answer was not clear but between the lines I read that technical leadership was distributed and collegial and that it "naturally" emerged as the result of the sprinting effort. Hum, interesting approach I thought, can't wait to see the results. This notion is actually part of the "official" Scrum approach: the Scrum master exercises leadership but essentially as a coach to help the team apply the Scrum methodology not as an expert in software development. OK, I buy that: The scrum master's role is to help the team work in an effective fashion. So who designs the software? Answer: The team members! I also like that. I never liked the notion that a software engineer simply writes code. But who is responsible for the design of the software? When the answer is again all team members I sort of have an issue. A good leader, in any context, relies on his team members, calls upon their expertise when needed and welcomes initiatives and suggestions but when a choice has to be made, it is the leader's role to make sure the choice is the best one all considering. It is also the leader's responsibility to make sure that other team members design and build stuff that fits into the big picture.

Well, after observing (and working) with the teams for some time, I generally agree that in most teams, technical leaders did emerge. However, the roles and responsibilities remain generally unclear and are not officially recognized, the technical coordination between teams is deficient (nobody is clearly mandated to exercise this role) and a unifying design and direction for the various business functions produced by the various teams is clearly lacking. In fact, most software developers feel they are not given strong or clear technical orientations (or that these orientations arrive very late in the game, when most of the software has already been implemented) and there is clearly a sense of frustration with this aspect of their work.

When did leadership and governance become taboo in the agile world? Since when are we able to run lean, efficient and agile projects and organizations without clear, affirmative and recognized leadership? How are these concepts at odds with the concept of agility? I was comforted to read that the issue of unclear leadership and in particular of technical leadership in the Scrum methodology has been recognized by others [2]. Still others [3] seem to openly recognize the need for technical leads in Scrum teams.

Strong leadership in software development projects does not imply that decisions and orientations are taken blindly and without concertation and discussion. It does not mean that decisions and orientations cannot be challenged. However, without leadership (and in the case I am particularly interested in), without strong and recognized technical leadership inside the team and between teams, everybody is empowered with the capacity to go in whichever direction they prefer or feel is the best. You risk ending up with a camel (or a dog)!

References

Read more...

Benefits of a Repository Manager

>> Monday, September 28, 2009

For organizations and projects of any reasonable size that use Maven as their build tool, the deployment and use of a Maven repository manager is a must. As Maven users know, Maven requires access to standard format repositories from where it can retrieve the various artifacts that projects declare dependencies on. In fact, Maven's operation is based on the existence of public repositories that follow the Maven nomenclature and versioning scheme (artifact coordinates in mavenese). Since artifacts declare dependencies on other artifacts, it is quite impressive to see how quickly a reference to a major open source framework translates into dependencies on a multitude of other, public, artifacts. By configuring Maven to automatically retrieve these from the web, the task of retrieving all the various jars one by one and of making sure they have the right version number suddenly disappears.

This is the first obvious benefit that comes with the use of Maven. However, what new Maven users do not always realize is that the right way to do things is to host your own repository manager and to thus control and optimize artifact distribution and uploads. There are various repository management solutions available out there and a web page on Codehaus that compares three of the most popular ones is available here.

In my view, the most important benefits that come with a corporate repository manager are the following:

  1. The caching of artifacts used by the various projects in the organization (released artifacts are fetched from the web once and made available to all from then on).
  2. The grouping of several repositories into groups to reduce the complexity of maven configurations (settings.xml files)
  3. The possibility to install non-public artifacts required or used by the various projects (these can be legacy modules, third party modules or other)
  4. The possibility to enforce various access control policies for both reading and deploying artifacts (snapshots or releases)
  5. The flexibility brought on by the fact that repositories can be easily added, removed, reconfigured, disabled, reenabled etc
  6. The possibility that some some repository managers provide in terms of procurement so that an organization can control which artifact is available to whom for legal or technical considerations.
  7. The possibility to automate certain tasks such as the removal of outdated snapshots or the reindexing of public repositories.
  8. The availability of a user-friendly console to manage access, search, upload, remove and update artifacts.
I recently deployed the "Nexus Pro" repository manager from Sonatype for approximately 50 developers and I must say that I was impressed with the ease of use, flexibility, power and feature set of this particular solution. The Apache Archiva solution can also be a good choice. I do not have any first-hand experience with any other product.

Have fun with Maven and your corporate repository manager.

Laurent

Read more...

The perceived risk of OSS

>> Sunday, September 27, 2009

It is my experience that managers in most large corporations still feel threatened by OSS (open source software). Managers are insecure with the idea that software modules (I will adopt the term "artifact" from the Maven parlance) can be introduced more or less intentionally and very easily in software systems. In the Java world, OSS modules are just part of the landscape and nobody reasonably envisages the development of general business applications without taping into the rich OSS ecosystem. Furthermore, a build system like Maven works on the premise that it has direct or indirect access to different public artifact repositories. Moreover, industry standard frameworks such as Spring are built and operate on top of a stack of freely available and industry standard open source artifacts.

I recently performed an audit on various Java-based applications under development in a large organization. The assignment was to generate the list of artifacts used by the various sub-projects. The concern was that developers were using "unapproved" and potentially "dangerous" artifacts. A first analysis revealed that, all in all, the various projects used or included approximately 140 different OSS artifacts. That revelation came as a shock to most managers but was not a surprise to me. In any event, the result of the audit did not calm the worries inside the organization.

In an effort to get a better grasp on the issue, we removed from the list all artifacts that came as dependencies of either Spring or Axis2 (the two main frameworks used by the projects) and of a third, commercial system that was central to the overall solution. This reduced the number from 140 to less than 20. In other words, what was first perceived as the result of somewhat uncontrolled, developer initiatives ("anybody could and was adding dependencies to more or less obscure artifacts") was in fact the result of using industry standard OSS frameworks (Spring and Axis2) and of a commercial system that has dependencies on a large number of OSS artifacts.

To me, the paradox is that many managers perceive OSS as a threat or, at the minimum, a risk while it should be perceived as an opportunity. When you stop and think about it for a little while, if managing risk is an important concern (and it should be)  for an organization that develops and uses software then how do you mitigate risk when developing software? To me, that is a very real and important question.

I agree that risk is a somewhat fuzzy concept and that it can be evaluated from many different perspectives depending on whether you are a legal advisor, a financial officer, a quality control specialist, a security officer, an operations person, a marketer, a software developer etc. You may be concerned by eventual law suits, by the eventuality of  intrusions and/or theft of corporate data, by the perrenity of your supplier and his solution, by the stability and performance of the software, by the short and long-term TCO of a given solution etc.

Now, consider the following options and assess the importance of risks of each one:

Option 1: Develop most, if not all, your software in-house using a large number of freshly hired staff or consultants and go live with N thousand lines of more or less proven code.

Option 2: Minimize the amount of in-house written code and favor the use of mature and well supported and recognized commercial but closed source software.

Option 3: Minimize the amount of in-house written code and favor the use of mature and widely accepted  and recognized open source code.


My own opinion is that from whichever perspective you look at the problem of risk management and mitigation, option 3 is very often the less risky and thus, the better one. This does not imply that OSS should be used without any precautions and guidance. In fact, in the modern world, a new role is required within organizations. I call this role "artifact librarian" and with it, one must put in place systems and procedures to ensure that teams follow guidelines and policies from both a technical and a legal standpoint. That being said, the objective of the organization should not be to restrict or limit the use of OSS. On the contrary, it should be to encourage it while managing the OSS selection and introduction process.

So long,

Laurent

Read more...

Agile Reflections?

>> Friday, September 18, 2009

Welcome

I am starting this blog in an effort to share my thoughts (reflections or réflexions in French) on the agile phenomenon, on where it seems to be heading and where I think it should go. My hope is to contribute to the debate and to bring some refreshing and maybe provocative ideas on the whole matter. Being a rather practical and down to earth guy, I also want to share tricks of the trade and experiences that can be useful to others.

I have been an agile movement aficionado for quite some time. It dates back to when I was a Smalltalker and was attending conferences and seminars on the C3 project and its follow-ups. I have practiced agile in many situations and projects and have always tried to stay away from formal methodologies while maintaining a keen and acute preoccupation for development team productivity. As as sidebar, I have a real concern for productivity as a concept and I feel it fits very well with the agile one. As fa as I am concerned, productivity has everything to do with moving along at the highest velocity possible without compromising quality (however you measure such a thing).

Coming back to agile methodologies, I recently landed in the middle of a big project in a large organization that has officially adopted the Scrum/Agile methodology. While familiar with Scrum concepts, it was quite a shock for me to see to what extent it has been formalized and codified and made into something that really smells more and more like one of the traditional methodologies at least to the dimension of how prescriptive it is (or how people present it and apply it). We have Scrum rules and metrics embedded in tools, we have all sorts of scrum "ceremonies" (from daily scrums to sprint retrospectives), we have poker planning, full-time scrum masters, burndown charts etc.

Coming into this later project, my first "reflection" was: where there is buck to be made, there will be products sold and consultants to offer advice but that is my cynical side. My second thought was Gee! this organization is really happy to pay all these people to do this amount of talking and discussing. I can think of more boring things to do but are we really being productive? After a few months of this formal scrum regime I am sort of getting use to it and it not that bad. Nevertheless, I still think we could be more effective if the organization would spend more time assessing the why, what, who and how of any given meeting instance (a ceremony in Scrum parlance) instead of "going by the book". I will take the time to elaborate on this with details and examples in a follow-up article.

To close-off this initial article, I want to mention that I have been working (sometimes fighting) with Maven technology a lot lately. There are many interesting and useful things to write about Maven. It should take up a relatively large part of the bandwidth in the next few weeks.

So long,

Laurent

Read more...

  © Blogger template Webnolia by Ourblogtemplates.com 2009

Back to TOP