Benefits of a Repository Manager

>> Monday, September 28, 2009

For organizations and projects of any reasonable size that use Maven as their build tool, the deployment and use of a Maven repository manager is a must. As Maven users know, Maven requires access to standard format repositories from where it can retrieve the various artifacts that projects declare dependencies on. In fact, Maven's operation is based on the existence of public repositories that follow the Maven nomenclature and versioning scheme (artifact coordinates in mavenese). Since artifacts declare dependencies on other artifacts, it is quite impressive to see how quickly a reference to a major open source framework translates into dependencies on a multitude of other, public, artifacts. By configuring Maven to automatically retrieve these from the web, the task of retrieving all the various jars one by one and of making sure they have the right version number suddenly disappears.

This is the first obvious benefit that comes with the use of Maven. However, what new Maven users do not always realize is that the right way to do things is to host your own repository manager and to thus control and optimize artifact distribution and uploads. There are various repository management solutions available out there and a web page on Codehaus that compares three of the most popular ones is available here.

In my view, the most important benefits that come with a corporate repository manager are the following:

  1. The caching of artifacts used by the various projects in the organization (released artifacts are fetched from the web once and made available to all from then on).
  2. The grouping of several repositories into groups to reduce the complexity of maven configurations (settings.xml files)
  3. The possibility to install non-public artifacts required or used by the various projects (these can be legacy modules, third party modules or other)
  4. The possibility to enforce various access control policies for both reading and deploying artifacts (snapshots or releases)
  5. The flexibility brought on by the fact that repositories can be easily added, removed, reconfigured, disabled, reenabled etc
  6. The possibility that some some repository managers provide in terms of procurement so that an organization can control which artifact is available to whom for legal or technical considerations.
  7. The possibility to automate certain tasks such as the removal of outdated snapshots or the reindexing of public repositories.
  8. The availability of a user-friendly console to manage access, search, upload, remove and update artifacts.
I recently deployed the "Nexus Pro" repository manager from Sonatype for approximately 50 developers and I must say that I was impressed with the ease of use, flexibility, power and feature set of this particular solution. The Apache Archiva solution can also be a good choice. I do not have any first-hand experience with any other product.

Have fun with Maven and your corporate repository manager.



The perceived risk of OSS

>> Sunday, September 27, 2009

It is my experience that managers in most large corporations still feel threatened by OSS (open source software). Managers are insecure with the idea that software modules (I will adopt the term "artifact" from the Maven parlance) can be introduced more or less intentionally and very easily in software systems. In the Java world, OSS modules are just part of the landscape and nobody reasonably envisages the development of general business applications without taping into the rich OSS ecosystem. Furthermore, a build system like Maven works on the premise that it has direct or indirect access to different public artifact repositories. Moreover, industry standard frameworks such as Spring are built and operate on top of a stack of freely available and industry standard open source artifacts.

I recently performed an audit on various Java-based applications under development in a large organization. The assignment was to generate the list of artifacts used by the various sub-projects. The concern was that developers were using "unapproved" and potentially "dangerous" artifacts. A first analysis revealed that, all in all, the various projects used or included approximately 140 different OSS artifacts. That revelation came as a shock to most managers but was not a surprise to me. In any event, the result of the audit did not calm the worries inside the organization.

In an effort to get a better grasp on the issue, we removed from the list all artifacts that came as dependencies of either Spring or Axis2 (the two main frameworks used by the projects) and of a third, commercial system that was central to the overall solution. This reduced the number from 140 to less than 20. In other words, what was first perceived as the result of somewhat uncontrolled, developer initiatives ("anybody could and was adding dependencies to more or less obscure artifacts") was in fact the result of using industry standard OSS frameworks (Spring and Axis2) and of a commercial system that has dependencies on a large number of OSS artifacts.

To me, the paradox is that many managers perceive OSS as a threat or, at the minimum, a risk while it should be perceived as an opportunity. When you stop and think about it for a little while, if managing risk is an important concern (and it should be)  for an organization that develops and uses software then how do you mitigate risk when developing software? To me, that is a very real and important question.

I agree that risk is a somewhat fuzzy concept and that it can be evaluated from many different perspectives depending on whether you are a legal advisor, a financial officer, a quality control specialist, a security officer, an operations person, a marketer, a software developer etc. You may be concerned by eventual law suits, by the eventuality of  intrusions and/or theft of corporate data, by the perrenity of your supplier and his solution, by the stability and performance of the software, by the short and long-term TCO of a given solution etc.

Now, consider the following options and assess the importance of risks of each one:

Option 1: Develop most, if not all, your software in-house using a large number of freshly hired staff or consultants and go live with N thousand lines of more or less proven code.

Option 2: Minimize the amount of in-house written code and favor the use of mature and well supported and recognized commercial but closed source software.

Option 3: Minimize the amount of in-house written code and favor the use of mature and widely accepted  and recognized open source code.

My own opinion is that from whichever perspective you look at the problem of risk management and mitigation, option 3 is very often the less risky and thus, the better one. This does not imply that OSS should be used without any precautions and guidance. In fact, in the modern world, a new role is required within organizations. I call this role "artifact librarian" and with it, one must put in place systems and procedures to ensure that teams follow guidelines and policies from both a technical and a legal standpoint. That being said, the objective of the organization should not be to restrict or limit the use of OSS. On the contrary, it should be to encourage it while managing the OSS selection and introduction process.

So long,



Agile Reflections?

>> Friday, September 18, 2009


I am starting this blog in an effort to share my thoughts (reflections or réflexions in French) on the agile phenomenon, on where it seems to be heading and where I think it should go. My hope is to contribute to the debate and to bring some refreshing and maybe provocative ideas on the whole matter. Being a rather practical and down to earth guy, I also want to share tricks of the trade and experiences that can be useful to others.

I have been an agile movement aficionado for quite some time. It dates back to when I was a Smalltalker and was attending conferences and seminars on the C3 project and its follow-ups. I have practiced agile in many situations and projects and have always tried to stay away from formal methodologies while maintaining a keen and acute preoccupation for development team productivity. As as sidebar, I have a real concern for productivity as a concept and I feel it fits very well with the agile one. As fa as I am concerned, productivity has everything to do with moving along at the highest velocity possible without compromising quality (however you measure such a thing).

Coming back to agile methodologies, I recently landed in the middle of a big project in a large organization that has officially adopted the Scrum/Agile methodology. While familiar with Scrum concepts, it was quite a shock for me to see to what extent it has been formalized and codified and made into something that really smells more and more like one of the traditional methodologies at least to the dimension of how prescriptive it is (or how people present it and apply it). We have Scrum rules and metrics embedded in tools, we have all sorts of scrum "ceremonies" (from daily scrums to sprint retrospectives), we have poker planning, full-time scrum masters, burndown charts etc.

Coming into this later project, my first "reflection" was: where there is buck to be made, there will be products sold and consultants to offer advice but that is my cynical side. My second thought was Gee! this organization is really happy to pay all these people to do this amount of talking and discussing. I can think of more boring things to do but are we really being productive? After a few months of this formal scrum regime I am sort of getting use to it and it not that bad. Nevertheless, I still think we could be more effective if the organization would spend more time assessing the why, what, who and how of any given meeting instance (a ceremony in Scrum parlance) instead of "going by the book". I will take the time to elaborate on this with details and examples in a follow-up article.

To close-off this initial article, I want to mention that I have been working (sometimes fighting) with Maven technology a lot lately. There are many interesting and useful things to write about Maven. It should take up a relatively large part of the bandwidth in the next few weeks.

So long,



  © Blogger template Webnolia by 2009

Back to TOP