my problem w/ ibiblio is that while it has a lot of artifacts, it's missing a bunch as well (ie: testNG).

i'd prefer not to create a local repository for these files as it's just one more thing for me to maintain and keep up-to-date.

what is everyone else doing to deal w/ this scenario?

As you just said, ibiblio doesn't handle all the jars that we can need. The only solution to that is to create you own repository with the missing libs.
Just use a chain resolver that handles your local repository and ibilio for the projects that can be found there. Using such a solution, you have just to maintain a minimum of libs and ivy files into your local repo.

-- Matthieu

At our company, while we were still using a maven build, we had very good luck with maven proxy.

http://maven-proxy.codehaus.org/

We set this up on a local machine, and had our build point at it. It would fetch and cache jars from ibiblio, mimicing the directory structure on-demand. Thus, only the first person requesting a jar would have to deal with the ibiblio slowness. All subsequent requests would then be retriieving the jar over a LAN, which was quite snappy.

For artifacts not available on ibiblio (of which there are legion), we would just download them ourselves and put them in the appropriate place in the maven proxy cache. Worked like a charm.

This worked perfectly well with ivy, as you would imagine.

However, we've now moved to checking in all of our artifacts into our Subversion repository. I've written an Ivy resolver for subversion, and it works well. Once Xavier sets up the ivy tools site at sourceforge, we can get that checked in and available for anyone else that's interested.

i will have to take a look at the maven proxy - thx for the link. i do agree that my only refuge at this point is to setup my own repository.

may i ask why you've decided to check in all of your artifacts into version control (i assume you mean 3rd party and "yours").

i used to do that at a previous job for 3rd party jars (the build was structured so dependant jars were built first), but found that drastically slowed down "fresh" checkouts, and was just a general PITA when adding new libraries b/c i'd have to go back and create a classpath reference.

you eliminate the checkout problem b/c of the subversion resolver, but i'm curious what the gain is. i used to think it would be wise b/c when i'd tag a build, i'd always be able to check out the exact same jars that were used the first time around. this may still hold true for jars that you generate if you aren't changing the version number all the time, but doesn't seem to make much sense for 3rd party libs (as they seem to change version #s on each release). given that my ivy.xml files will all be tagged, i'll still be using the correct version of the lib at compile time.

i've been battling w/ this topic ever since i started work on this new project, and i may decide to go back and check stuff into version control (although i'd need a cvs ivy resolver).

The artifacts are checked in to a completely different part of the subversion repository than our projects, so tagging our projects has no effect on the artifacts. As you said, the ivy descriptor for our project ensures we get the same set of jars every time, regardless of where the jars come from.

The reason we've checked them in is because it centralizes the bits needed for building our various projects. By giving someone access to the source repository, we've essentially given them access to the artifact repository as well. There's no additional steps needed to make sure they have permissions to some local machine share so that they can add or upgrade dependencies as needed. We have contractors working overseas, and we have to give them access to all the servers needed to build and work with our product. Limiting their access as much as possible makes our IT department happy. And it reduces the possible points of failure for our build: if the source repository is available, then you can be assured that the build should work the same for everyone.

The question is then, why use ivy at all? If we're checking in the jars into the source repository, why not just check them into the project itself?

Our source repository hosts multiple projects, each with multiple branches. These projects each have their own dependencies, and these dependencies change over time and with each branch. So the desire to manage those dependencies in an intelligent way is not lessened just because we choose to retrieve the dependencies from the same repository as the source code.

Plus, using ivy helps maintain control of exactly what dependencies we need and why. Our main project, a webapp, currently deploys with 40-50 3rdparty jars, and that number is only increaing. Using ivy, we never have to guess why, for example, we're deploying commons-codec.

Thanks for those explanations, scott !

I'm not really a big fan of putting jars under source control management, that's why ivy has no built-in scm based resolver. But i now understand the advantage it can have mainly for administration. I still consider it's better to set up a shared directory when it is possible, but as you point it we are often sure that a scm is available to everybody working on a project, this is not the case for a shared directory or web site. Good point, it will make me further think about a CVS resolver.

Xavier

i second that, thanks for the great explaination.

i am planning on checking in my local ivyrep files into version control, so i guess i either need to look into writing a cvs resolver (when will the subversion one be ready so i can just mimic that one?) or publish the ivy files to the local maven repository i'm going to set up.

It sounds like Xavier may be planning on writing a CVS resolver, now that I've "shown him the light". :) As for the Subversion resolver, I'm just waiting for Xavier to setup the Ivy Tools project on Sourceforge/java.net/codehaus/some other project hosting site. If you're itching to code, jgangemi, I can email you a snapshot of what I have right now.

Creating a Subversion resolver was simplified greatly by writing a URLConnection adapter for it. While all you really need to do is implement DependencyResolver, there's a huge amount of functionality you'll get for free by extending AbstractURLResolver. All of the Ivy resolvers extend that. The trick, though, is that (as its name suggests) it is coupled tightly with the java URL mechanism. So if you can represent your dependency source (in this case, a CVS repository) as a URLConnection, you're most of the way there.

If I had a recommendation for making the API easier to extend, I would want AbstractURLResolver to be less dependent on the URL class, and work more closely with the java.io classes (e.g., InputStream/OutputStream). There's just too much logic in AbstractURLResolver to not extend that class. If there were some intermediary class, like an AbstractStreamResolver, for example, I would've had more flexibility in how the SVN resolver could've been implemented.

I agree with you, scott, it would really be better if most of the logic found in AbstractURLResolver could be found elsewhere. At the beginning, it was quite a simple class, but it evolved a lot with Ivy versions, and now it's difficult to write a resolver which does not extend AbstractURLResolver, and should thus conform to url. I'll see what can be done in this area to make resolver writing simpler and less dependent on url.

Concerning the opening of a sourceforge project, I tried to create one on thursday, and unfortunately it was rejected. Here is the description I provided:
Quote:

The rejection detail do not give much detail, it's a generic one saying that they lacks information to accept the project. Anyway, some have suggested to use java.net instead (codehaus will not accept such a project, they maybe could accept Ivy itself but not IvyCommunity). So I'll see if I can better open a java.net project, with hope that it won't be rejected. But I'm afraid it will still take some time to have this project setup. Sorry all for the delay.

Finally, concerning the CVS resolver, I have to check with my company staff if I write one soon or not. Waiting for this, a solution can be to checkin your files in CVS, and checkout somewhere in your filesystem. You can then use a filesystem resolver using the checked out files. If you add a cvs update call to your build file before calling resolve, it should be ok, and quite easy to change when CVS resolver will be available.

Xavier

Re: Sourceforge. That's too bad. Perhaps, though, you need to punch up the description a bit. I gave it a shot with this:

Quote:

I'll let you know if it gets through. If it does, I'll happily make you, Xavier, the project admin.