Wednesday 27 February 2008

Maven updates its plugin metadata then says it can't find the plugin - WTF?

Trying to generate the IntelliJ project files for a Maven 2 project.
Then I got an error:

mvn idea:idea
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'idea'.
[INFO] artifact org.apache.maven.plugins:maven-idea-plugin: checking
for updates from central
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).


Project ID: org.apache.maven.plugins:maven-idea-plugin

Reason: Error getting POM for
'org.apache.maven.plugins:maven-idea-plugin' from the repository:
Failed to resolve artifact, possibly due to a repository list that is
not appropriately equipped for this artifact's metadata.
org.apache.maven.plugins:maven-idea-plugin:pom:2.2-SNAPSHOT

from the specified remote repositories:
central (http://repo1.maven.org/maven2)


Now I look in my Maven repository, in the maven-idea-plugin directory, in

\.m2\repository\org\apache\maven\plugins\maven-idea-plugin

and I find the maven-metadata-central.xml which has this:

<?xml version="1.0"?><metadata>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-idea-plugin</artifactId>
<versioning>
<latest>2.2-SNAPSHOT</latest>
<release>2.1</release>
<versions>
<version>2.0-beta-1</version>
<version>2.0</version>
<version>2.1</version>
<version>2.0-beta-2-SNAPSHOT</version>
<version>2.0-SNAPSHOT</version>
<version>2.1-SNAPSHOT</version>
<version>2.2-SNAPSHOT</version>
</versions>
<lastUpdated>20070716221242</lastUpdated>
</versioning>
</metadata>

But when I look in the location:

http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-idea-plugin/

And get the maven-metadata.xml, it has:

<metadata>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-idea-plugin</artifactId>

<versioning>
<latest>2.1</latest>
<release>2.1</release>

<versions>
<version>2.0-beta-1</version>
<version>2.0</version>
<version>2.1</version>
</versions>

<lastUpdated>20070604220104</lastUpdated>

</versioning>

</metadata>


The only possible reason I can think of for this happening is that
Maven is getting the metadata from somewhere else. We are using
Artifactory and one of the repositories we have set up is an entry
pointing to http://people.apache.org/repo/m2-incubating-repository.
The artifactory.config.xml contains this:


<remoteRepository>
<key>apache-m2-incubating</key>
<handleReleases>true</handleReleases>
<handleSnapshots>true</handleSnapshots>
<excludesPattern>org/artifactory/**,org/jfrog/**,au/com/company/**</excludesPattern>
<url>http://people.apache.org/repo/m2-incubating-repository
<proxyRef>work-proxy</proxyRef>
</remoteRepository>


Now in this repository, there is a directory for maven-idea plugin, at

http://people.apache.org/repo/m2-snapshot-repository/org/apache/maven/plugins/maven-idea-plugin/

and there is a file
http://people.apache.org/repo/m2-snapshot-repository/org/apache/maven/plugins/maven-idea-plugin/maven-metadata.xml
that contains:


<metadata>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-idea-plugin</artifactId>
<version>2.0-beta-2-SNAPSHOT</version>
<versioning>
<latest>2.2-SNAPSHOT</latest>
<versions>
<version>2.0-beta-2-SNAPSHOT</version>
<version>2.0-SNAPSHOT</version>
<version>2.1-SNAPSHOT</version>
<version>2.2-SNAPSHOT</version>
</versions>
<lastUpdated>20070716221242</lastUpdated>
</versioning>
</metadata>


Now the strange thing is that this info is not exactly the same as
what's in my local repository. So it looks like the
maven-metadata-central.xml I have for maven-idea-plugin did *not* come
from this location. So where did it come from? I should probably add
to the exclusion list for the above repository so it doesn't try and
get maven plugins from there. But from the error message I got, it
looks like it only goes to the central repository at repo1.maven.org
for plugin updates. If that's the case, why does it seem to go
somewhere else for it's plugin updates?

What the hell is Maven actually doing?

Thursday 21 February 2008

Maven woes # 2923928923829: Maven metadata not reflecting what's on maven repository

(File under "Maven!!! GRRRRRR!")
Alternate longer title: What the hell is wrong with Maven and why do
they push out these SNAPSHOT plugins with releases to other SNAPSHOT
jars that don't even exist? As a matter of principle, should you
really be forcing non-release, snapshot software upon your users?
Here's what I was getting:
Error: Reason: Error getting POM for
'org.apache.maven.plugins:maven-archetype-plugin'
Unable to run Maven archetype because it is trying to download a
plugin that it can't get from the central Maven repository.
Turns out it's because of an inconsistency between the settings it has
in the location
\[username]\.m2\repository\org\apache\maven\plugins\maven-archetype-plugin\maven-metadata-central.xml
and what is actually on
http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-archetype-plugin/
C:\dev>mvn archetype:create -DgroupId=au.com.company.myapp -DartifactId=my-app
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'archetype'.
[INFO] org.apache.maven.plugins: checking for updates from central
[INFO] artifact org.apache.maven.plugins:maven-archetype-plugin:
checking for updates from central
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).

Project ID: org.apache.maven.plugins:maven-archetype-plugin

Reason: Error getting POM for
'org.apache.maven.plugins:maven-archetype-plugin' from the repository:
Failed to resolve
artifact, possibly due to a repository list that is not appropriately
equipped for this artifact's metadata.
org.apache.maven.plugins:maven-archetype-plugin:pom:2.0-SNAPSHOT

from the specified remote repositories:
central (http://repo1.maven.org/maven2)
for project org.apache.maven.plugins:maven-archetype-plugin
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: < 1 second
[INFO] Finished at: Wed Feb 20 17:35:46 EST 2008
[INFO] Final Memory: 1M/254M
[INFO] ------------------------------------------------------------------------

Solution:
1. Go into
\[user]\.m2\repository\org\apache\maven\plugins\maven-archetype-plugin\maven-metadata-central.xml
rename this file or delete it.
2. Get the file
http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-archetype-plugin/maven-metadata.xml
Download it to .m2\repository\org\apache\maven\plugins\maven-archetype-plugin\
and save it as maven-metadata-central.xml
3. Rerun the above process
mvn archetype:create -DgroupId=au.com.mydomain.myapp -DartifactId=my-app
now maven metadata is pointing to files that actually exist on the repository.
But the big questions here are:
Why is there a discrepancy at all? What other repositories could the
maven-metadata-central.xml be talking about if not the main
http://repo1.maven.org? Where did the maven-metadata-central.xml come
from anyway?
As a matter of design, why do I end up receiving - without my asking
for it - bleeding edge snapshot plugins? Shouldn't the policy be to
only use release versions?
Okay, maybe this argument is not really valid in this instance because
we are talking about the archetype-plugin. From the looks of what's on
the repository, it went out as 1.0-alpha-3 then to 1.0-alpha-5, then
1.0-alpha-7 - and now it's at 2.0-alpha-1. Good god, is this thing
*ever* going to get to an actual release number, one that doesn't have
greek letters after the .0?

Update on 6 Oct 2011:
NOTE: As commenters below have pointed out, the mixup normally occurs when you are - probably unknowingly - getting artifacts from both release and snapshot repositories.
The thing to remember is to make a distinction between release and snapshot repositories. The fact that you are pointing to a public repository is not enough. A repository obviously has to be public so that other people can access it. The question is: what types of artifacts are the repository providing? They may actually have both types.  

Friday 15 February 2008

iPhoto alternative found! Photo Mechanic - retains your directory structure, doesn't have its ownf format for your photos, allows editing of EXIF metadata

I've been trying to move my photos to my new iMac, but the vexed issue
of whether or not to use iPhoto is.. uh, pretty vexing. I've heard
iPhoto stores all of its photos in pretty much one database file, and
I've got about 50Gb of photos accumulated over the past 5 years, so I
want to be sure that 1) it won't slow down to a crawl and try to load
all my photos every time it runs; 2) will retain any EXIF information
I have in them and allow edits to them as well.

I found this blog entry about a program called PhotoMechanic that
seems to fit the bill and do a lot that I've gotten used to in Picasa
on the PC. It retains all the photos in their own directories,
according to how I've organised them, it allows editing of a *lot* of
EXIF metadata (which on the PC i perform using the excellent utility
Exifr), and it doesn't force you to add them to its database of
photos.

It doesn't allow photo editing, and is not as good at slideshows, but
I don't do much of the latter anyway. I hope I find a clean way of
integrating the photo editing into my photo workflow.

http://www.comatosed.ca/writing/writing/reviews/exif_iphoto_vs_photomechanic1.html

I'll download it and see if it's a better fit with the way I use my photos.

<2 minutes later>

Oh great. They don't sell it online.

"We do not have web sales setup yet. To place an order for the
products above, please contact Camera Bits, Inc. directly. We accept
Visa, Mastercard, and American Express credit cards for payment."

Haven't these people heard of Kagi? (http://www.kagi.com/index.php)
Shareware vendors have been using this for years!

Where is my Picasa for OS X? Pleeeease, Google-matrix-skynet, out with
it already! But put in a plug-in architecture so that people can
develop Flickr uploaders. I know you have your Picasa Web Albums but
it's not in the same league, not even the same country, not even the
same planet as Flickr.

Thursday 14 February 2008

I've never been to me

I'm feeling kinda emotional tonight, as I found out that my bank
balance is NEGATIVE, which kinda threw shit on my plans to have a
romantic Valentine's dinner with my wife at the local Vietnamese
place.

Now when I find myself in times of trouble, I turn to the classics. In
this case, the nearest one on hand was Charlene's "Never Been To Me".
I'd never listened to it closely before, but I worked out the part
where she's doing the narration, and I didn't realise it's actually a
bit of a cynical song.

"Hey, you know what paradise is? It's a lie, a fantasy we create about
people and places as we want them to be.

But you know what truth is? It's that little baby you're holding. It's
that man you fought with this morning, the same one you're going to
make love with tonight.

That's truth. That's love."

D'ya hear that people?

PARADISE IS A LIE! A FANTASY WE CREATE ABOUT PEOPLE AND PLACES AS WE
WANT THEM TO BE!!!

Now if that ain't fodder for your next up-and-coming emo band, I don't
know what is. Come on kids, work on those Charlene samples and get
'em into those anthemic choruses! I believe in you! Yizzgaarnoff,
maaaate!

That's the truth. That's love.

Wednesday 13 February 2008

Configuring Subversion authentication using Windows Active Directory and Basic Auth from a file - problems encountered and workaround

At work we use Apache+SVN+SSPI to authenticate to Subversion using
Windows authentication. This keeps everyone's SVN logins updated
whenever their network logins change. All well and good, but then we
had to set up a build server using Cruisecontrol, and we are not able
to get a Windows network username for that. So we had to create a user
set up in a config file, and use Basic Auth for that.

WHAT DIDN'T WORK:

The first attempt at configuring Apache's httpd.conf, which was the
example show in other blogs and forum posts was something like:
# Location of our Subversion repository
<Location /svn>
DAV svn
SVNListParentPath on
SVNParentPath "D:\repository\svn"
AuthName "Subversion repositories"
AuthType SSPI
SSPIAuth On
SSPIAuthoritative Off
SSPIDomain NEWS
SSPIOfferBasic On
# need to set this up so only select users/ group can have chk-in access.
# might want to restrict who can read as well.
# make sure this file is outside the SVNParentPath
AuthzSVNAccessFile "D:\repository\svnaccessfile.txt"

AuthType Basic
# make sure this file is outside the SVNParentPath
AuthUserFile "D:\repository\svn-httpasswd.txt"
AuthBasicAuthoritative Off
Require valid-user
</Location>

The problem was it never worked. The Basic Authentication never
worked. Only the Windows Authentication allowed me to get in.




WHAT WORKED:


So I ended up creating another Apache <Location> for use solely by
CruiseControl. And this has worked fine without any problems.

# Location of our Subversion repository
<Location /svn>
DAV svn
SVNListParentPath on
SVNParentPath "D:\repository\svn"
AuthType SSPI
AuthName "Subversion repositories - Active Directory authentication"
SSPIAuth On
SSPIAuthoritative On
SSPIDomain NEWS
SSPIOfferBasic On
# need to set this up so only select users/ group can have chk-in access.
# might want to restrict who can read as well.
# make sure this file is outside the SVNParentPath
AuthzSVNAccessFile "D:\repository\svnaccessfile.txt"
Require valid-user
</Location>
# we were unable to get the /svn working to use BOTH Active Directory
and Basic Auth from a file,
# so we have to set up another one.
# Location of our Subversion repository used only by CruiseControl
# all others should use the standard /svn URL defined above
<Location /cruisecontrol>
DAV svn
SVNListParentPath on
SVNParentPath "D:\repository\svn"
AuthName "Cruisecontrol Subversion repositories"
AuthType Basic
# make sure this file is outside the SVNParentPath
AuthUserFile "D:\repository\svn-httpasswd.txt"
Require user cruisecontrol
</Location>

Tuesday 12 February 2008

GTD - toread

I have yet to achieve GTD nerdvana. Feeling a bit overwhelmed with my
lists, not checking them enough, and haven't implemented enough task
breakdowns and specific, more physical "Next Actions". Actually, I
only have a big to-do list, not broken down into Projects (with more
than one action) and simple To Do list. And I don't have that Someday/
Maybe list.

I mentioned it to a friend and he sent me a link:

http://lifehacker.com/software/feature/practicing-simplified-gtd-335269.php


Then all these links got added, as the article links to these, etc etc.

http://lifehacker.com/software/geek-to-live/the-art-of-the-doable-to+do-list-270404.php

http://www.43folders.com/2007/06/19/buffington-igtd-01

http://www.43folders.com/2005/09/12/building-a-smarter-to-do-list-part-i

http://www.43folders.com/2005/09/13/building-a-smarter-to-do-list-part-ii

http://www.macworld.com/article/51703/2006/07/augworkingmac.html


http://lifehacker.com/software/plain-text/geek-to-live--monitor-your-mac-and-more-with-geektool-244026.php

http://lifehacker.com/software/getting-things-done/getting-into-the-weekly-review-habit-278118.php

http://lifehacker.com/software/top/geek-to-live--empty-your-inbox-with-the-trusted-trio-182318.php

http://lifehacker.com/software/personal-organizers/geek-to-live--extreme-makeover-filing-cabinet-edition-155333.php

http://jonathanstoolbar.blogspot.com/2007/02/12-to-do-find-to-do-list-manager.html


Then found software that looks perfect for GTD (and it's free):

http://www.thinkingrock.com.au/

Sunday 3 February 2008

Email to Flickr about Microsoft bid for Yahoo + Flick reply

Sent this email to Flickr recently, just to tell them I care. Yes, I truly do:

"I am terribly depressed about the news of the Microsoft bid for
Yahoo. Would there be any way on earth of ensuring that MS keep their
fists of ham away from Flickr, thus keeping it as the world's best
photo sharing site? Somehow, I think the answer is no. The very least
they would do is re-brand this sucka, which I can live with. But
almost certainly they won't leave it at that. That would be too
different from past behaviour.

I can't imagine that this mass acquisition of web properties won't tie
into their grand scheme of "fixing this platform independent internet
problem". Of course they'd use this to extend their hold on the
desktop. A Flickr uploader that requires Vista *and* Silverlight?
Start boning up on your .Net, guys... And maybe they'll get some of
those ad dollars going to Google... ads and IE-only pages on Flickr,
even for paid users... coming soon.

Good thing I don't keep stuff on Yahoo Mail, or I'd be in utter torment. "

--------------

(updated)

And last night got a reply from Flickr:

"Hello,

Thank you for contacting Flickr Customer Care.

On January 31, 2008, Yahoo! received an unsolicited proposal from Microsoft to acquire the Company. The Company's Board of  Directors will evaluate this proposal carefully and promptly in the context of Yahoo!'s strategic plans and pursue the best course of action to maximize long-term value for shareholders.

We remain committed to you and to continue to provide you with industry-leading products and service.

Thank you again for contacting us. If you have any other questions, please feel free to reply to this email.

Regards,


Maria
Flickr Customer Care"

Saturday 2 February 2008

Unix tips: count total number of lines in a directory tree of files

to count the number of lines in a subdirectory of files.

find monthly_reports/ -iname '*.csv' \
| xargs -n 1 wc -l \
| cut -d' ' -f1 \
| (SUM=0; while read NUM; do SUM=$(($SUM+$NUM)); done; echo $SUM)

PROBLEM: this has trouble if file names have spaces in them.


To find out how many lines in total are contained in a subdirectory of
files, there is the "Wc" command, which is a recursive version of "wc"

Wc -l monthly_reports/*/*.csv

-l specifies count the lines