Problem and solution: "Content not allowed in prolog" when loading XML as string, even without extra characters

I was parsing XML files in a unit test, but was doing it incorrectly.

I was loading them from classpath, then wanted to have them in a string and compare them using XMLUnit.

WRONG WAY

InputStream devStream = getClass().getClassLoader().getResourceAsStream("api-response--dev.xml");

String devXml = devStream.toString();

I was getting "Content not allowed in prolog" when I tried to parse this as XML, even though there was no BOM or extra characters in the source XML file.

Turns out I was doing it incorrectly. Because if you call toString(), you end up using the default encoding of your platform. 

RIGHT WAY

InputStream devStream = getClass().getClassLoader().getResourceAsStream("api-response--dev.xml");

String devXml = CharStreams.toString(new InputStreamReader(devStream, "UTF-8"));

How to fix "Unlink of file '.git/objects/pack/pack-(longhash)' failed. Should I try again? (y/n)

possible tags: Git GC, git cleanup repository, 

Every now and then while using Git, like say when you do a "git pull" on a repository which regularly gets a lot of changes, you will get a message from Git like this:

Auto packing the repository for optimum performance. You may also

run "git gc" manually. See "git help gc" for more information.

Counting objects: 499419, done.

Delta compression using up to 24 threads.

Compressing objects: 100% (164548/164548), done.

Writing objects: 100% (499419/499419), done.

Total 499419 (delta 238160), reused 493660 (delta 233258)

And then you might see this:

Unlink of file '.git/objects/pack/pack-061698b76ddcbea46c9d31874b73c9a66d87f790.pack' failed. Should I try again? (y/n)

n

Unlink of file '.git/objects/pack/pack-061698b76ddcbea46c9d31874b73c9a66d87f790.idx' failed. Should I try again? (y/n)

You'll find that if you press "n", you'll find yourself pressing "n" a *lot* of times... this is pretty much the point where you should just go:

Ctrl-C 

and then just do a 

$ git gc

to manually pack your repository.

$ git gc

Counting objects: 499419, done.

Delta compression using up to 24 threads.

Compressing objects: 100% (159646/159646), done.

Writing objects: 100% (499419/499419), done.

Total 499419 (delta 238160), reused 499419 (delta 238160)

Removing duplicate objects: 100% (256/256), done.

Checking connectivity: 499419, done.

Done!

If there's an error, isn't it better to fail and show the error?

# if we didn't have a placeholder, we wouldn't have to escape the single quote

# but if we have a placeholder, then we have to escape the single quote

#   otherwise: single quote doesnt show AND placeholder doesnt receive our parameter!

mail.body.donate=Hi there, I'd like to donate ${0}

I was using Spring's MessageResource to load this value from a properties file, passing in a single item array with the value I wanted to show after the word "donate"

But for some reason, it kept showing only this:

Hi there, Id like to donate ${0}

I've had this problem for too days, and then eventually I removed this and replaced it with a message that only had letters and spaces, aside from the placeholder

Hi there I would like to donate ${0}

And it replaced it with:

Hi there I would like to donate $300

So it looked like it was either the "," or "'" (single quote) that was the problem.

Eventually, I found this page:

http://www.mscharhag.com/java/resource-bundle-single-quote-escaping

This part had the answer:

Java code:

1 2 3 4 5

private void printMessage(String code, Object... args) {   Locale locale = new Locale("en");   String message = messageSource.getMessage(code, args, locale);   System.out.println(message); }

1 2 3

printMessage("test.message1"); printMessage("test.message2", "John"); printMessage("test.message3", "message");

Output:

1 2 3

John's message Johns message Johns {0}

The first message does not take any arguments, so no MessageFormat is applied and the single quote does not need to be escaped. The second and third messages, however, are formatted by a MesssageFormat which processes the single quote characters. In these messages the single quotes should better be escaped with another single quote character otherwise they won't show up in the output.

I'm not a fan of this "fail, but partly work". I'd rather have complete failure with a clear indication of where things failed, rather than partially working, but no indication of why things are failing.

When a deployment doesn't work, and logs don't show anything: Restart it! (AKA It was working on the other QA box!)

PROBLEM:

We have a number of QA (Quality Assurance, ie, testing) boxes. Release candidates (RC) go on a specific QA box, one that is more prod-like. The problem was an application that was working on the other QA boxes but not on the RC QA box.

We tailed the log files while testing the functionality - no errors.

 

We had a look at the libraries that were deployed. They were as expected.

We manually made calls to the service endpoints using curl - we were getting a 404! 

Okay, now what's going on?

Was it a mistake in the jar we released - in the past we've had some code not getting merged into master during release. Opened the jar - yup, contents were there.

Was it a mistake in the URL? Nope. It works on the other QA box.

Now doing a "ps" would usually show the libraries being loaded on the classpath, but this app was different and didn't actually do that. 

Could it be that the *old* version was still running? We had no quick way of knowing this. But the fact that the webapp didn't know this new endpoint even though we had the new jar file in the filesystem was pointing to this.

So we restarted the process. That fixed it! Functionality all working either through direct calls via curl or the webapp.

LESSON:

When investigating something that should already be working, the steps should be: 

1) Check the logs. 

2) Check that it's actually deployed 

3) If 1 or 2 don't show anything wrong, restart the application.  

Saves so much time.

The culprit was a Jenkins job that deployed the package but didn't successfully kill the old process. 

How do we check that the process is running the new code? Our plan is to have an endpoint that can be queried to show release information - so we know the libraries in use by the application currently running. Our Jenkins job should then check this URL after it's deployed the package, and send out an alert if it's not getting the expected results.

4) Have an endpoint showing the state of the running process, library versions and configuration

5) Get Jenkins to check the release info endpoint after it deploys a package

 

Reading in a file when directory names have spaces in them

PROBLEM:

We were running some errors in our Jenkins build of some code, even though the code works on our computers. 

java.io.FileNotFoundException: File '/var/lib/jenkins/workspace/Email%20Project%20Release/target/test-classes/com/company/mailFolder/sample_email.txt' does not exist

I ssh'd to the Jenkins box and tried to ls on the directory name:

and I got

ls: cannot access 
/var/lib/jenkins/workspace/Email%20Project%20Release/target/test-classes/com/company/mailFolder/sample_email.txt: No such file or directory

Then I realised it was the '%20' -- it was trying to load a url encoded directory name..

Doing an ls this directory worked:

ls -l "/var/lib/jenkins/workspace/Email Project Release/target/test-classes/com/company/mailFolder/sample_email.txt'

I looked in the code and it was doing this:

File mailFolder = new File(getClass().getResource("mailFolder"));

File mailFile = new File(mailFolder, "sample_email.txt");

I tested doing the first line with a folder that had spaces in it and I found it would return a path with any spaces replaced by '%20'

SOLUTION:

From stackoverflow, I found that the best way to get the correct folder reference was to do this

File mailFolder = FileUtils.toFile(getClass().getResource("mailFolder"));

File mailFile = new File(mailFolder, "sample_email.txt");

At first, I tried to load the file directly and it only returned null. Getting the File reference to the folder first, then creating the reference to the file is the only way that works, despite what the Stackoverflow article said (REF).

REF:

http://stackoverflow.com/questions/10536183/resource-files-not-found-from-junit-test-cases

Move module in git repo to its own repository but retain history

http://ariya.ofilabs.com/2014/07/extracting-parts-of-git-repository-and-keeping-the-history.html

This is so much better than having the history of all files start from scratch.

My favourite moment in "Halt and Catch Fire"

S01E09, at around 22:45

Gordon: "... You hit an overhang, and you can't climb up, and it's too steep to climb down. You realise what the word "jump" really means. And if I held your hand and counted to three.. I wasn't sure you'd come with me."

Donna: "We're married, Gordon. We already jumped."

That line just hit me. Hard.