Where am I? Or, adding a counter to a running R process

A quick post in response to a question that came up on Facebook via Liz Freedman and Nic Campione: how do you add a counter/ticker to a running process in R? This is something that can be really helpful if you’re still in the process of trying to debug some code, as a process that involves thousands of iterations may be either stuck and broken, or it could be just taking its sweet time to finish, and without any output during the iterations, you can’t know which it’s doing. So, in one line, here’s a simple way of adding a counter to a for loop (although while and other style of loops follow the same logic):

for (i in 1:1000) {
  if (i%%100==0) print(paste("This is iteration number ", i))
  #all your complex code can go down here, whatever it is
}

In this case, I just added a rule that printed the iteration number every 100 loops (the code can be run directly in a terminal if you want to see what I mean). The two percent signs right beside each other are the modulo operation, so that when the remainder of the iteration is zero (that is, every 100 loops), it prints out a message. The number can be changed easily so that it is every 1000 instead of 100, and you can also use different rules, but I find that this is a relatively quick and easy way to add a counter.

As for making the whole process run faster, using something like the foreach package (and associated packages like doMC or doParallel) can significantly speed things up if you have a multicore processor on your computer.

My new home in Grande Prairie

museum94-992

The new museum, when it’s done.

At the beginning of this month I started my new job in Grande Prairie with the Philip J. Currie Dinosaur Museum near Grande Prairie, Alberta. Although northern Alberta doesn’t have the same sort of recognition that southern Alberta does when it comes to dinosaurs, this museum is looking to change that (and I’m going to do everything I can to make sure that happens). Getting hired on at the museum will let me continue to do the work I’ve started in northern and western Alberta (and other northern places, too), and set me up even closer to my field sites. Despite the whirlwind first couple of weeks, I’ve already been able to spend time out in the field, looking for new sites that might yield something interesting.

stomped_024

The museum right now, which now has concrete poured and is not just literally a hole in the ground.

For the most part though, the next year will be a lot of work making sure that when this museum opens that we have a great story inside. This means a lot of reading and a lot of research, making sure that all of the details of the exhibits are accurate and up to date. What many people don’t realise is just how active the science of  palaeontology really is, with new papers being published on a daily basis. Even just within dinosaur palaeontology, there is a new species published about every 2 weeks, and even in the last year, there have been 5 species named just from Alberta, which is already one of the best studied areas in the world.

I’m really looking forward to working with the great team we have, and I can’t wait to take my two kids on a tour once it’s done.

Of data repositories and unforeseen circumstances

I was recently doing some research into the reproductive habits of crocodylians, looking for some possible information that might also apply to dinosaurs. While doing this, I was referred to a paper written by the the crocodile expert John Thorbjarnarson (who I will admit, I hadn’t actually heard of before), who tabulated reproductive data such as numbers of eggs, body mass, etc., on all of the living species of crocodylians. “Great”, I thought to myself, this is exactly the kind of data I as looking for. I started looking through the paper in a bit more detail, and saw that although he had already gathered all of the data that I was looking for, he had only published summaries of that data. No problem, I was thinking, the paper says to just contact the author for the original dataset. A little searching later turned up something a little more depressing than an email address: his obituary. He had tragically and unexpectedly died of malaria in 2010. 

This brings me to the point of this post, about data archiving. John likely couldn’t have placed the data he collected in a long term, accessible archive like Dryad or Morphobank (and the many others that serve similar purposes) because the paper was published in 1996. However, the excuses I sometimes hear still that “the data is in the paper” and “if people want it, they can just ask me” don’t really apply in 2013. Of course, this is somewhat of a unique situation, but how many other scientists have been struck down by unforeseen circumstances? How many hours of work in compiling this data set (and many others) have been lost, even if it was something more mundane like a computer crash? The point is, you never know what the future may hold, but you can make sure that your scientific legacy persists well beyond yourself. You never know how far down the line (17 years in this case) someone might be looking to carry on your work.

TNT and Ubuntu

TNT (or ‘Tree analysis using New Technology’) is a relatively fast parsimony-based phylogenetics program. However, to be quite frank, it is also totally confusing. It took me some time, but I have put together a script that does pretty much everything I want it to do now, so I can just reuse the script every time I need to do something. I’m posting it here, in the hopes that other people will not have to spend so long figuring this all out. Just note though, the actual commands are in this type of font.Of course, everything that follows is what I’ve figured out through reading papers and trawling the internet, so suggestions are appreciated.

First things first, make sure your data is in the correct format. Theoretically, TNT can handle .nex files, but then again, my old car could theoretically do 220 km/h because the speedometer went that high. Your best bet is to just take your .nex file and reformat it to .tnt format, which pretty much involves deleting everything except the actual matrix, and then editing the first few lines. The first line should read ‘xread’, with the next line containing the number of characters, a space, and the number of taxa. (You can add a line between those for a comment, just be sure to put the comment in quotes)

For example, a simple data set might look like this:


xread
‘sample phylogeny’
5 3
taxonA 00011
taxonB 01111
taxonC 11111
;

While TNT is not the most user-friendly program, using scripts can help a lot, as you often will run the same set of analyses on different data sets. For my purposes, most of the time I want to run a basic tree search to find all of the most parsimonious trees, find a consensus tree from that (if necessary), map the characters, get bootstrap values and finally find the Bremer supports. At the bottom of the page I’ve posted my default TNT script that I use. Note that it does include comments (lines with a # in front) that will need to be deleted for the script to run properly. Also, replace ‘samplephylo’ with whatever you’ve named your .tnt file. Finally, when you’ve done all that, you need to move both the script and the .tnt file into the folder that contains all the files for the TNT program (on mine, the folder is called ‘tnt64-no-tax-limit’).

Finally, once you’ve done all that, you can run TNT (usually by navigating in a terminal to that same ‘tnt64-no-tax-limit’ using the change directory command [i.e. cd]), and then running TNT by typing something like ‘sudo ./tnt’. Note that TNT typically doesn’t run properly unless you run it as root/administrator (same applies to Mac OS X, except you’d run it as ‘sudo ./tnt.command’).

Often, TNT will throw error messages for no apparent reason. A few things to make sure of is that all the file names are letters or numbers only (no punctuation, including periods or underscores, and no spaces), that both the command script and .tnt file are in the same folder as the TNT program and that the program is being run as root/administrator. Hopefully by putting this up here, more people will be able to finally figure out how to use TNT.


#take everything after here and put it in a file in the tnt folder
#delete all the lines that start with the # sign
#replace the ‘samplephylo’ with the name of your file. I recommend converting
#your matrix to .tnt format, which is pretty simple
#check out the example.tnt file in the tnt folder to see how it’s done
#the most imprtant part is the first few lines,
#but be sure to convert any polymorphisms to [] brackets
#load the data file
procedure samplephylo.tnt;
#write all the output to this file
log samplephylo.out;
#do a basic run – finds a bunch of parsimonious trees quickly
mult ;
#this should find all the MPRs
bbreak=tbr;
#find the consensus tree
nelsen * ;
#map the synapomorphies on the tree
apo [ ;
#export the MPRs and the consensus tree to a file
export samplephylo.tre;
#do a bootstrap analysis – might need to up the runs
resample ;
#the stuff below does a Bremer analysis
#it’s a bit convoluted, but is the only way to get
#TNT to do a Bremer, because TNT isn’t really meant to
#do a Bremer analysis
hold 1000; sub 1 ; bbreak=tbr;
hold 2000; sub 2 ; bbreak=tbr;
hold 3000; sub 3 ; bbreak=tbr;
hold 4000; sub 4 ; bbreak=tbr;
hold 5000; sub 5 ; bbreak=tbr;
hold 6000; sub 6 ; bbreak=tbr;
hold 7000; sub 7 ; bbreak=tbr;
hold 8000; sub 8 ; bbreak=tbr;
#calculate the Bremer support from the suboptimal trees
bsupport ;
#quit the program – you can always do this manually
quit

MrBayes and Ubuntu

A quick tutorial on how to install MrBayes on an Ubuntu system.

Step 1. Open Software Centre

Step 2. Search for ‘mrbayes’ and hit enter

Step 3. Click ‘Install’

Or, alternatively, open a terminal and type ‘sudo apt-get install mrbayes’. And to run MrBayes, open up a terminal and type ‘mb’. That’s it. I just spent 15 minutes downloading and compiling the software from source, when it was already done for me. Oh well.

pdflatex errors and R

I was getting an error checking packages in R on an Ubuntu 10.10 machine using the ‘R CMD check’ command saying:

LaTeX errors when creating PDF version. This typically indicates Rd problems.

I have no idea what R’s problem with pdflatex was (I’ve been using pdflatex for writing papers just fine) but once I installed the texlive-full package (a metapackage that installed all sorts of other packages) the check ran without any errors. If in doubt, this package seems to install everything that might be missing (along with the kitchen sink).

ape and geiger in R

Just a quick tip if you want to use the geiger package in R on an Ubuntu system, there are a few things you need to make sure you do. First, make sure you’ve installed the gfortran package and lapack-dev package in order to build the source packages. As well, you may need to reinstall the r-base package after you’ve done this. I was getting an error message like this:

/usr/lib/liblapack.so.3gf: undefined symbol: ATL_chemv

It seemed to go away after I reinstalled r-base and restarted the R session, although I’m not sure I needed to do both. Either way, it works now.