CiteULike + BibDesk: Sync your references and live smarter
June 3rd, 2009 by dario
It should be no surprise that many of us love Zotero, especially since they added support for reference sharing and synchronization.
I am probably the only exception in the AP team. As a longstanding MacTeX user, I keep my references organised with BibDesk, one of the sweetest pieces of (open source) software ever written for TeX users working on Mac OS. When hunting for references, I use CiteULike as a fast and effective solution to bookmark and tag papers. My workflow usually starts with an exploratory phase based on CiteULike. As soon as I have read a paper and need to cite it, I export its reference from CiteULike into BibDesk, filing the PDFs with the help of the autofile functionality in BibDesk. So far I have been quite happy with this workflow even if it involves a little bit of fiddling to correctly import references into my local library.
After some research and discussion in the support forums, I discovered that CiteULike and BibDesk can seamlessly integrate with each other. BibDesk has an external file group option that allows you to read structured references from any online source. CiteULike has a bibtex export feature that allows you to expose your references for consumption by software like BibDesk. This is all you need to get the two to work together. Here’s how my new workflow looks like:
1. Create a new external file group in BibDesk
Click on the cog icon on the bottom left of the BibDesk window and select “Add External File Group”. This will add a new folder in your library pointing to a remote source.
2. Point the new group folder to CiteULike
CiteULike can generate bibtex files on-the-fly from any reference list, e.g. your reference library, the list of papers for a specific author, papers tagged with a specific keyword etc. The bibtex export filter for any of these lists can be obtained by adding the /bibtex prefix to the corresponding URL. For instance, to export in bibtex format my full reference library:
http://citeulike.org/user/dartar/
I simply need to type the following URL:
http://citeulike.org/bibtex/user/dartar/
The following URL parameters are a useful addition:
-
fieldmap=
: allows you to specify translation rules for custom fields, e.g. record creation date (what CiteULike calls Posted-At must be translated to BibDesk’s Date-Added) - do_username_prefix=0&key_type=4 toggles an option to create bibtex keys using an AuthorYearTitle scheme when none is available (which I prefer to the default numeric key)
The final URL we will use is the following:
http://citeulike.org/bibtex/user/dartar?fieldmap=posted-at:date-added&do_username_prefix=0&key_type=4
3. Browse your CiteULike library from BibDesk
Once you have pointed BibDesk to the URL of the CiteULike bibtex filter, you can start directly browsing your CiteULike library from BibDesk. You will see an “Import” button on the left of each item, which is greyed out when the reference is already imported in your local BibDesk library.
4. Import selected items
Click on “Import” to download a reference and file it in your local BibDesk library: you are done!
There is definitely room for further improvement (in particular to allow bidirectional sync’ing of references and PDFs and custom bibtex keys in CiteULike), but this solution will save you a lot of time when working with references between a local and a remote library. Kudos to the BibDesk and CiteULike team for being so responsive to feature requests from their user community!
On a related note, if you are a Mendeley user, you will be happy to learn that today their integration with CiteULike is finally seeing the light.
June 3rd, 2009 at 7:07 pm
I’ve never used BibDesk (I prefer JabRef for making bibtex documents), but Zotero is certainly capable of exporting to bibtex. I’m very curious as to what makes you prefer CiteULike for the paper-gathering (rather than local storing and formatting) part of the process.
June 3rd, 2009 at 8:04 pm
Kate, the main difference between Zotero on the one hand and CiteULike, Connotea, Bibsonomy and the like on the other hand is that the former is a client-side reference manager while the latter are server-side ones.
I have two old posts on AP focusing on what makes in my opinion online reference managers superior: availability and collaborative functionality.
Since Zotero introduced collaborative features the distinction may have become blurred.
June 3rd, 2009 at 8:15 pm
Great post, Dario, though I gotta say the real-time sync with citeulike that Mendeley just rolled out is a bit easier still.
June 4th, 2009 at 1:25 am
Is there a windows alternative to Bibdesk that is able to do carry out these steps? (Especially browsing the citeulike library and creating the authorname-year-firstword bibtex keys?)
June 4th, 2009 at 7:00 am
Mr. Gunn, I think you are right, but at the moment I am very happy with BibDesk and I don’t need an actual PDF manager, as BibDesk and Quicksilver (and Spotlight when needed) do a great job.
Zephyrus, I am afraid I can’t help with windows!
June 4th, 2009 at 4:21 pm
I don’t use citeUlike much. I know it has a bigger network, but the killer feature for me right now is RDF metadata, and only connotea and bibsonomy support it. Bibsonomy shows a lot of promise, it’s the most RDF-friedly, but community is mainly compSci.
My workflow is still centered on Web of science, though. This is the closest we have right now to full coverage. The other social networks cannot warrant that all papers are there.
There’s a comparison of ref managers here:
http://en.wikipedia.org/wiki/Comparison_of_reference_management_software
June 4th, 2009 at 4:38 pm
Jose, don’t get me wrong: I don’t use CiteULike to *look up* references: I access articles wherever I need (Google Scholar, WoS, ScienceDirect) and use CiteULike to *save* references in my online library.
Your RDF point is interesting, do you mean you can export your Bibsonomy library metadata as RDF? That’s something CiteULike may want to copy from the competitors
June 4th, 2009 at 8:03 pm
There’s also shorter comparison of reference managers at Nature Network.
Zephyrus, Mendeley is cross-platform and works well on Windows. You can sync your Citeulike references to Mendeley and use it to format a bibliography in Open Office or Word.
(I work for Mendeley, so I’m biased.)
June 6th, 2009 at 11:41 pm
Thank you SO much – I have a very similar workflow (which in my case includes Lyx and BibDesk) and I haven’t been able to find a good alternative that allows me to get references from all my devices (including the iPhone) with a simple click into a database. Thanks to your nifty trick here this workflow has just gotten rid of some awkward steps (using the web feature in BibDesk) and stopped me being envious of Mendeley (which I like, but still feel is very much a beta version or Zotero, which I really want to like, but even in the new version has not been able to convince me to give up my true and trusted Lyx and BibDesk combo. BibDesk’s ability to edit BibTeX directly is still the killer feature for me. Thanks again!
June 15th, 2009 at 6:40 pm
[...] It’s such a big improvement over my old EndNote / Microsoft Word workflow. I just came across this great blog post on Academic Productivity about how to connect BibDesk to [...]
July 4th, 2009 at 5:19 am
[...] Academic Productivity » CiteULike + BibDesk: Sync your references and live smarter [...]
July 4th, 2009 at 3:14 pm
[...] Academic Productivity » CiteULike + BibDesk: Sync your references and live smarter [...]
July 13th, 2009 at 12:16 am
I’ve used CiteULike since the early days, and really like the ease of posting with bookmarklets and also the fact that the default is to share all references (but not the associated PDFs). Like you, I prefer to work with my references in desktop software, which in my case is Bookends. (The developer has recently added an interface to CIteULike, which is most helpful.) But my perennial problem is this:
I never remember offhand what portion of my CiteULike database I have imported into Bookends, so I always end up duplicating references or leaving stuff out. If it were possible to sort my CiteULike library by date added, perhaps I could just make a habit of noting down when I last synced up my databases and work from there, but as far as I can work out that isn’t possible. How do you (or your readers) get around this problem?
July 13th, 2009 at 7:59 pm
One simple way to get around this problem is to tag your entries on CiteULike, either as NotImported or Imported or something similar. Just a thought.
July 13th, 2009 at 8:16 pm
You should probably convince the Bookends developer to check records in CiteULike and grey them out if they match those already stored in your local database – manually syncing references sounds really awful. CiteULike records have a unique identifier (Citeulike-Article-Id) that could be used for this purpose. Your CiteULike library is sorted by date added by default.
July 15th, 2009 at 4:48 am
Thanks for your comments, Joachim and Dario. I guess Joachim’s advice is best for the short term and Dario’s for the longer term. I’ve found hitherto that it’s quite difficult to remember consistently to follow sound tagging practices such as the one Joachim suggests.
Dario, I know my CiteULike library is by default sorted by date added, but I don’t know how to select a bunch of entries by that criterion, for example everything that I added since June 1st this year.
July 16th, 2009 at 12:17 am
An update: The Bookends developer has said he’s going to add DOI checking to the Bookends Browser when using CiteULike, so that articles already in the Bookends database can be automatically marked as such. And at CiteULike, although there isn’t a way to search for articles by date posted, the developers pointed out that it’s not that hard to use the checkboxes on recent articles to add a tag such as “recent”, for subsequent use in exports.
All in all, a rather nice outcome. Thanks again.
September 5th, 2009 at 3:02 pm
We’ve since added explicit “sort by date posted”. You can even specify an explicit date range.
http://www.citeulike.org/howto?show=sec2-16
September 18th, 2009 at 11:14 am
Hello. I am relishing all of this information. I need a new workflow and would love to sort out how to sync Bibdesk with CiteULike. I tried what was suggested but perhaps I have got the wrong end of the stick.
Would I put the following in as the URL: http://citeulike.org/bibtex/user/dartar?fieldmap=posted-at:date-added&do_username_prefix=0&key_type=4
BUT instead of dartar I would put my CiteULike user?
Thanks!
Amber
October 15th, 2009 at 12:13 am
[...] post will become a nerd talk). On this one, I used BibDesk as a reference manager. I found a post (par ici) which explains how to synchronize your CiteULike library with BibDesk (it is only a one-way thing [...]
October 16th, 2009 at 6:21 pm
: Yes, you’ve got it correct (e.g. http://citeulike.org/bibtex/user/amber?fieldmap=posted-at:date-added&do_username_prefix=0&key_type=4).
I’ve a question about syncing a GROUP with BibDesk. I have the following URL (I’ve hashed-out values for privacy):
http://www.citeulike.org/bibtex/group/#####/?rsskey=###############&fieldmap=posted-at:date-added&do_username_prefix=0&key_type=4
This URL is a group I’ve created that is private, hence the rsskey entry. When I put the URL in a browser, it downloads a file as I’d expect. However, when I follow the steps listed and try to add this URL to BibDesk, I get an error. The error says “The operation could not be completed.” and goes on to indicate that “No other information is available about the problem.” Additionally, I get a warning “!” icon to the right of the external library name in BibDesk.
Ideas?
As an aside: I’m trying to share resources as I come across them with my dissertation committee. I have moved from Zotero to CiteULike, as I think I’ll have better luck with CiteULike’s focus on collaboration, sorting, blogging/forum creation, etc. CiteULike + Zotero-esque browser plug-in would be ideal, as my workflow right now is (1) Pull citation from journal via Zotero plug-in; (2) Export citations from Zotero to CiteULike; (3) Upload files as needed.
Thanks for the post, dario!
October 22nd, 2009 at 7:21 pm
UPDATE: I’ve solved my group sync problem. The URI follows:
http://www.citeulike.org/bibtex/group/#####/library?fieldmap=posted-at:date-added&do_username_prefix=0&key_type=4&rsskey=#####
Hopefully this can help someone else.
December 25th, 2009 at 1:10 pm
Hi Dario and othes, please contact me: reinaldo.opus
Call for Papers/Abstracts
PracTeX Journal: Issue 2010-1
Issue theme: “LaTeX Academic Work Bench”
http://www.tug.org/pracjourn/
Dear LaTeX and TeX Users,
Since its first edition in 2005, the PracTeX Journal has presented
a wide range of articles on the practical use of LaTeX and TeX.
Among these articles are ones that describe tools and techniques
that can be used in teaching.
The PracTeX Journal 2010-1 issue has the theme “LaTeX Academic Work Bench”.
The goal of this issue is to present ideas on the use of LaTeX tools
for education, teaching, and classroom purposes. We are looking for
articles that can discuss the development of the tools, and their use and effectiveness. Actual examples and LaTeX sources are encouraged.
** Scope
The scope of the issue includes, but is not limited to :
- tools that assist the students/authors in preparing
graphics, indexes, bibliographies, and other parts of documents;
- text manipulation tools;
- tutorials;
- short videos;
- free or almost free tools;
- teaching texts;
- homework styles;
- cross-platform tools.
We encourage you to submit original papers describing your experiences
using LaTeX and TeX tools in an academic setting, and also papers
on tool development work in progress or completed.
** Submission Guidelines:
If you would like to submit an article or technical note for
publication please contact the editors .
We will work with you to prepare the article. Also see http://tug.org/pracjourn/submit.html
for the Journal’s guidelines.
** News
Submissions in any language are acceptable.
Since its beginning in 2005, the PracTeX Journal has accepted
papers not only in English but also Portuguese, Spanish, Dutch,
German, Norwegian, Chinese, Korean, Romanian, and Italian.
However, please contact the editors in English, and submit
an English abstract.
Best Regards,
Francisco Reinaldo
Paul Blaga
2010-1 Issue Editors
Lance Carnes
Main Editor
February 8th, 2010 at 5:02 pm
[...] the External group function in bibdesk. The URL for mine is (from this tutorial) [...]
February 16th, 2010 at 4:54 pm
A little late to this thread, but I’ve been using a script to sync my library and PDFs between Bibdesk and Citeulike for a while now. It pulls the whole Bibtex database down, all the personal PDFs, and adds a Local-Url field to the .bib so that Bibdesk can find the PDFs. If you make any changes in Bibdesk, you can just re-import the whole file to citeulike, making sure to match the articles by cite key. If you keep your PDFs in citeulike, this is great. The script is available here:
http://github.com/hugomallinson/citeusync
February 17th, 2010 at 11:14 am
:
Nice script. You might like to use the JSON API to make the script more efficient. Also you’ll need to put a small delay in the main “page” loop or there’s a risk the user’s IP might get automatically blocked
August 16th, 2010 at 9:10 pm
Hi,
Thanks for putting this up.
I had a quick question: is there a way of getting the DOI from CiteULike without the http://dx.doi.org prefix so as to automatically populate BibDesk’s DOI field?
Thanks!
January 4th, 2011 at 7:36 am
This is great. Not sure I got from the post how you deal with manging pdfs in your workflow. do you upload .pdfs into citeulike first? if so, how do you get them into bibdesk? Hugo, it looks like you have a script for this that syncs. does this mean that if you add a pdf in bibdesk, that this pdf can be synced to citeulike? or does it only work in transferring the pdf one way (from citeulike to bibdesk?)
two way syncing would be great. I’m still looking for a good way to manage downloading pdfs and organizing them. Been organizing them in bibdesk, but that requires downloading, naming, and linking to the bibdesk entry for ea paper.
thanks!
November 18th, 2011 at 10:24 am
[...] BibTeX files for a user or group can be pulled from a URL on the command line with tools [...]