Recent Posts

The Rockefeller Archive Center just wrapped up a little less than five month long project to separate our archival and bibliographic data, both in our collections management system and our discovery interface. You might have read a little bit about this in Katie Martin’s blog post describing the process of removing our library data from ArchivesSpace, but that only covers a portion of the entire project. The processing group working on the export scripts in mid-May, finished in mid-June, and we gathered requirements, built, tested, and launched the system from mid-June to the first week of October. Now that our new library discovery system has gone live, this post will provide more context to the project, outline the process, discuss the technical profile of the system, and finally share some lessons we learned along the way.

Following on the initial implementation of our data pipeline back in March, we’ve been looking closely at ways to increase the performance of the pipeline, by which we mean how many objects the pipeline is able to process in a given time interval. This has been a fairly lengthy process with a lot of learning along the way, so I thought it would be good to document some of what we’ve learned here, both for our future reference as well as for the benefit of someone else trying to do this kind of thing in the future.

Over the last few months, Patrick Galligan of the digital strategies team along with Amy Berish, Darren Young, and myself of the processing team have worked on a collaborative project to export JSON representations of bibliographic data and delete all corresponding resource, subject, agent, and top container records in ArchivesSpace. This was just the first step of a larger project to create a new interim discovery system for library data separate from our archival discovery system, DIMES.

As of this past July 1st, the Digital Programs team (we’re often colloquially referred to as the D-Team) is now officially known as Digital Strategies. This seemingly insignificant change of one word in our name reflects tectonic shifts, both to our team’s role as well as a larger organizational approach to technology and technical expertise.

This month, we completed our project to create an ArchivesSpace Helpers Package. While the end product looked slightly differently than initially scoped–we’re about to embark on a project to add some of the code we’ve created to the ArchivesSnake library–we created a library of functions and tests that use ArchivesSnake with common use cases. This is the second of two blog posts, where project team members reflect on the project. Here are the code owners, who were paired with code helpers on a rotating basis, and were responsible for writing the code for each sprint, as well as a reflection from me, the scrum master and product owner.