Middlebury

MiddHistory Internship

This page contains records and documentation relating to my internship in Special Collections working on MiddHistory Online, running from January 2010 to April 2010. -J. Isler (Mon. & Tues. 9a.m.-1p.m., Wed. 1p.m.-5p.m.) x3488 in Special Collections or email me with questions.


Meeting/Worksession notes

2010: January 12

Meeting with: M. Lynch, B. Carson, A. Wentink, D. Rougeau, J. Isler
Discussed plans for internship: 3 phases, permissions and accessibility, and documentation

  • Phase 1 Technical/back-end work: scanning, metadata, uploading, overseeing students working on raw material going into CONTENTdm
  • Phase 2 Populating the front-end of MiddHistory: selecting materials for the modules currently up and running, i.e. Presidents, Buildings, and Founders & Trustees
  • Phase 3 Administrative: contacting other institutions, obtaining agreements and consent to digitize and grant accessibility to images, etc.
  • Phase 4 Marketing plan (if time allows): creating a comprehensive launch and awareness campaign

(3 hrs. work)

2010: January 14

Familiarization with current project documentation and status of project
(3 hrs. work)

2010: January 18

Meeting: with D. Rougeau and J. Isler
Familiarization with CONTENTdm project client view of presidential images; discussion of workflows and necessary metadata fields; where to find images, where to find metadata, status of projects--both in terms of uploading and in terms of approvals; items need to be in CDM before inclusion in MiddHistory Online, etc.

  • Need to get this stuff all in one place, and make sure both students and staff can pick up a project mid-stream and continue the work!
  • Need to consult with R. Jenkins and M. Warner on metadata workflows, and M. Lynch on project workflows (big-picture, as it relates to Digital Archives Team and Collection Management)?

(4 hrs. work)

2010: January 19

Discussed with A. Wentink the need for concrete workflows and project status record-keeping--received approval for moving ahead on this work. Gathering existing resources together for project documentation and creation of workflows. Emailed D. Rougeau and A. Wentink with specific questions and outlined location of existing pieces of information.
(4 hrs. work)

2010: January 20

More discussion of folder setup and structure, etc. Emailed B. Carson for info. on technical pieces and workflows. Outlined location of existing project materials in email. Getting approval from other parts of LIS on the cross-departmental pieces (metadata). (4 hrs. work)

2010: January 25

Preparing metadata fields document using CDP and local best practices. Discussion with D. Rougeau about workflows (specifically, stages at which metadata is captured.) Discussed necessity of capturing certain data at time of image scanning. Will review metadata document tomorrow and bring to A. Wentink for input. (4 hrs.)

2010: January 26

More work on metadata fields draft document. Questions about mapped fields have been noted and will confer with D. Rougeau, A. Wentink, and then catalogers (R. Jenkins, M. Warner, T. Simpkins). Discussed document with T. Simpkins. (4 hrs.)

2010: January 27

Finalizing data dictionary for MiddHistory project. Meeting with B. Carson. Based on meeting with T. Simpkins yesterday, will recommend that Special Collections in close conjunction with the Digital Archives Team, create a basic MINIMUM standards for ALL future digital projects. This will only be a guideline, and additional enhancement may be appropriate based on the project needs. The metadata will need to be revisited, and carefully thought-through, and customized (and a unique data dictionary created) prior to the start of every new digital project at Middlebury. This will ensure sustainability and interoperability for these collections. (4 hrs.)

2010: January 28

1-hour check-in meeting with A. Wentink on progress of work, next steps, and filling out the internship learning agreement form for submission to Syracuse.

2010: February 1

Creating project workflows wiki article akin to the document created by H. Raum Procedures for digital projects. (3 hrs.)

2010: February 8

Working on adding photos of Presidents to CONTENTdm and getting the workflow in place. Emailed B. Carson for clarification on some admin settings for the College Archives collections on Cdm, and requesting info. on accessing full-res tiffs from secure server storage. (4 hrs.)

2010: February 9

  • Uploading existing scanned images of Middlebury Presidents into Cdm Project Client. Testing the metadata dictionary and fine-tuning this part of the workflow. D. Rougeau retrieved the Presidential and Trustee originals from the stacks for consultation in adding metadata; and oversaw the setup of the new project in the Cdm Project Client.
  • Added documentation for GoogleDocs to a Google Doc and sent to A. Wentink and D. Rougeau.
  • Had an error when opening the Cdm project client after closing it and restarting the computer. The solution is to close the project client, connect to libscan1 folder (entering credentials), and re-open the project client:
Cdm Error Message
  • Lesson learned: Log into libscan1 before opening shared projects in the project client. (4 hrs.)

2010: February 10

Adding metadata for Presidents and Trustees! Testing workflow and data dictionary. (4 hrs.)

2010: February 15

  • Metadata for Presidents & Trustees. A. Wentink resolved question about Seth Storrs' dates (1950 general catalog contained an error).
  • Training for reference desk shift in Special Collections with A. Wentink. (4 hrs.)

2010: February 16

  • More metadata work for Presidents & Trustees. Good progress today--only about 18 of the 40 images left to catalog.
  • ACS meeting tour of Special Collections. (4 hrs.) = 50 hrs. completed

2010: February 17

  • Evening reference shift in Special Collections (referred one student request to A. Wentink for follow-up)
  • Installed Cdm on monitor workstation; upgraded some stuff on this machine to make it more usable
  • As a result of metadata work, was thinking about ways to make our Cdm Digital Collections at Middlebury pages more interactive. Can we implement a commenting feature as T. Reese describes here in his blog post? Has this already been tried? Are there other options out there we can try? What other social media features would be useful on our digital collections pages--tagging, "share this" buttons? If they are not already built-into CONTENTdm, is there any way these types of functionality could be easily added? (4 hrs.)

2010: February 22

  • Continued metadata addition for Presidents & Trustees. Amusing facial hair on these guys. Consulted with D. Rougeau on some loose ends, and uploaded the completed images for approval. (4 hrs.)

2010: February 23

  • It appears the approval and indexing of the items I created was not successful. None of the items are in the digital collections pages. Searches by Title, Subject, and across and within the relevant collections yield nothing that was uploaded and indexed yesterday. D. Rougeau consulted with R. Jenkins for advice, but it is a larger issue. She has sent an email to M. Lynch and B. Carson (who is away today).
  • Unable to log into the administrative interface for MiddHistory Online (page load error, firefox detecting a redirect loop) sent an email to B. Carson who is out but will look into it tomorrow. (4 hrs.)


2010: March 1

  • Responded to M. Lynch and B. Carson about separate MiddHistory login issue. B. Carson is looking into the problem--relating to CAS.
  • Correspondence on Friday Feb. 26 regarding the CONTENTdm collection items not being searchable. R. Jenkins explained that the items are indeed available via browsing the collections (but not searchable). Wrote to B. Carson and M. Lynch about how to fix. It appears the solution is a manual rebuild of entire collections that must be run from the Linux command line. Waiting to see if this solves the problem. Does the new version of Cdm fix this bug? Need to establish at least a temporary workflow for the timely rebuilding of collections to ensure searchability.
  • Update: College Archives collection was rebuilt in the morning; MHO login issue was resolved later in the afternoon. Thanks to B. Carson and M. Lynch. (3.5 hrs.)

2010: March 2

  • Work agenda for today: Working with D. Rougeau and A. Wentink on adding GoogleDocs links into Special Collections and College Archives pages. Going through Mediawiki overview and basic training with D. Rougeau and A. Wentink. (3 hrs.)

2010: March 3

  • Meeting with R. Jenkins to discuss MHO subject headings; discussed strategies for cleaning up materials in the College Archives digital collection in Cdm.
    • For future discussion: Should we rename "Subjects" to "Topics" in our digital collections searching--is that easier and more intuitive for users and catalogers?
  • Meeting with A. Wentink to discuss how job changes will affect workflows within Special Collections. (4 hrs.)

2010: March 8

  • Cleaning up some of the existing materials in Cdm (subject headings in College Archives collection)
    • Making edits to items previously uploaded and approved and indexed: If you use Project Client to search for existing items within a collection, select them, and add them to the project, any changes you make will require re-uploading, approving and indexing to be effective.
  • Deleting existing (duplicate items)
  • Re-populating the MHO portal with the correct images. (4 hrs.)

2010: March 9

  • Updating workflow docs.
  • Cleaning up College Archives controlled vocabularies. (4 hrs.)

2010: March 10

  • Cleaning up College Archives controlled vocabularies
  • Helping A. Wentink and D. Rougeau prepare College Archives and Special Collections events notifications on the blog and the website by adding an RSS sidebar node with feeds from tags on the LIS blog: Special Collections, College Archives, exhibits, lectures and Abernethy Lecture. (4 hrs.)

2010: March 16

  • Observe B. Ellis' FYSE instruction session 9:30-10:30.
  • In College Archives Cdm collection, move college building names from the Subjects, Geographical to the Subjects, Name (Persons) field. Clean up Subjects, Geographical field. (4 hrs.)
  • Evening work cleaning up Cdm fields (2 hrs.)

2010: March 17

  • Cleaning up Cdm fields; locked fields
  • Populating presidents & trustees portal (4 hrs.)
  • Evening work cleaning up fields (2 hrs.)
  • 96.5 hrs. total completed thus far

2010: March 24

  • (3.5 hrs.) = 100
  • Unlocking arbitrarily locked items
  • Cleaning up Cdm headings
  • When adding items back into project client: make sure items selected to be added to the client are not locked (make sure you don't select all and add if some are locked and some are free). This will lock all items selected. You'll need to go into the Cdm Administration, Items, and Lock Administration to unlock the items. Indexing and the manual reindex will not fix this locked issue.

2010: March 29

  • (4 hrs.) = 104
  • Tying up loose ends for making metadata changes in the project client
  • Final fixes for College buildings controlled vocabulary. Successfully moved from Subjects, Geographical to Subjects, Name
  • MiddHistory portal: Items (placeholders) added to timeline
  • Wrote to B. Carson about undefined error for several items when adding to timeline
  • Meeting with A. Wentink on next steps for Phase 3 of internship
  • Rename the categories on MiddHistory: to Presidents, Founders & Trustees
  • Systematically searching the archives and special collections of UVM Special Collections, then Vermont Historical Society, then Vermont State Archives for primary source materials (and some pamphlets), on Presidents, Founders and Trustees, and Alumni created at Middlebury or with subject matter relating to Middlebury. Also asking these institutions for other potentially relevant collections

Presidents (from selected list)
Founders & Trustees (from list)
Alumni: including Slade, William, 1786-1859, Weeks, John M. (John Moseley), 1788-1858, and Tichenor, Isaac, 1754-1838

2010: March 30

  • (4 hrs.) = 108
  • Started searching UVM special collections for materials relating to the persons selected for MiddHistory Online. Keeping track via this link: Searching other institutions I'm looking in the UVM catalog, the UVM libraries special collections database, and the UVM special collections finding aids. I haven't found anything of note at the moment, but there's much more to search, and I'll have to contact the curators to see if they have anything they know of that isn't listed in any of the online resources (there's always more hidden away in any archive and special collection!)

2010: March 31

  • (4 hrs.) = 112
  • Finished searching the UVM catalog, special collections database, and the finding aids online. Sent A. Wentink and D. Rougeau an email and invitation to edit the GoogleDocs spreadsheet keeping track of the searches I've done and the overall status of the MHO project activities.
  • Requested that A. Wentink review the items I located at UVM and make suggestions for other Presidents, Trustees and Founders names we can ask about. The hope is that they may have other uncataloged materials we can view and possibly obtain permission to use as part of our MiddHistory Online endeavor.
  • See spreadsheet

2010: April 5

  • (3 hrs.) = 115
  • Searching ArcCat just in case...
  • Started searching Vermont Historical Society online collections
  • See spreadsheet

2010: April 6

  • (4 hrs.) = 119
  • Continued searching Vermont Historical Society online collections
  • Finished searching VHS online collections See spreadsheet.

2010: April 7

  • (4 hrs.) = 123
  • Searching Vermont State Archives online catalogs and resources See spreadsheet
  • Finished searching VSA

2010: April 12

  • (4 hrs.) = 127
  • Email/meeting with A. Wentink to discuss results of searching and next steps
  • Tweaking arrangement of MHO persons
  • Begin drafts of requests for assistance from Vermont archival institutions

2010: April 13

  • (4 hrs.) = 131
  • Added a few more names to the searching list and searched library online holdings
  • Read up on the Jedediah Burchard scandal (Chapter 5 of Stameshkin's The town's College: Middlebury College, 1800-1915)

2010: April 19

  • (4 hrs.) = 135
  • Letters to institutions

2010: April 20

  • (4 hrs.) = 139
  • Letters to institutions

2010: April 26

  • (4.5 hrs.) = 143.5
  • Check our holdings to make sure we do not have any materials we are requesting from UVM
  • Areas to check in the archives for materials
  • Send UVM letter by the end of this week
  • Discussed future plans for MHO and digital collections work

2010: April 27

  • (6.5 hrs.) = 150
  • Reviewing local holdings
  • Finalizing letter to UVM/VHS/VSA

Tasks and responsibilities

Phase 1: Technical phase This includes scanning, metadata creation, and uploading of content into CONTENTdm for the back-end of the project—this may include managing student employees performing related tasks (Deliverable=Collected Wiki documentation)

  • CONTENTdm & metadata
    • Create workflow for CDM work
    • Document status of input and any clean-up work
    • Define clear input fields for technicians and specialists, using accepted standards and providing clear access to these guidelines
  • Content status
    • Consolidate location of project materials
    • Document status of input and any clean-up work

Phase 2: Selection and "population" phase

  • Populate MHO portal and ensure portal functionality (Deliverable=Populated web interface)

Phase 3: Administrative phase

  • Contacting external institutions with relevant materials to obtain consent and cooperation for digitization of or shared use of already-digitized materials, for inclusion in MHO. (Deliverable=Institutional agreements/digital objects)

Links and resources

MHO Documents

Metadata for MHO Data dictionary as a GoogleDoc; draft in progress
College Buildings Controlled Vocab exists as a GoogleDoc; draft in progress
MiddHistory Online link to portal
MiddHistory documentation created by Brendan
Placeholder for MHO This is where workflow for CDM piece of MHO should live.

MHO Workflow

Some materials for MHO require digitization, while some are already digitized.

Selection criteria
Basic criteria for deciding which images to scan include:

  • Image must be sharp and have good tonal separation
  • Image must be clearly identifiable to provide necessary metadata
  • Image must be interesting and well composed
  • Image must have historical significance or broad appeal and enduring value

Selecting text files for scanning
Basic criteria for selecting text material to scan:

  • Material should have enduring value
  • Material should have historical significance and a broad base of interest and appeal
  • Material should be in the public domain (published before 1924)
  • Material should be unique or not widely held
  • Material should fall within the parameters of the College Archives Collection Development Policy
  • Material should not already be available from other online open access sources

Saving Scanned Files

Whether you are creating image files or text files they need to be saved to the \\Middarchive\libscan1\spec_coll\archives (and appropriate subfolder), which has been designated for digital assets we have created. To save files to Middarchive you need to map a network drive to this location:

  • Right click My Network Places
  • Click Map Network Drive
  • Do not type anything on the "Drive" line.
  • Type \\middarchive\libscan1 in the "Folder:" box
  • Click Finish and a Connect to middarchive window will open
  • Enter the appropriate credentials
  • Click OK or press the Enter key

Creating Image Files

Digital versions of photographs, maps, stereo views, etc. can be created using the scanners in Special Collections. Photographs and portraits are normally scanned at 600 dpi and larger size maps are normally scanned at 300 dpi for the .TIF master files. Image files that are to be included in the Digital Collections at Middlebury College gateway need to be imported into CONTENTdm software from the Middarchive file server and then metadata about the images will be added. The CONTENTdm software will automatically create derivative .JPG files and .GIF files (small "thumbnails") as part of the importing process. Digital projects that have any controlled vocabulary fields (e.g. subject headings that conform to LC subject headings or other standard vocabularies) need to be reviewed by one of the catalogers before the metadata records and images can be approved and made available online. D. Rougeau should be notified when a reasonable number of items have been uploaded for approval*. Maps that are too large to scan on our oversize scanner can be scanned on the 42" wide Vidar scanner in room 318 in BiHall.
(*)May substitute this with “uploaded material will be approved on a regular basis—once every two weeks” or something to this effect.

Scanning Procedures

Items are selected for scanning by the project manager.

  1. Check the glass flat of the scanner to be sure it is clean
  2. Remove the clear protective sleeve or cover and place item on face down on scanner.
  3. Log into the workstation
  4. Click Start>Programs
  5. Epson Scan icon on the left
  6. Click Epson Scan on the bottom menu bar
  7. Click Preview
  8. Left click and drag the mouse to crop the image as needed
  9. Click Scan
  10. Click Other in the Location box
  11. Click Browse
  12. Click Rougeau on SCAN-1048 (or other name as appropriate)
  13. Click libscan1 on Samba 3.0.33-3.7
  14. Click spec_coll>archives> the appropriate subfolder
  15. Click OK
  16. File Name Prefix
  17. Follow the naming convention for this section of the archives
  18. Image Format
  19. Type: click TIFF (*.tif)
  20. Be sure there is not a checkmark in the box
  21. Overwrite any files with the same name
  22. Repeat steps as necessary

Please note that the Epson settings should be:

+Mode: Professional mode
+Original
++Document type: Reflective
++Document source: Document table
++Auto Exposure Type: Photo
+Destination
++Image Type: 24 bit color
++Resolution: 600

Joining images, such as map sections

  1. Open Adobe Photoshop software
  2. Open one of the images to be joined
  3. Click Image on the top menu bar
  4. Click Canvas Size
  5. Move the white center square to one side
  6. Open second image to be joined
  7. Change opacity in the layers window to about 50%
  8. Select an area where the sections overlap and magnify
  9. Function keys may be used: Ctrl A=select; Ctrl C=copy; Ctrl V=paste
  10. Move the second image until it meshes with the first section precisely
  11. Change the opacity back to 100%
  12. Save and then collapse layers

Batch adding scanned items

  1. Open Project Client (Click icon on desktop)
  2. Navigate and select the appropriate project to which the items will be added
  3. In the Common Tasks at left, click Add Multiple Items
  4. Click Browse to navigate to the location of the items on libscan1
  5. Highlight your choice and click Next
  6. Leave checked the Yes box (Do you want CONTENTdm to generate display images from items you import?)
  7. Click Image options at bottom left, and click Image Rights to add band stating "Courtesy of the College Archives at Middlebury College" at the bottom of each item. Change the font size to 10pt.
  8. Click Next; Verify the number of items to be added, and click Add Items

Controlled vocabularies

All fields designated as using a controlled vocabulary should be compatible with the appropriate thesaurus or resource, as designated in the project Metadata Dictionary. New terms will not appear in the list of controlled vocabulary terms until they have been approved after the metadata records been uploaded. To add a batch of new subject headings:

  1. Open CONTENTdm Administration
  2. Click Collections>Field properties
  3. Click Edit in the controlled vocabulary field you wish to edit
  4. Click Administer controlled vocabulary
  5. Type new subject headings, one per line, in the Add term box
  6. Click Add when you are finished adding new terms

To edit or delete subject headings which have already been approved: (Subject headings must be deleted from individual metadata records before being deleted from the thesaurus!)

  1. Open CONTENTdm Administration
  2. Click Collections>Field properties
  3. Click Edit in the controlled vocabulary field you wish to edit
  4. Click Administer controlled vocabulary
  5. Type existing subject headings, one per line, in the Delete term box
  6. Click Delete when you have finished typing terms

A list of all approved subject headings used in the controlled vocabulary thesaurus may be printed at follows:

  1. Open CONTENTdm Administration
  2. Click Collections>Field properties
  3. Click Edit in the controlled vocabulary field you wish to edit
  4. Click Administer controlled vocabulary
  5. Click Browse the defined vocabulary for this field
  6. Click File in the top menu bar>Print

To clear out a messy batch of controlled vocabulary terms (Say, for instance, a preset group of terms was added to the CV and it is found that they have errors or inconsistencies and you've already added other, correct terms to the collection, and want to delete the incorrect terms but retain the correct terms):

  1. In Cdm administration for the collection in question, Click Collections, then Fields, then click Edit in CV field
  2. Click Administer shared CV
  3. Click Browse the defined vocab for this field, opens in new tab, select all and copy (ctrl+c)
  4. Paste this list into a Notepad (.txt) file just in case something goes amiss
  5. Paste (ctrl+v) into Delete terms box, and click delete (NOTE: There is a limit of 1024 terms that you may delete or add at any one time; deleting or adding larger numbers of terms will require multiple copy and pastes)
  6. Then, click Verify the existing field contents agains the defined... (list of existing terms used in records will appear)
  7. Select all, copy, and paste these terms into the Add term box (verify that they are correct) and click Add
  8. Re-index collection; Delete notepad file.

Adding Metadata

Image files often do not have a real title, so a short, descriptive and distinctive title should be created that would be useful in retrieving the item in a keyword search mode.
Carefully follow the instructions for metadata creation described in the project Metadata Dictionary.
For this project, the original, physical items will be retrieved from the archives storage and should be consulted when Technicians or Catalogers are adding metadata.
Make sure to omit punctuation (like periods) at the end of controlled vocabulary fields.

Approving Metadata

Metadata records may be approved by D. Rougeau, A. Wentink, or a designated Cataloger. Metadata records may be approved by logging into the web-based administration module: http://middarchive.middlebury.edu/cgi-bin/admin/start.exe

  1. At the admin home page click Items
  2. Scroll down to highlight the current collection you want and click Change
  3. On the next screen click Approve
  4. There are usually two sections on the Approve Items page:
  5. The first section is for controlled vocabulary and items in this section must be approved first.
    1. If the new subject headings look fine, just click Add on each line
    2. If the new subject headings need to be changed, just click Edit and click Edit on the next screen to view the full metadata record.
    3. Make any changes that are necessary and click Approve
  6. The second step of the approval process is to review metadata records for individual pending items
    1. You may either click Approve All without viewing the full metadata record, or click Edit to view the full metadata record.
    2. When viewing the full metadata record you may click View this item if you would like to view the scanned image, or you may click Controlled Vocabulary on the subject line to view the thesaurus that has been created for that project
    3. After viewing the full record you may make changes as necessary and then click Approve or approve the record without making changes.
  7. Click Index in the top menu bar and then click Start Index

Changing Metadata post-approval

Changes may be made to individual items in the Cdm administration by clicking Items and Edit.
Changes may also be made to existing items in the Project Client. This process describes how to make changes to groups of items in the Project Client (useful for bulk-addition of terms, or fixing the same error in lots of records).

  1. In the Project Client, open the collection in question, and
  2. Click the find in collection tab
  3. Enter search term in the white search box at top left of the tab, click search
  4. Select records to edit (click or select all if necessary)
  5. In the left-hand Item tasks area, click Add to Project (edit). Items now appear in the project area tab (to the right of Find in collection tab)
  6. Make changes and edits (fill-down or make individual changes)
  7. Select all records and Upload for approval (Follow approval and indexing instructions as above)

Related projects at Middlebury

Hans' Existing Procedures Doc This could be useful for creating an overarching view of the process
Digital Collections & Metadata Collection of metadata docs for other digital projects
Need to create something like this: Metadata for DLA

Standards Documents

CDP DC Metadata Best Practices
BCR's CDP Digital Imaging Best Practices

Deliverables

  • (This) Wiki page of internship project work
  • Wiki/Documents for digital project workflows
  • Status of project work
  • Meetings and documentation related to project work
Powered by MediaWiki