CAA News Today

Strong case studies of digital humanities scholarship exist, and compelling resources continue to proliferate. Scholarly societies, including the American Historical Association (AHA), Modern Language Association (MLA), and more recently the College Art Association (CAA), and Society of Architectural Historians (SAH) have developed guidelines for use by institutions and individuals. Funded by a grant from the Andrew W. Mellon Foundation, CAA and SAH formed a joint task force that explored existing guidelines and literature, surveyed faculty and administrators, and developed Guidelines for the Evaluation of Digital Scholarship in Art and Architectural History. Adopted by the boards of both societies, the guidelines are available for download on the SAH and CAA websites and we encourage their adoption and use. The task force reviewed many resources in the course of eighteen months. Some of these are described below to provide a context for current digital evaluation in addition to the recently published CAA/SAH guidelines.

Sources such as the Journal of Interactive Pedagogy and Pedagogy and Manifold Scholarship, and sites such as Mapping Gothic France, HyperCities, and ArtHistoryTeachingResources.org are places where digital scholarship is being practiced, peer reviewed, and successfully included in promotion and tenure portfolios. Benchmark portals such as the NINES (Networked Infrastructure for Nineteenth-Century Electronic Scholarship) NEH Summer Institutes (Storify of the proceedings can be found here) offer useful white papers: “Digital Humanities Scholarship: Recommendations for Chairs in Language and Literature Departments“; “Guidelines for Promotion and Tenure Committees in Judging Digital Work”; and “Statement on Authorship.” Kristine M. Bartanen’s excellent “Digital Scholarship and the Tenure and Promotion Process“ contains valuable guidance. Published to Academic Commons in July 2014, Bartenan provides a comprehensive literature review and resource portal for the evaluation of digital scholarship. The review contains both model guidelines for promotion and tenure and appendices for a range of resources, literature, and organizations pertinent to understanding and developing evaluation methods.

In the fall of 2012 the Journal of Digital Humanities published their fourth issue, which centered on the “importance of assessment and the scholarly vetting process around digital scholarship.”[1] (The full journal issue can be accessed here.) The issue addresses the need for guidelines and dialogue for assessing digital scholarship in the humanities, offers case studies and examples, and reflects on the successes, issues, and failures of the methods employed. Though much of the content is not art or architectural history-specific, it is still relevant to CAA’s and SAH’s memberships. A broad overview of the contents of this journal issue is provided below so that readers can zero in on the material most germane to their own needs. Readers are encouraged to make use of the excellent bibliographies attached to each of the issue’s essays.

The issue is divided into four major sections: “The Problem Stated,” “Approaches,” “Institutional Guidelines,” and “Resources.” In the introductory section of “Living in a Digital World: Rethinking Peer Review, Collaboration, and Open Access,” Sheila Cavanagh highlights the importance of retraining processes and expectations of evaluation around digital scholarship, including “encouraging faculty who hire, tenure, and mentor junior scholars to acknowledge the complicated factors in the world of digital scholarship . . . for example, faculty often have difficulty identifying appropriate experts to participate in more traditional peer review processes.”[2] In “Evaluating Collaborative Digital Scholarship (or, Where Credit is Due),” Bethany Nowviskie highlights the inability of many humanities evaluative committees to adequately understand and appropriately review and reward the type of collaborative work that digital scholarship is often predicated upon. She contends that “the T&P [tenure and promotion] process is a poor fit to good assessment (or even, really, to acknowledgement) of collaborative work, because it has evolved to focus too much on a particular fiction. That fiction is one of ‘final outputs’ in digital scholarship.”[3]

The “Student Collaborators’ Bill of Rights“ from UCLA’s Digital Humanities program provides important guidance about how to understand, review, and reward collaborative digital humanities scholarship, especially in light of hierarchies of power experienced by student and “alt-ac” collaborators.[4] Nowviskie hails certain aspects of the AHA Guidelines for Evaluation of Digital Scholarship, particularly their underscoring of collaborative digital work as a type of perpetual peer review. At a National Endowment for the Humanities “Off the Tracks” workshop in 2011, Nowviskie, Matt Kirschenbaum, Doug Reside, and Tom Scheinfeldt collaborated on a provocative “Bill of Rights” for collaborators that might serve as a guidepost for promotion and tenure committees to consider. In Nowviskie’s words, “We drew on our experience administering MITH [Maryland Institute for Technology in the Humanities], the Scholars’ Lab, and the Roy Rosenzweig Center for History and New Media—three centers that are sites for a great deal of collaboration among people who may have similar backgrounds as scholars and technologists, but whose formal institutional status may vary a great deal. We drafted something we called a ‘Collaborators’ Bill of Rights,’ which was later endorsed by the full workshop assembly and posted for public comment. Basically, it’s an appeal for fair, honest, legible, portable (this is important!), and prominently displayed crediting mechanisms. It also offers a dense expression of underlying requirements for healthy collaboration and adequate assessment from the point of view of practicing digital humanists, with special attention to the vulnerabilities of early-career scholars and staff or non-tenure-track faculty.” Their recommendations are as follows:

1. Committees must consider not only the products of digital work but the processes by which the work was (and perhaps continues to be) co-created;
2. Scholars (even while they ask to have their critical agency as individuals taken seriously in tenure and promotion cases) are obligated to make the most generous and inclusive statements possible about the contributions of others;
3. Credit should be expressed richly and descriptively, but also in increasingly standardized forms, legible within a variety of disciplines and communities of practice;
4. We must negotiate expressions of shared credit at the outset of projects and continually, as projects evolve;
5. We must promote fair institutional policies and practices in support of shared assertion of credit, such as those which make collective and individual ownership over intellectual property meaningful and actionable;
6. And, finally, we must accept that collaborators themselves, regardless of rank or status, have the ultimate authority and responsibility for expressing their contributions and the nature of their roles.

Other relevant literature includes Todd Presner’s essay “How to Evaluate Digital Scholarship,” directed at the audiences within the art and architectural history-specific community: “The document is aimed, foremost, at Academic Review Committees, Chairs, Deans, and Provosts who want to know how to assess and evaluate digital scholarship in the hiring, tenure, and promotion process. Secondarily, the document is intended to inform the development of university-wide policies for supporting and evaluating such scholarship.”[5] Presner offers a succinct and clear checklist for promotion and tenure boards, engaging them in dialogue around initial review, crediting, intellectual rigor, impact, peer review, and experimentation, among other topics.

Geoffrey Rockwell’s “Short Guide to Evaluation of Digital Work” deepens and extends Presner’s checklist in compelling and practicable ways, and Laura Mandell’s essay “Promotion and Tenure for Digital Scholarship” offers a useful and clear coda full of concrete examples such as the HyperCities project and Voyant, a software text analysis tool. It is not overstating the importance of these three essays alone to suggest that every promotion and tenure board member might be asked to read them to educate themselves on the shifting boundaries of their discipline, and the ways in which their analysis of promotion and tenure portfolios must change and adapt. In addition, in the third section of the journal issue, the MLA’s Guidelines for Evaluating Work in Digital Humanities and Digital Media is similarly designed to “help departments and faculty members implement effective evaluation procedures for hiring, reappointment, tenure, and promotion. They apply to scholars working with digital media as their subject matter and to those who use digital methods or whose work takes digital form.”[6]

James Smithies’s essay “Evaluating Scholarly Digital Outputs: The Six Layers Approach” offers a different style of checklist, a bird’s-eye view of the different types of digital scholarship output, but one that is most useful if the reader—part of a promotion and tenure board or otherwise—is already informed about and open to considering digital scholarship on par with analog modes. Katherine D. Harris’s “Explaining Digital Humanities in Promotion Documents” is a comprehensive and fruitful overview of her foray into learning “how best to sell Digital Humanities, scholarly editing, and Digital Pedagogy to my colleagues.”[7] Harris, associate professor of English literature at San Jose State University, shares her statement for promotion and her approach to the practical question of formatting and outlining her digital scholarship as part of her promotion and tenure binder. With few models to consult, she crowdsourced her questions via Twitter and other channels in order to produce her promotion and tenure document, finally settling on a narrative-based approach that outlines her own collaborative and individual contributions to the field, and includes examples of her concrete research and pedagogy projects in progress and completed, publications, conferences, and supporting quotations from the Digital Humanities Community Wiki and Digital Humanities Now.

While every essay in the fall 2012 Journal of Digital Humanities issue is well worth consultation, the final resource to flag is the MLA’s “Evaluation Wiki of the Committee on Information Technology.” This report takes the form of a checklist of documents that digital scholars should amass as part of preparation for promotion and tenure, and it might be used in tandem with Harris’s model statement.

Both CAA and SAH leadership teams encourage the wide dissemination and adoption of the newly published guidelines, as well as a CAA 2017 panel or forum models that facilitate questions, feedback, and discussion around the issues raised.

Michelle Millar Fisher is a doctoral candidate in Architectural History at the CUNY Graduate Center and a Curatorial Assistant in the Department of Architecture and Design at The Museum of Modern Art.

[1] Daniel J. Cohen and Joan Fragaszy Troyano, “Closing the Evaluation Gap,” Journal of Digital Humanities vol. 1, no. 4 (Fall 2012): i. They continued: “As digital humanities continues to grow and as more scholars and disciplines become invested in its methods and results, institutions and scholars increasingly have been debating how to maintain academic rigor while accepting new genres and the openness that the web promotes.”

[2] Ibid, 9.

[3] Ibid, 17.

[4] Authored by Haley Di Pressi, Stephanie Gorman, Miriam Posner, Raphael Sasayama, and Tori Schmitt, with contributions from Roderic Crooks, Megan Driscoll, Amy Earhart, Spencer Keralis, Tiffany Naiman, and Todd Presner, this document endorses the Collaborators Bill of Rights, developed during the workshop “Off the Tracks—Laying New Lines for Digital Humanities Scholars” held at the Maryland Institute for Technology in the Humanities (MITH), January 20–21, 2011, and described below. The UCLA document adds principles to safeguard students when working on projects with scholars who are senior to them.

[5] Ibid, 36.

[6] Ibid, 91–94.

[7] Ibid, 71.