Edwina L. Rissland, David B. Skalak, and M. Timur Friedman
The BankXX system models the process of perusing and gathering information for argument as a heuristic best-first search for relevant cases, theories, and other domain-specific information. As BankXX searches its heterogeneous and highly interconnected network of domain knowledge, information is incrementally analyzed and amalgamated into a dozen desirable ingredients for argument (called argument pieces), such as citations to cases, applications of legal theories, and references to prototypical factual scenarios. At the conclusion of the search, BankXX outputs the set of argument pieces filled with harvested material relevant to the input problem situation. This research explores the appropriateness of the search paradigm as a framework for harvesting and mining information needed to make legal arguments. In this paper, we discuss how we tackled the problem of evaluation of BankXX from both the case-based reasoning (CBR) and task-performance perspectives. In particular, we discuss how various system parameters--start node, evaluation function, resource limit--affected BankXX from the CBR perspective and how well BankXX performs its assigned task of gathering information useful for legal argumentation by running BankXX on real legal cases and comparing its output with the published court opinions for those cases.