Efficient Generation of
Justifications
for Collective Decisions
Layout: typeset by the author using LATEX.
Efficient Generation of Justifications
for Collective Decisions
Nienke Reints 11899239
Bachelor thesis Credits: 18 EC
Bachelor Kunstmatige Intelligentie
University of Amsterdam Faculty of Science Science Park 904 1098 XH Amsterdam Supervisor A. Boixel
Institute for Logic, Language and Computation Faculty of Science
University of Amsterdam Science Park 907 1098 XG Amsterdam
Abstract
When a group decision needs to be made there are various approaches to choose from. In some cases it might be desirable to have the possibility of providing the agents with a justification for why a given decision would be a reasonable compromise in a specific situation. These justifications rely on normative principles, called axioms, the agents consider desirable. Unfortu-nately, such justifications are computationally hard to generate. Currently, a naive approach taken by Boixel and Endriss is capable of finding justifica-tions: however, the approach is not yet efficient. In this thesis, we restrict our attention to axioms that refer to at most two profiles. To speed up the search for justifications, two new algorithms are proposed: the first one is designed to find justifications solely based on axioms that refer to a single profile, the second algorithm is designed for axioms both referring to one and two profiles. Both algorithms appear to be more efficient than the naive ap-proach. This is mainly because the new algorithms only use the information provided by the axioms when it is deemed to be useful.
Contents
1 Introduction 4 2 Voting theory 7 2.1 Model . . . 7 2.2 Justifications . . . 10 3 Algorithm 13 3.1 Naive approach . . . 13 3.2 Design . . . 14 3.2.1 First algorithm . . . 15 3.2.2 Second algorithm . . . 18 4 Results 20 4.1 Implementation . . . 20 4.2 First algorithm . . . 20 4.2.1 Justifications . . . 20 4.2.2 Scalability . . . 21 4.2.3 Worst-case scenarios . . . 22 4.3 Second algorithm . . . 24 4.3.1 Justifications . . . 24 4.3.2 Order of axioms . . . 24 4.3.3 Worst-case scenarios . . . 265 Conclusion and Discussion 27
1
Introduction
When a group decision needs to be made, there are various approaches to choose from. Classically in voting theory, a voting rule is chosen to aggregate the indi-vidual preferences of the agents into a collective one. Different voting rules satisfy different normative principles (axioms) constraining the outcomes that can be se-lected in different situations. The Pareto principle, for example, states that if in a given situation all the agents prefer an alternative A over an alternative B, then the latter cannot possibly be part of the outcome. A voting rule is charac-terized by certain axioms if and only if it is the only rule satisfying those axioms. For example, if there are only two alternatives, then the majority rule is the only rule satisfying these three normative principles: equal treatment of voters and of alternatives, and positive responsiveness (whenever some voter raises a, possibly tied, winner in her ballot, then it should become the unique winner) [1]. The axioms characterizing a rule can then be seen as justifying the outcomes it gives.
Such justifications could be helpful when an important decision needs to made or to convince people why the outcome is a good compromise. Despite the use-fulness, very few studies have addressed how to generate justifications of election outcomes independently from the aggregation method that has been or will be used to aggregate the individual preferences. Cailloux and Endriss [2], for exam-ple, have used tools from Artificial Intelligence to permit people to reason about the different voting rules. They also provided a model which generates a justi-fication for an outcome in case it corresponds to the one selected by the Borda rule; the model automatically generates justifications based on the axioms charac-terizing the Borda rule [3]. This approach of generating justifications by Cailloux and Endriss is interesting; however, because the approach is bounded to the Borda count it is limited.
Instead of trying to justify the outcomes given by a specific voting rule, Boixel and Endriss [4] sought to generate a justification for why a target outcome would represent a reasonable compromise in a concrete situation in terms of various nor-mative principles. The notion of justification they develop has both a nornor-mative and an explanatory component. The explanation shows how the selection of the target outcome follows from concrete instances of the normative principles consid-ered. These justifications should be more likely to be accepted by agents than a general justification because they are tailor-made to the situation.
Boixel and Endriss encode the restrictions that are imposed by axioms in propo-sitional logic. These formulas are then given as input to a satisfiability (SAT) solver; this SAT solver searches for a rule satisfying these restrictions. The result can then be used to prove, for example, that there exists no rule satisfying these restrictions. This corresponds to the proof of the base case of an impossibility theorem; Geist and Peters [5] reviewed on proving the base-cases of such theorems using this type of automated reasoning tool. Moreover, the tool is also helpful to quickly test interesting ideas and the resulting proofs are easily translatable to be understandable for a human.
However, searching for the justifications for why a target outcome represents a reasonable compromise is computationally hard. Using the model created by Boixel and Endriss [4] in a small scenario with three agents and three alternatives, it takes up to 30 minutes to find the justifications or to prove that no justification can be found for a given claim. In the model of Boixel and Endriss, the complex-ity is mainly due to the generation of all of the axiom instances. An instance is a concrete restriction imposed by an axiom on the possible outcomes that can be selected in a concrete situation. By restricting to axioms that refer to at most two profiles, generating all instances is only necessary in the worst-case scenario. Therefore, this project will be restricted to axioms that only refer to single- and two-profile axioms: the axioms Pareto principle and anonymity are examples of such single-profile and two-profile axioms respectively. Under such a restriction, the goal is to understand what can be improved in the algorithm of Boixel and Endriss and to design algorithms that are more efficient. Furthermore, these al-gorithms will be implemented and finally evaluated in terms of their practical efficiency through a small experimental study.
The goal is to compare the efficiency of the new algorithms with that of Boixel and Endriss. They ran an experiment with three voters and three alternatives: their goal was to check whether and how any outcome can be justified given any possible target profile. This experiment cannot fully be reproduced for various reasons; mainly because Boixel and Endriss use an axiom that can refer to more than two profiles. The experiment in this thesis focuses on the time-efficiency of the algorithms by running different scenarios. Moreover, various parameters, inherent to the algorithms, are changed to see how it affects their efficiency.
The research question in this thesis is: To what extent can the generation of justifications for collective decisions be made more efficient in practice when solely considering axioms that refer to at most two profiles? It is expected that there is a way to find justifications more efficiently when restricting to axioms that refer to at most two profiles. However, when no justification exists this might not be
the case as all restrictions of all axioms need to be considered.
In the next section, important notions from social choice theory will be recalled and the notion of a justification formally defined. The naive approach as well as the new algorithms will be presented in Section 3. Thereafter, the efficiency of the model is reported in Section 4. Finally, Section 5 concludes with a discussion and some suggestions for future work.
2
Voting theory
This section will first recall some relevant concepts used in voting theory [6]. There-after, several normative principles from the used corpus will be elucidated. Finally, the notion of a justification will be clarified.
2.1
Model
Let N∗ be a finite set of n voters (|N∗| = n) and let X be a finite set of m alternatives (|X| = m). Let L(X) represent the set of strict linear orders on X. A linear order i from L(X) is used to represent the preferences of an agent i
over the alternatives in X. For an electorate N ⊆ N∗, the collection of individual preferences of agents in N (a profile of preferences) can then be defined as N,
with N ∈ L(X)|N |. More generally, let L(X)+ be the set SN ∈2N ∗\{∅}L(X)N of
all possible profiles for any nonempty electorate featuring agents in N∗.
A voting rule can be used to aggregate individual preferences into a collective decision. All voting rules can be defined as a function F : L(X)+ → 2X/{∅},
mapping a profile to a specific outcome, a (nonempty) winning set of alternatives. This outcome is an element of the powerset of X denoted by 2X/{∅} indicating
the set of all possible subsets of alternatives excluding the empty set. A simple example of such a voting rule is the Borda count: each voter reports a linear order, the alternative with the highest Borda score win. The score of an alternative x is calculated as follows: an alternative receives m − 1 points whenever it is ranked first by an agent, m − 2 points whenever it is ranked second, etc. Finally, 0 points in case the alternative is ranked last, these points are thereafter summed over all voters.
Score(x) = (m − 1) · |{i | i ranks x first}| + (m − 2) · |{i | i ranks x second}| + . . .
+ (m − m) · |{i | i ranks x last}|
In classical voting theory there are some normative principles that a good voting rule should satisfy; note that, what is considered desirable highly depends on the context in which a collective decision is taken. These normative principles ensure fairness and proportionality among others. For example in a presidential election, it is required that the voting rule should return a single winner (i.e., the voting rule should be resolute). Moreover, in various scenarios it is desirable that no
voter has more influence on the outcome than any other (anonymity) and there should be no alternative that every voter prefers over the selected winner (Pareto principle).
Despite the fact that a good voting rule should satisfy many of these normative principles, no voting rule can satisfy all of them because some axioms contradict each other [6]. One theorem representative of this phenomenon is the Gibbard-Satterthwaite theorem about resolute voting rules for at least three alternatives; any such rule that is surjective and strategyproof (no voter can manipulate the outcome) will be a dictatorship; this dictatorship entails that only a single agent can influence the outcome of the voting rule. Thus, surjectivity, strategyproofness and non-dictatorship are incompatible [7, 8]. Moreover, even a more basic and desirable axiom (anonymity) cannot be satisfied in this case.
Voting rules are mechanisms defined independently from any concrete situation and in these general cases many appealing axioms are incompatible. However, when considering a concrete situation, it is possible that two axioms that are, in general, incompatible, become compatible. For example, the reinforcement axiom and the Condorcet principle are incompatible [9], but these axioms become compatible when, for example, considering voting rules that are defined for three or less voters. Thus, the selection of a specific outcome in a specific situation might comply with two axioms that are otherwise incompatible. Therefore, the generated justifications in this thesis are based on axioms rather than on voting rules.
In this thesis six of these axioms will be discussed to clarify the model.
• Pareto Principle: If all voters prefer x over y (x, y ∈ X) in a given profile N, then y cannot be the winner in this specific situation.
∀ N∈ L(X)N, {i | x i y} = N → y /∈ F ( N)
• Condorcet Principle; If there exists an alternative x∗ that is strictly
preferred to any y 6= x∗ by a majority of agents, then x∗ should be the unique winner. ∀ N∈ L(X)N, ∀x∗ ∈ X ∀y ∈ X \ {x∗}, {i | x∗ i y}| > |N | 2 → F ( N) = {x∗}
• Faithfulness: If there is only one voter i in N, her top alternative in N,
topi( N), should win.
• Cancellation; If a perfect tie occurs in all pairwise comparisons, then all alternatives should win.
∀ N∈ L(X)N, ∀x, y ∈ X, |{i | x i y}| = |{i | y i x}| → F ( N) = X
• Anonymity: If profile N0 can be obtained by switching voters names in
N, via a permutation α, then F ( N) and F ( N0) must be equal.
∀ N∈ L(X)N, ∀ N0∈ L(X)N 0
, α( N) = N0 → F ( N) = F ( N0)
• Neutrality: If profile N0 can be obtained by switching all occurrences of
the alternatives in N, via a permutation π, then F ( N0) can be obtained
from F ( N) by applying the same permutation π.
∀ N∈ L(X)N, ∀ N0∈ L(X)N 0
, π( N) = N0 → π(F ( N)) = F ( N0)
Based on these axioms the goal is to justify why a certain outcome is a good compromise. When considering profile N in Example 1 various axioms can be
used to base a justification on. For example, using the Pareto Principle it can be invoked that c should not be part of the outcome because every voter ranks a above c. Moreover, when solely considering pairwise comparisons, a is preferred to b by the majority of voters (namely voters 1 and 2), likewise, the majority also prefers a to c. The Condordet Principle states in this case that {a} should be the winner because it is the Condorcet winner. To capture the restrictions imposed by the axioms in a concrete situation, the full power of the axioms is not used. Instead, specific instances of those axioms, referring to concrete objects (profiles, alternatives, etc.) are used.
Example 1. The following profile is profile N.
Voter Preference 1 a 1 b 1 c
2 a 2 c 2 b
3 b 3 a 3 c
Finally, to link the voting rules and the axioms, an interpretation of an axiom is defined. The interpretation of an axiom A, I(A) ⊆ L(X)+ → 2X \ {∅}, is the set
of all voting rules, defined for a set of alternatives X and an electorate N ⊆ N∗, satisfying that axiom. For a set of axioms A, I(A) denotes the set of voting rules satisfying all the axioms in A: I(A) =T
2.2
Justifications
Given a target profile N∗, the goal is to find a justification for why a target
out-come X∗ ⊂ X is a reasonable compromise based on the instances of the considered axioms: these axioms are in the corpus A. The axioms that are considered can, for instance, be selected because these axioms are desired by the agents that want to justify their decision. In this thesis only axioms that refer to at most two profiles are considered.
Given one of those axioms A ∈ A, the set of all its instances referring to profile N is denoted with Inst N(A). Moreover, let Inst N(A) be the set of all instances
of a set of axioms (Inst N(A) =
S
A∈AInst N(A)) that refer to a profile N. More
generally, let Inst(A) be the set of all instances of a set of axioms that refer to all possible profiles.
The set of all axioms, A, can be split in two subsets: axioms that refer to one profile, A1, and axioms that refer to two profiles, A2. These subsets form a
partition of A: A1∪ A2 = A and A1∩ A2 = ∅.
The first subset of axioms, A1, is the set of axioms that state a restriction on
the outcome a voting rule can select for a single profile. This set includes among others the Condorcet principle, which selects the Condorcet winner if one exists, and the Pareto principle, which can prevent an alternative from being a possible winner.
The second subset of axioms, A2, is the set of axioms that state a restriction
on the outcome a voting rule can select for two different profiles. An example of such an axiom is anonymity, which states that the outcome of N should be the
same as the outcome of N0 if the set of individual preferences remains the same
but are ordered differently.
Given a target profile N∗, a target outcome X∗ and a corpus A, the properties
and requirements of a justification can be defined. The following definition of a justification is taken from Boixel and Endriss [4].
Definition 1. (Justification) A justification is a pair of sets of axioms hAE, ANi,
with AE being an explanation containing concrete instances of the axioms in the normative basis AN. This justification should satisfy the following requirements;
1. Explanatoriness. All voting rules F ∈ I(AE) satisfying the axiom instances
in AE select the target outcome for the target profile: F (
N∗) = X∗. This
is not the case for any proper subset of AE.
basis AN: AE ⊆ Inst(AN).
3. Adequacy. For all axioms A ∈ AN, the axiom is also in the provided corpus of axioms: AN ⊆ A.
4. Nontriviality. There exists at least one voting rule that satisfies all axioms in the normative basis: I(AN) 6= ∅.
Using the definition from above, an example of a justification can be written out more formally. Consider the target profile in Example 2, the goal here is to justify why {a, b} is a good compromise.
Example 2. The following profile is the target profile N∗.
Voter Preference 1 a 1 b 1 c
2 b 2 a 2 c
In this example there are seven possible outcomes: {{a}, {b}, {c}, {a, b},
{a, c}, {b, c}, {a, b, c}}. According to an instance of the Pareto principle, an axiom referring to one profile (A1), the alternative c cannot possibly be a winner
because both voters prefer a over c. The set possible outcomes for target profile is therefore reduced to {{a}, {b}, {a, b}}.
Moreover, according to an instance of the anonymity axiom the outcome of this profile should be the same as the outcome selected for the profile in which the names of the voters are switched. According to an instance of the neutrality axiom the outcome of the target profile is linked to the outcome of a profile in which alternatives a and b are switched. There is one profile N0 that can be
obtained by either switching the voters or switching the alternatives a and b:
Profile N0
Voter Preference 1 b a c 2 a b c
The only outcome in {{a}, {b}, {a, b}} satisfying the instances of anonymity and neutrality is {a, b}. The outcome of N∗ cannot, for example, be equal
to {a}, because according to neutrality F ( N0) should be equal to {b}, but
same argument is used to prove that {b} is not a possible outcome for N∗.
The instances used to justify the selection of outcome {a, b} under profile N∗
are in the set AE while the normative basis AN contains the axioms Pareto
3
Algorithm
To improve the efficiency of the algorithm written by Boixel and Endriss this thesis focuses solely on axioms that refer to at most two profiles. Under such restriction the complexity of the problem at hand is reduced, because generating all instances of all axioms is only necessary in a worst-case scenario. First this section shortly discusses the algorithm implemented by Boixel and Endriss, thereafter it discusses the algorithms that have been developed during this thesis and implemented in order to generate justifications in a more efficient way.
3.1
Naive approach
As mentioned in Section 1, the algorithm of Boixel and Endriss is capable of finding justifications for why a certain outcome is a good compromise given a corpus of axioms and a specific situation. The approach uses a SAT solver and an MUS-extractor. The former is a tool which decides whether a logical formula (in CNF) is satisfiable or not. If such a formula is unsatisfiable, an MUS-extractor can be used to find a minimal unsatisfiable subset; this subset is a minimal (with respect to set inclusion) set of clauses that together are already unsatisfiable [10] while any proper subset of this set will be satisfiable. Intuitively, this set corresponds to a concrete explanation.1
To find whether or not there exists a justification for a given claim, the algorithm of Boixel and Endriss searches for a proof by contradiction: it considers a goal con-straint cG : “F ( N∗) 6= X∗” which states that the rule cannot select the target
outcome and see whether a contradiction can be reached. Moreover, the algorithm finds the justifications in a naive way: it goes through all possible subsets of ax-ioms from the corpus (i.e., normative basis). Based on such a subset it generates the CNFs for all axiom instances (given N∗ and X). Next, the goal constraint is also translated to a CNF to be able to search for a proof by contradiction. The CNFs of the goal constraint and the instances are the input for the SAT solver. If the SAT solver returns that the CNF is unsatisfiable, then it means there exists no voting rule satisfying all the axioms in the normative basis considered, while at the same time selecting an outcome different from the target one for the target profile. In this case, a minimal unsatisfiable subset (of instances) containing the goal constraint is found using MUSer2 [10]; this tool returns a subset which corresponds to an explanation. Lastly, the nontriviality requirement is checked for
the corresponding normative basis to make sure that there exists at least one rule satisfying all the axioms. Combined with the explanatoriness requirement, we now know that any such rule will select the target outcome for the target profile.
However, this naive approach is computationally hard because it requires the generation of all axiom instances beforehand. The generation is time consuming because if there are m alternatives and n voters there are (m!)nprofiles. The total number of instances that need to be generated highly depends on the number of profiles: the cost of generating all instances increases rapidly when the number of voters and/or alternatives increases. Usually, when adding more alternatives the number of instances rises faster than when adding more voters (at least for the axioms that are considered in this thesis). Finally, the generation of all instances is often unnecessary because only in the worst case scenario, when no justification exists, all instances need to be generated; when there exists a justification, it is expected that the explanation will only use a limited number of instances.
The algorithms developed in this thesis start by generating instances that are close to the profile (i.e., instances in Inst N ∗(A)). More precisely, in the algorithms
the instances are not generated beforehand, but rather when it is necessary to expand the search.
3.2
Design
Two separate algorithms are designed to simplify the problem at hand. The first algorithm finds a justification based on axioms that solely consider one profile (A = A1). The second algorithm finds a justification based on axioms that refer
to at most two profiles (A = A1∪ A2). The justifications should satisfy the four
requirements stated in Definition 1. When taking a closer look, the adequacy and relevance requirements are the least remarkable: the adequacy requirement will be trivially satisfied because the algorithm will only find justifications based on axioms from the corpus that is considered, thus AN ⊆ A. The relevance requirement is also easily satisfied because one can easily keep track of which axiom gave rise to which instances, thus AE ⊆ Inst(AN). Furthermore, it is assumed that any
normative basis AN ⊆ A is nontrivial, because this is indeed the case in small scenarios and when considering the six axioms stated in Section 2.1. Thus, the algorithms search for a justification, but only the explanatoriness requirement should be given careful consideration.
3.2.1 First algorithm
As introduced in Section 2, a total of six axioms are used in this thesis. Of these, four axioms refer to a single profile: Pareto principle, faithfulness, Con-dorcet principle and cancellation. These axioms can be divided in three groups based on their type. The first type of axioms, fix-outcome, can enforce a set of alternatives to be the outcome in given a profile. An example of such an axiom is the Condorcet principle which forces the selection of the Condorcet winner if one exists. Axioms of the second type, exclude-alt, are axioms that prevent an alternative from being in the outcome. The Pareto principle is such an axiom; it forbids an alternative to be part of the outcome if it is Pareto dominated. Finally, the third type, force-alt, is also about specific alternatives and whether or not they can appear in the outcome of a specific profile: these axioms force an alternative to be in the outcome. In this thesis no axiom of this type is used; however, adding this type ensures that the algorithm can cope with all possible restrictions a single profile axiom may enforce.
The first type can also been seen as a combination of the other two types, how-ever, for clarity the distinction is made. The general approach of the first algorithm to search for a justification, without using axioms that fix the outcome, is by con-tradiction: starting with a set of all possible outcomes that can be assigned to the target profile, excluding the target outcome and the empty set (because no valid voting rule results in no outcome). This set of possible outcomes is reduced by applying the instances of axioms of type exclude-alt (or force-alt if there’s any). When there is no outcome left in the set possibleOutcomes for the target profile, the only possible outcome is the target outcome and thus a justification is found (line 20). In Algorithm 1 the three previously mentioned types of axioms are distinguished using the three if statements in lines 4, 8 and 14.
Additionally, in this first algorithm two other functions, outcome and alt, are used: both functions take an instance as input. The former, outcome, can be used to find what the outcome should be, given an instance of an axiom of type fix-outcome. For example, if c is a Condorcet winner in the target pro-file, the function outcome will return {c} as output given the unique instance of the Condorcet principle that refers to the target profile. An axiom of the type fix-outcome always gives rise to at most one instance per profile, because only one outcome can be selected. Likewise, the second function, alt, is used to find the alternatives that are forced to be in or excluded from the outcome by an instance of type force-alt or exclude-alt respectively. For instance, in the case a is a Pareto dominated alternative in the target profile the function alt returns
alternative a as output given the corresponding instance of the Pareto princi-ple. Any axiom of type exclude-alt gives rise to at most m − 1 instances; and an axiom of type force-alt gives rise to at most m instances.
Algorithm 1: Generating a justification grounded in a normative basis containing axioms that refer to a single profile at the time.
Input : Target profile N∗, Target outcome X∗, Corpus of single profile
axioms A = A1
Output: If it exists, a justification hAE, ANi for why F (
N∗) should be
equal to X∗
1 possibleOutcomes = 2X \ {∅, X∗} 2 AE = ∅, AN = ∅
3 for axiom A ∈ A1 do
/* If the axiom forces the final outcome */
4 if A.type() == fix-outcome then 5 I = Inst(A, N∗)
6 if outcome(I) == X∗ then
7 Return hAE = {I}, AN = {A}i
/* If the axiom prevents an alternative from being in the
outcome */
8 if A.type() == exclude-alt then 9 for instance I ∈ Inst(A, N∗) do
10 toRemove = {O | O ∈ possibleOutcomes and alt(I) ∈ O} 11 if toRemove 6= ∅ then
12 possibleOutcomes = possibleOutcomes \ {toRemove} 13 AE = AE∪ {I}, AN = AN ∪ {A}
/* If the axiom forces an alternative to be in the outcome */
14 if A.type() == force-alt then 15 for instance I ∈ Inst(A, N∗) do
16 toRemove = {O | O ∈ possibleOutcomes and alt(I) /∈ O} 17 if toRemove 6= ∅ then
18 possibleOutcomes = possibleOutcomes \ {toRemove} 19 AE = AE∪ {I}, AN = AN ∪ {A}
/* If there is no remaining possible outcome */
20 if possibleOutcomes == ∅ then 21 Return hAE, ANi
3.2.2 Second algorithm
In the second algorithm both single- and two-profile axioms are considered (A = A1∪ A2). When taking into account the two-profile axioms, the search for a
jus-tification becomes more complicated because any explanation will refer to several different profiles of preferences.
The problem is represented as a graph in which each node contains information about objects that have been used so far; the currently visited profiles, the used instances (AE) and the corresponding axioms (AN). The nodes are connected by
instances that talk about the visited profiles. In contrast to the naive approach, these instances are generated at the moment they are required. The search is done using a breadth-first approach, thus the instances are required when more nodes in the graph need to be explored. The root of the graph contains the target profile, the goal constraint (F ( N∗) 6= X∗), and an atLeastOne requirement for the target
profile; the latter represents the requirement that a voting rule should select at least one alternative for each profile. The function, atLeastOne, is designed to generate the CNFs that represent this requirement imposed on a given profile. This function receives a profile as input and returns a CNF encoding the fact that the outcome of the profile should contain at least one of the alternatives. From the root node, the graph is then expanded by generating the instances that talk about the target profile and using those instances to go farther away from the target profile at each step. When all nodes have been explored and all tentative explanations in all nodes were satisfiable there exists no justification for why F ( N∗) = X∗; the algorithm
returns in line 21.
Moreover, different from the first algorithm, Algorithm 2 relies on a SAT solver because the restrictions of the two profile axioms are more complicated than the single profile axioms. The SAT solver is used to check whether the potential AE
obtained at a given step satisfies the explanatoriness requirement; if so the conjunc-tion of formulas representing instances in AE should be unsatisfiable; meaning that
a justification is found (line 10). Finding an unsatisfiable set of instances results in a justification that satisfies the explanatoriness requirement: at the previous step the set of instances was satisfiable (we were able to find at least one rule satisfying all instances while selecting an outcome different from the target one). However, this is not the case anymore after adding the last instance. Despite the fact that in Algorithm 2 the number of calls to the SAT solver is larger than in the naive approach, the CNFs given to the solver are relatively small in the breadth-first ap-proach. Therefore, this does not influence the efficiency of the second algorithm. Moreover, the use of the SAT solver in the breadth-first approach will be more effi-cient than in the naive approach of Boixel and Endriss, because the naive approach
still generates useless instances in scenarios where a justification can be found. Fi-nally, in the worst-case scenario Algorithm 2 still needs to generate all instances of all axioms, and look at all possible combinations of instances to determine that no justification can be found.
Algorithm 2: Generating a justification grounded in a normative basis containing axioms that refer to at most two profiles at the time.
Input : Target profile N∗, Target outcome X∗, Corpus of single- and
two-profile axioms A = A1∪ A2
Output: If it exists, a justification hAE, ANi for why F (
N∗) should be equal to X∗ 1 AE = {atLeastOne( N∗)} ∪ {F ( N∗) 6= X∗} 2 AN = ∅ 3 usedProfiles = { N∗} 4 root = Node(usedProfiles, AE, AN) 5 root.setDiscovered
/* Ensures each node will only be explored once */
6 Q = [root] /* Q is a queue (First in / First out) */ 7 while Q 6= ∅ do
8 currentNode = Q.dequeue() 9 if currentNode.AE is unsat then
10 Return hcurrentNode.AE, currentNode.ANi 11 for axiom A ∈ A do
12 for profile ∈ currentNode.usedProfiles do
13 for instance I ∈ Inst(A, ) such that I /∈ currentNode.AE do 14 usedProfiles = currentNode.usedProfiles ∪ profiles(I) 15 AE = currentNode.AE ∪ {I} ∪ {atLeastOne( ) | ∈
profiles(I)}
16 AN = currentNode.AN ∪ {A}
17 nextNode = Node(usedProfiles, AE, AN) 18 if nextNode not discovered then
19 nextNode.setDiscovered
20 Q.enqueue(nextNode)
4
Results
4.1
Implementation
Both Algorithms 1 and 2 have been designed and implemented to generate jus-tifications more efficiently. The algorithms have been implemented and run with Python version 3.8.1 on a machine equipped with 16GB RAM and a Dual-Core Intel Core i5 processor running at 2.9 GHz. The SAT solver that has been used in the second algorithm is Lingeling2 provided by the Python package pylgl3; this version and comparable ones are widely used in state of the art research.
The code can be found on http://doi.org/10.5281/zenodo.3909465.
4.2
First algorithm
4.2.1 Justifications
To check the correctness of the first algorithm various cases have been tested and checked by hand. This is manageable, because the first algorithm only deals with restrictions imposed on a single profile, therefore, the justifications are not too complicated. In this section two examples are elucidated.
In the case of Example 3 the goal is to justify why F ( N∗) = {a}: the first
algorithm finds two distinct explanations in approximately 0.00048 seconds. The first explanation uses two instances of the Pareto principle. First, it states that b should not be in the outcome because it is Pareto dominated by a; second, it states that c should not be in the outcome F ( N∗) for the same reason. This
ex-planation is found by deleting outcomes that contain a or b from the set of possible outcomes. The second explanation solely uses one instance of the Condorcet principle; because a is the Condorcet winner, F ( N∗) should be equal to {a}.
Example 3. The following profile is the target profile N∗.
Voter Preference 1 a 1 b 1 c
2 a 2 c 2 b
2http://fmv.jku.at/lingeling/ 3https://pypi.org/project/pylgl/
To demonstrate the correctness in larger scenarios, Example 4 shows a profile with five voters and five alternatives. In this case Algorithm 1 finds exactly one justification in approximately 0.0006945 seconds: this justification uses one in-stance of the Condorcet principle which states that {a} should be the winner because it is the Condorcet winner in this profile.
Example 4. The following profile is the target profile N∗.
Voter Preference 1 a 1 b 1 d 1 c 1 e 1 f 2 b 2 a 2 d 2 c 2 f 2 e 3 c 3 a 3 b 3 d 3 e 3 f 4 b 4 a 4 d 4 c 4 e 4 f 5 c 5 a 5 d 5 e 5 f 5 b 4.2.2 Scalability
The algorithm can be run with even more agents and alternatives; however, the structure of the justifications will remain the same because the restrictions single-profile axioms can enforce are not very complex and our corpus of axioms is quite limited. Two different tests have been run to examine the efficiency of the algo-rithm. In the first test the algorithm is run with all possible profiles in combination with all possible outcomes for a fixed number of agents n and a fixed number of alternatives m. Table 1 reports on the times it took to run all scenarios with a fixed m and n on average (this includes worst-case scenarios and all other cases).
n = 1 n = 2 n = 3 n = 4
m = 1 0.0005836869 0.001438458 0.001158284 0.0011220311 m = 2 0.009315332 0.017472230 0.037651317 0.055413494 m = 3 0.044497340 0.382443445 1.391916422 7.275758213 m = 4 0.562774211 7.955245132 245.7194585 2728.749187
Table 1: Time (seconds) to run all profiles in combination with all alternatives
It can be seen that running the algorithm over all profiles in combination with all possible outcomes does not take a long time in small scenarios. However, when the number of alternatives and voters is increased it takes up to 45 minutes (= 2728 seconds) to search for all justifications. This is due the the great amount of profiles
(4!4 = 331776) and possible outcomes (24 − 1 = 15). To get times that can be
compared easily, the numbers in Table 1 are each divided by the number of profiles (m!n) and by the number of possible outcomes (2m− 1) and are presented in Table
2. Thus, the values in Table 2 are times it took to run the algorithm with one target profile and one target outcome on average for a specific m and n.
n = 1 n = 2 n = 3 n = 4
m = 1 0.0005836869 0.001438458 0.001158284 0.0011220311 m = 2 0.000517518 0.0004853397 0.000522935 0.000384816 m = 3 0.000151351 0.0002168047 0.000131511 0.000114572 m = 4 0.000104217 6.1383064e-05 7.89993115e-05 3.65541033e-05 Table 2: Time (seconds) to run one profile and one alternative on average
As can be seen all times in Table 2 are very low, even in cases with more than three alternatives/ voters. This means that Algorithm 1 is efficient in average cases.
4.2.3 Worst-case scenarios
In a worst-case scenario when no justification exists the algorithm is able to run even in larger scenarios. In Figure 1 a 3D-plot can be seen of different combinations of worst-case scenarios; the number of voters varies from 1 till 49 and the number of alternatives from 2 till 23, finally, the vertical axis shows the time it took on average to run Algorithm 1 for a specific m and n.
Figure 1: Plot worst-case scenarios Algorithm 1
In this graph it can be seen that increasing the number of voters and alternatives does not influence the time it takes to search for a justification up until m ≈ 20. However, when m ≥ 20, it is visible that the amount of time that elapses in a worst-case scenario increases: but these time are still very reasonable given the size of the scenarios considered.
4.3
Second algorithm
Because the second algorithm can deal with single- and two-profile axioms, the justifications can be more elaborate and thus it can find more justifications as well as justifications based on different kind of arguments; making them more likely to be accepted by the voters. However, the search becomes more complicated due to the two-profile axioms.
4.3.1 Justifications
To first demonstrate that the justifications returned by the algorithm satisfy the explanatoriness requirement, we are going to search for a justification for why all alternatives should win in case the target profile corresponds to a Condorcet para-dox; the target profile is stated in Example 5. In this profile all alternatives occur in every position exactly once: therefore, a logical outcome is to let all alternatives be in the outcome (F ( N∗) = {a, b, c}). When using this target profile and
tar-get outcome, the algorithm runs in approximately 25.59 seconds, searches through approximately 2000 nodes and generates 66000 instances: of these there are both approximately 20000 instances of neutrality and anonymity. The justifica-tion that is returned solely uses instances of anonymity and neutrality. Example 5. The following profile is the target profile N∗.
Voter Preference 1 a 1 b 1 c
2 b 2 c 2 a
3 c 3 a 3 b
4.3.2 Order of axioms
The efficiency of the second algorithm may depend on the ordering of the axioms in the corpus, to examine if this is the case, the algorithm is run with the same target profile and outcome, but with a different ordering of the axioms. In the first run the corpus is sorted such that the single-profile axioms are first ([cancellation, faithfulness, Condorcet principle, Pareto principle, neutrality, anonymity]), thereafter, the corpus is sorted with the two-profile axioms first ([neutrality, anonymity, cancellation, faithfulness, Condorcet
principle, Pareto principle]). The target profile that is used can be found in Example 6, (this example has already been mentioned in Section 2.2) the goal is to justify F ( N∗) = {a, b}. The number of nodes, instances and time it took in
the different cases can be found in Figure 2.
Example 6. The following profile is the target profile N∗.
Voter Preference 1 a 1 b 1 c
2 b 2 a 2 c
(a) Time (b) Nodes
(c) Instances
Figure 2: Influence of different axiom orderings on the efficiency of Algorithm 2.
When comparing the numbers in Figure 2, it can be seen that if the single-profile axioms are ordered first, the algorithm is approximately three times faster, searches
through less nodes and generates three times less instances. Thus, ordering the corpus with the single-profile axioms first seems to be a good heuristic to follow in practice: this could be the case as the single-profile axioms can greatly reduce the set of possible outcomes, thereafter, the two-profile axioms can be used to quickly rule out the remaining possible outcomes.
4.3.3 Worst-case scenarios
To examine what happens if no justification exists (i.e., the worst-case scenario), different situations are tested of which one is stated in Example 7. In this scenario the goal is to justify F ( N∗) = {b}: as it can be seen, letting b be the unique
winner does not seem to be a reasonable compromise because it is ranked last by all voters. In this case Algorithm 2 does not—as expected—find any justification. It returns in approximately 1.13398 seconds after having explored 801 nodes and generated 6858 instances.
Example 7. The following profile is the target profile N∗.
Voter Preference 1 a 1 b
2 a 2 b
When running a worst-case scenario with m ≥ 2 or n ≥ 2 it takes too long to run in a reasonable time. This is expected because in the worst-case scenario all possible sets of instances need to be explored. This number of instances grows exponentially as the search progresses. When comparing this result to the naive approach Algorithm 2 is not an improvement in worst-case scenarios.
5
Conclusion and Discussion
In order to be able to generate justifications more efficiently than the naive ap-proach, two different algorithms have been designed, implemented and evaluated on their efficiency in this thesis. In the case a justification can be found both algorithms are faster than the algorithm of Boixel and Endriss: this is mainly be-cause the new algorithms will less often generate instances that are not necessary, opposed to the naive approach which generates all instances in the beginning; re-sulting in unused instances.
To be more specific, the first algorithm is very fast no matter whether a justifi-cation exists or not; given a scenario with 40 voters and 20 alternatives it still runs within a reasonable time (approximately 11 seconds). The algorithm of Boixel and Endriss could not deal with such a large scenario at all.
The second algorithm is also more time-efficient than the algorithm of Boixel and Endriss whenever there exists a (small-sized) justification. However, when the justification is either too complex or does not exist, the second algorithm per-forms worse than the naive approach. This is due to the breadth-first approach used in the second algorithm: if the scenario becomes larger then the amount of nodes and instances increases rapidly and searching through all the nodes becomes time-consuming. Even though this is an undesirable result, the algorithm is still an improvement on the naive approach because the amount of instances generated in the case a (simple) justification exists is smaller and the algorithm is thus less time-consuming in those cases.
In the second algorithm the main problem is the trade-off between time and space efficiency. In most cases this algorithm is efficient with respect to time. However, when looking at the space complexity it is less efficient than the naive approach due to the breadth-first search: all nodes that are in the queue should be saved as well as the nodes that are already explored. The amount of information in these nodes builds up quickly as the search progresses because the information of the previous nodes is copied and also stored in the new nodes. To address this issue, one could try to implement the algorithm in a more clever way: by, for example, trying to avoid generating the same instance twice.
The first algorithm on the other hand is more time-efficient and space-efficient: however, the possible justifications that can be generated using instances of single-profile axioms are not very interesting, and it is very unlikely to find a justification that solely uses single-profile axioms in a very heterogeneous profile of preferences. Because the second algorithm still is not perfectly efficient there are many oppor-tunities to improve this algorithm. First, one could think about different heuristics
to use to explore the graph more efficiently; these heuristics can be gained via dif-ferent ways. For example, one could look at which axioms are preferred by the voters. Moreover, one could examine which axioms most often appear together in a justification. It is also showed in Section 4 that when it comes to efficiency, the order in which the axioms are considered does matter: this could be further studied and optimized.
Second, currently a total of six axioms have been implemented in the second algorithm, because of this the justifications that can be found are all quite similar. It would be interesting to implement more axioms, thus broadening the range of justifications that can be found. It is also possible that, given more axioms, the algorithm will find a justification faster, because there are more possible instances. Finally, at this moment the maximum number of nodes increases rapidly when the number of alternatives and/or voters increases. The more nodes are explored the more complex the resulting justifications will be: this may not be desirable because the goal is that the justification can be understood by the voters. To solve this, a future research could try to estimate the maximum number of nodes that should be explored in order for the justifications to be still understandable. This would also help in a worst-case scenario as one would not need to explore all nodes because the search can be stopped early in case it is known that there will not be anything useful above a certain threshold.
References
[1] Kenneth O. May. A set of independent necessary and sufficient conditions for simple majority decision. Econometrica: Journal of the Econometric Society, 20(4):680–684, 1952.
[2] Olivier Cailloux and Ulle Endriss. Arguing about voting rules. In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-2016). IFAAMAS, 2016.
[3] H. Peyton Young. An axiomatization of Borda’s rule. Journal of Economic Theory, 9(1):43–52, 1974.
[4] Arthur Boixel and Ulle Endriss. Automated justification of collective decisions via constraint solving. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-2020). IFAAMAS, 2020.
[5] Christian Geist and Dominik Peters. Computer-aided methods for social choice theory. In Ulle Endriss, editor, Trends in Computational Social Choice. AI Access, 2017.
[6] Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D. Procaccia, editors. Handbook of Computational Social Choice. Cambridge University Press, 2016.
[7] Allan Gibbard. Manipulation of voting schemes: A general result. Economet-rica: Journal of the Econometric Society, 41(4):587–601, 1973.
[8] Mark Allen Satterthwaite. Strategy-proofness and Arrow’s conditions: Exis-tence and correspondence theorems for voting procedures and social welfare functions. Journal of Economic Theory, 10(2):187–217, 1975.
[9] H. Peyton Young. Social choice scoring functions. SIAM Journal on Applied Mathematics, 28(4):824–838, 1975.
[10] Anton Belov and Joao Marques-Silva. MUSer2: An efficient MUS extractor. Journal on Satisfiability, Boolean Modeling and Computation, 8(3-4):123–128, 2012.
