5.3 Deliberative Strategies Analysis
5.3.2 Results
In this section, we present the most prevalent deliberative strategies in successful and unsuccessful discussions. Since the total number of utterances in RfC discussions is higher than the total number of utterances in Predecessors (See 3), we cannot compare frequent patterns based on the frequency of a specific sequence. We specify a ranking number for each sequence so that the most frequent sequence is ranked first. Thus, we compare the most prevalent sequences in pairs of discussions based on their ranking number. First, we extract the most frequent sequences from RfC and Predecessor pairs in two different tables. Then, we join the tables into a single table on the ”sequence” field, which is common in both tables. Finally, we query two groups of patterns from the joined table: 1) Frequent successful patterns, and 2) Frequent unsuccessful patterns. For successful patterns, we extract those patterns with high ranking (the most frequent) in RfC discussions and low ranking in Predecessor discussions.
For unsuccessful patterns, we flip the conditions, patterns with a low ranking in RfC discussions and a high ranking in Predecessor discussions. We repeat the process above for each dimension (act, relation, frame) separately. In the following subsections, we analyze the results of the pattern mining process in each dimension.
Discourse act patterns
Applying the PrefixSpan algorithm on the discourse act dimension of RfC-Predecessor pairs, we extract 1604 sequences in RfC discussions and 333 frequent patterns in Predecessor ones. We do outer join to keep all sequences in a single table to compare and analyze successful and unsuccessful patterns.
Successful and unsuccessful patterns Table 17 shows the first 20 successful and unsuccessful patterns. The first glance at successful patterns in the table reveals the most frequent labels are final-ization, recommendation, and understanding. We found that in 95% of identified successful patterns, there is at least one utterance labeled as finalization. Additionally, in 70% of patterns, there exists at least one utterance with understanding label, and 60% of the patterns, have at least one recommenda-tionutterance. On the contrary, 10% of the patterns contain social-act and in 20% of them there is at least one utterance with evidence label. Interestingly, there is no questioning class in the successful patterns.
On the other hand, according to the second part of the table, where it shows unsuccessful patterns, it is clear that the most frequent classes in failed discussions are understanding and questioning, with a frequency of 85% and 75%. Figure 11 compares the results of analysis on both successful and unsuccessful patterns. While in successful patterns finalization and recommendation classes are the most prevalent labels, questioning class stands out in unsuccessful discussions. In successful patterns, a sequence is considerably more likely to be finished with a finalization class (80%), while in unsuccessful patterns, the majority of patterns (55%) are finished with a questioning class. As a
result of our analysis, it can be said that a conversation is more likely to fail if there is a questioning class present but no recommendation or finalization classes. It is important to have to follow up utterances with recommendation label if there exists a questioning utterance.
Discourse act classes 0%
25%
50%
75%
100%
evidence
finalization questioning
recommendatio
social_act
understanding Successful (RfC) Unsuccessful (Predecessor)
Figure 11: Distribution of the discourse act dimension classes in the first 20 successful and unsuc-cessful patterns in RfC-Predecessor pairs
Argumentative relation patterns
We conduct the same process and analysis for the argumentative relation dimension. We applied the PrefixSpan algorithm and extract 363 patterns from RfC discussions and 253 patterns from the Predecessor discussions.
Successful and unsuccessful patterns Table 18 shows the 20 most frequent patterns in successful and unsuccessful discussions concerning argumentative relation attribute. The most frequent class in successful patterns is the support label. As illustrated in the figure 12, in 90% of successful patterns there are at least two utterances with a support label. In contrast, the existence of attack label in the same discussions shows very low frequency, so there is no pattern in a successful discussion that contains two or more utterances with attack label. In other words, every utterance with attack label is followed by another utterance which is labeled as support (50%) or neutral (50%). The situation is exactly the other way around in unsuccessful patterns. The second part of table 18 reveals that most frequent patterns in unsuccessful discussions contain attacking utterances. In 80% of unsuccessful patterns, there are at least two utterances with attack label. On the contrary, supportive utterances rarely happened in the unsuccessful patterns, so that some patterns in the table contain only one support label. Figure 12 confirms that the theme of utterances in successful patterns is supportive, whereas, in unsuccessful discussions, people mostly write the utterances with attacking tone.
Table 17: Top 20 successful/unsuccessful patterns in the Discourse act dimension. rfc/pred rank:
The ranking of the sequence in RfC/Predecessor discussions. rfc/pred freq: The frequency of the sequence in RfC/Predecessor discussions. diff rank: rfc rank - pred rank
Successful sequences rfc rank rfc freq pred rank pred freq diff rank
’recommendation’, ’finalization’, ’finalization’ 76 113 null null null
’understanding’, ’understanding’, ’finalization’, ’finalization’ 84 110 253 47 -169
’recommendation’, ’recommendation’, ’finalization’ 82 110 247 48 -165
’recommendation’, ’understanding’, ’understanding’, ’finalization’ 81 110 193 54 -112
’finalization’, ’recommendation’, ’finalization’ 59 122 164 58 -105
’understanding’, ’evidence’, ’understanding’, ’finalization’ 91 108 196 53 -105
’recommendation’, ’evidence’ 96 106 198 53 -102
’understanding’, ’understanding’, ’recommendation’, ’finalization’ 68 118 166 58 -98
’recommendation’, ’finalization’, ’understanding’ 50 128 135 64 -85
’understanding’, ’evidence’, ’finalization’ 48 132 118 68 -70
’understanding’, ’recommendation’, ’understanding’, ’finalization’ 57 124 127 66 -70
’understanding’, ’finalization’, ’finalization’ 27 157 84 77 -57
’evidence’, ’understanding’, ’understanding’, ’finalization’ 95 107 151 60 -56
’understanding’, ’finalization’, ’understanding’, ’finalization’ 49 131 98 73 -49
’understanding’, ’recommendation’, ’finalization’ 26 160 75 81 -49
’understanding’, ’finalization’, ’understanding’, ’recommendation’ 72 116 117 68 -45
’recommendation’, ’understanding’, ’finalization’ 28 155 72 82 -44
’social-act’, ’finalization’ 56 125 93 74 -37
’understanding’, ’social-act’, ’finalization’ 86 110 120 68 -34
’understanding’, ’finalization’, ’recommendation’ 36 145 66 83 -30
Unsuccessful sequences rfc rank rfc freq pred rank pred freq diff rank
’understanding’, ’questioning’, ’questioning’ 265 80 89 75 176
’evidence’, ’understanding’, ’questioning’ 222 84 58 86 164
’understanding’, ’finalization’, ’questioning’ 213 85 80 78 133
’finalization’, ’understanding’, ’questioning’ 169 90 53 89 116
’recommendation’, ’understanding’, ’questioning’ 168 90 82 78 86
’understanding’, ’recommendation’, ’questioning’ 172 90 86 76 86
’questioning’, ’understanding’, ’questioning’ 151 94 67 83 84
’evidence’, ’questioning’, ’understanding’ 143 96 68 82 75
’understanding’, ’understanding’, ’questioning’, ’understanding’ 112 103 46 95 66
’finalization’, ’questioning’, ’understanding’ 116 101 51 90 65
’finalization’, ’understanding’, ’social-act’ 154 93 91 74 63
’social-act’, ’understanding’, ’understanding’ 147 95 92 74 55
’evidence’, ’understanding’, ’understanding’, ’understanding’ 128 99 78 78 50
’evidence’, ’questioning’ 89 109 43 97 46
’evidence’, ’recommendation’, ’understanding’ 104 105 59 86 45
’understanding’, ’understanding’, ’questioning’ 69 117 26 119 43
’questioning’, ’questioning’ 99 106 57 87 42
’finalization’, ’questioning’ 71 116 34 110 37
’understanding’, ’social-act’, ’understanding’ 74 116 38 103 36
’understanding’, ’questioning’, ’recommendation’ 115 102 81 78 34
Argumentative relation classes 0%
25%
50%
75%
100%
attack neutral support
Successful (RfC) Unsuccessful (Predecessor)
Figure 12: Distribution of the argumentative relation dimension classes in the first 20 successful and unsuccessful patterns in RfC-Predecessor pairs
Frame patterns
We also applied the PrefixSpan algorithm on the frame labels in the RfC-Predecessor discussion pairs and found 782 patterns in RfC discussions and 310 patterns in Predecessor discussions. Among the extracted patterns we filter successful and unsuccessful patterns in the same way explained above.
Successful and unsuccessful patterns The analysis of the frame dimension in our study reveals some interesting insights into the patterns of successful and unsuccessful discussions. According to our findings, which are presented in table 19 and illustrated in figure 13, the most prevalent class in successful patterns is dialogue, with 65% of patterns containing at least two utterances labeled as such. The second most prevalent class in successful patterns is neutral, with 60% of patterns containing at least two utterances labeled as such. On the other hand, the most prevalent class in unsuccessful patterns is verifiability, with 60% of patterns containing at least two utterances labeled as such. The least common class in both successful and unsuccessful patterns is writing. These results suggest that the use of dialogue and neutral frames is more prevalent in successful discussions, while the use of verifiability frames is more prevalent in unsuccessful discussions.
Table 18: Top 20 successful/unsuccessful patterns in the Argumentative relation dimension.
rfc/pred rank: The ranking of the sequence in RfC/Predecessor discussions. rfc/pred freq: The fre-quency of the sequence in RfC/Predecessor discussions. diff rank: rfc rank - pred rank
Successful sequences rfc rank rfc freq pred rank pred freq diff rank
’neutral’,’support’,’neutral’,’support’,’support’ 86 120 null null null
’neutral’,’support’,’support’,’support’ 74 126 236 46 -162
’neutral’,’support’,’support’,’neutral’,’support’ 98 114 235 46 -137
’neutral’,’neutral’,’support’,’support’ 65 136 179 56 -114
’neutral’,’neutral’,’support’,’neutral’,’support’ 79 124 186 55 -107
’neutral’,’attack’,’support’,’support’ 82 123 181 56 -99
’attack’,’neutral’,’support’,’support’ 68 131 166 59 -98
’attack’,’neutral’,’support’,’neutral’,’support’ 83 121 177 57 -94
’attack’,’support’,’support’ 38 155 106 74 -68
’attack’,’support’,’neutral’,’support’ 50 145 115 72 -65
’neutral’,’support’,’neutral’,’neutral’,’support’ 70 129 133 66 -63
’support’,’neutral’,’support’,’support’ 47 147 108 73 -61
’support’,’support’,’support’ 37 157 88 80 -51
’support’,’neutral’,’support’,’neutral’,’support’ 60 138 111 72 -51
’neutral’,’support’,’attack’,’neutral’,’support’ 99 114 147 63 -48
’neutral’,’support’,’attack’,’support’ 55 142 101 76 -46
’neutral’,’attack’,’neutral’,’neutral’,’support’ 89 118 135 66 -46
’neutral’,’support’,’neutral’,’support’,’neutral’ 49 146 87 80 -38
’support’,’support’,’neutral’,’support’ 53 145 90 79 -37
’neutral’,’neutral’,’neutral’,’support’ 44 149 77 84 -33
Unsuccessful sequences rfc rank rfc freq pred rank pred freq diff rank
’support’,’neutral’,’attack’,’attack’,’neutral’ 174 93 83 83 91
’attack’,’neutral’,’attack’,’neutral’,’attack’ 171 94 92 79 79
’attack’,’neutral’,’attack’,’neutral’,’neutral’ 158 99 81 84 77
’support’,’attack’,’neutral’,’attack’,’neutral’ 132 105 56 98 76
’attack’,’neutral’,’neutral’,’attack’,’neutral’ 166 96 91 79 75
’attack’,’neutral’,’attack’,’support’,’neutral’ 172 94 98 77 74
’support’,’neutral’,’attack’,’neutral’,’attack’ 133 104 66 89 67
’attack’,’neutral’,’attack’,’attack’ 147 101 86 82 61
’support’,’neutral’,’attack’,’neutral’,’neutral’ 121 108 62 92 59
’attack’,’attack’,’neutral’,’neutral’ 128 105 71 86 57
’support’,’neutral’,’neutral’,’attack’,’neutral’ 120 108 65 89 55
’support’,’attack’,’attack’,’neutral’ 107 112 54 102 53
’attack’,’attack’,’neutral’,’attack’ 136 103 85 83 51
’neutral’,’attack’,’neutral’,’neutral’,’neutral’ 143 102 93 78 50
’neutral’,’attack’,’neutral’,’attack’,’neutral’ 87 120 38 116 49
’neutral’,’neutral’,’neutral’,’attack’,’neutral’ 148 101 99 76 49
’attack’,’attack’,’support’,’neutral’ 140 102 95 78 45
’support’,’neutral’,’attack’,’attack’ 105 112 63 92 42
’neutral’,’attack’,’attack’,’neutral’ 76 125 35 122 41
’attack’,’attack’,’attack’ 115 109 76 85 39
Frame classes 0%
20%
40%
60%
80%
dialogue verifiability neutral writing
Successful (RfC) Unsuccessful (Predecessor)
Figure 13: Distribution of the frame dimension classes in the first 20 successful and unsuccessful patterns in RfC-Predecessor pairs
3-Dimension patterns
We combined all three argumentative attribute labels and carry out a similar process to find 3-dimension patterns. We extract 145 sequences from RfC discussions and 65 sequences from the Predecessor conversations.
Successful and unsuccessful patterns We analyze trending successful and unsuccessful discus-sions in 3-dimensional patterns extracted from RfC-Predecessor pairs. According to table 20, one of the most frequent 3-dimension patterns in both successful and unsuccessful discussions is (act: under-standing, relation: neutral, frame: neutral) with a frequency of 45% and 40%, respectively. We can ignore this label as we believe that the high frequency of this shared pattern in both groups is mainly down to the frequent occurrence of understanding class in discourse act attribute and neutral class in argumentative relation and frame attributes (See figures 8, 9, and 10). The second most prevalent 3-dimension label in successful discussions is (finalization, support, dialogue) with at least one occur-rence in 35% of the twenty patterns in the table. It confirms the importance of support and dialogue actions in argumentative relation and frame attributes. Interestingly, there are only two occurrence of attacklabel in the successful patterns as following:
(’recommendation,attack,neutral’)
(’understanding,attack,neutral’—’finalization,support,dialogue’)
The first one is accompanied by a recommendation discourse act label, which shows that the attack action is delivering a recommendation and can be considered a productive action. The second attack is followed by (finalization, support, dialogue) label, which consists of three labels that belong to successful discussions (See figures 11, 12, and 13). On the other hand, in unsuccessful discussions,
the most frequent 3-dimension label is (understanding, neutral, verifiability) with at least one occur-rence in 45% of the patterns in table 20. As expected, the number of patterns containing attack label in unsuccessful discussions is way more than successful ones. For instance, (understanding, attack, verifiability) and (understanding, attack, neutral) occurred at least one time in 15% and 10% of pat-terns, respectively. The key takeaway from this analysis is the fact that if an attack action occurs in a particular discussion, it is critical to be followed by a support (argumentative relation) and dialogue (frame) labeled utterance to avoid discussion derailment.
Table 19: Top 20 successful/unsuccessful patterns in the Frame dimension. rfc/pred rank: The rank-ing of the sequence in RfC/Predecessor discussions. rfc/pred freq: The frequency of the sequence in RfC/Predecessor discussions. diff rank: rfc rank - pred rank
Successful sequences rfc rank rfc freq pred rank pred freq diff rank
’neutral’, ’dialogue’, ’neutral’, ’neutral’, ’dialogue’ 93 111 206 50 -113
’dialogue’, ’dialogue’, ’neutral’, ’neutral’ 97 109 199 51 -102
’dialogue’, ’neutral’, ’dialogue’, ’neutral’, ’dialogue’ 76 118 159 57 -83
’dialogue’, ’neutral’, ’neutral’, ’neutral’ 62 125 141 63 -79
’dialogue’, ’neutral’, ’verifiability’, ’neutral’, ’dialogue’ 91 113 164 56 -73
’dialogue’, ’neutral’, ’neutral’, ’verifiability’ 69 122 129 65 -60
’dialogue’, ’verifiability’, ’dialogue’, ’dialogue’ 82 116 137 64 -55
’dialogue’, ’dialogue’, ’verifiability’, ’dialogue’ 81 116 135 64 -54
’dialogue’, ’writing’, ’neutral’ 66 123 119 67 -53
’dialogue’, ’dialogue’, ’neutral’, ’dialogue’ 47 138 95 72 -48
’neutral’, ’neutral’, ’neutral’, ’dialogue’ 52 133 97 71 -45
’dialogue’, ’verifiability’, ’dialogue’, ’neutral’ 74 119 116 67 -42
’dialogue’, ’verifiability’, ’verifiability’, ’dialogue’ 78 118 118 67 -40
’neutral’, ’neutral’, ’neutral’, ’verifiability’ 88 113 125 65 -37
’dialogue’, ’neutral’, ’neutral’, ’dialogue’ 35 151 71 81 -36
’dialogue’, ’verifiability’, ’neutral’, ’neutral’ 77 118 110 69 -33
’dialogue’, ’neutral’, ’dialogue’, ’dialogue’ 54 130 87 74 -33
’neutral’, ’dialogue’, ’dialogue’, ’neutral’ 95 109 128 65 -33
’neutral’, ’dialogue’, ’neutral’, ’neutral’ 56 128 89 73 -33
’dialogue’, ’writing’, ’dialogue’ 44 142 74 80 -30
Unsuccessful sequences rfc rank rfc freq pred rank pred freq diff rank
’verifiability’, ’writing’, ’verifiability’ 257 80 85 74 172
’neutral’, ’verifiability’, ’dialogue’, ’verifiability’ 155 95 69 83 86
’verifiability’, ’verifiability’, ’verifiability’, ’dialogue’ 176 91 92 72 84
’neutral’, ’verifiability’, ’verifiability’, ’verifiability’ 173 91 99 71 74
’verifiability’, ’neutral’, ’dialogue’, ’verifiability’ 148 96 80 76 68
’verifiability’, ’verifiability’, ’neutral’, ’verifiability’ 128 100 63 85 65
’verifiability’, ’neutral’, ’verifiability’, ’verifiability’ 127 100 64 84 63
’neutral’, ’verifiability’, ’dialogue’, ’dialogue’ 117 102 66 84 51
’verifiability’, ’dialogue’, ’verifiability’, ’neutral’ 105 106 58 87 47
’verifiability’, ’writing’, ’dialogue’ 119 102 73 80 46
’neutral’, ’dialogue’, ’writing’ 143 96 98 71 45
’verifiability’, ’verifiability’, ’verifiability’ 85 115 41 104 44
’writing’, ’writing’ 126 100 82 75 44
’verifiability’, ’dialogue’, ’verifiability’, ’dialogue’ 94 111 56 89 38
’neutral’, ’verifiability’, ’dialogue’, ’neutral’ 84 115 53 92 31
’writing’, ’dialogue’, ’dialogue’ 125 100 96 71 29
’verifiability’, ’neutral’, ’dialogue’, ’dialogue’ 110 104 83 75 27
’neutral’, ’verifiability’, ’verifiability’, ’neutral’ 75 118 48 97 27
’verifiability’, ’verifiability’, ’neutral’, ’dialogue’ 87 114 62 85 25
’writing’, ’verifiability’, ’dialogue’ 104 106 79 77 25
Table20:Top20successful/unsuccessfulpatternsbycombiningallthethreedimension.rfc/predrank:Therankingofthesequencein RfC/Predecessordiscussions.rfc/predfreq:ThefrequencyofthesequenceinRfC/Predecessordiscussions.diffrank=rfcrank-predrank. null:thepatterndoesnotexist. Successfulsequencesrfcrankrfcfreqpredrankpredfreqdiffrank ’understanding,neutral,neutral-POV’—’understanding,support,neutral-POV’4764nullnullnull ’social-act,neutral,neutral-POV’5062nullnullnull ’finalization,support,dialogue’—’finalization,support,dialogue’6136nullnullnull ’recommendation,neutral,neutral-POV’—’finalization,support,dialogue’5361nullnullnull ’understanding,neutral,dialogue’—’finalization,support,dialogue’2288nullnullnull ’questioning,neutral,neutral-POV’—’understanding,neutral,neutral-POV’5857nullnullnull ’understanding,neutral,verifiability’—’finalization,support,dialogue’2784nullnullnull ’understanding,neutral,dialogue’—’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’5957nullnullnull ’understanding,attack,neutral-POV’—’finalization,support,dialogue’3570nullnullnull ’recommendation,attack,neutral-POV’7253nullnullnull ’finalization,support,dialogue’—’finalization,support,dialogue’—’finalization,support,dialogue’6356nullnullnull ’recommendation,support,neutral-POV’4068nullnullnull ’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’—’understanding,neutral,verifiability’6456nullnullnull ’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’7653nullnullnull ’recommendation,neutral,neutral-POV’—’understanding,neutral,verifiability’6555nullnullnull ’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’—’finalization,support,dialogue’4664nullnullnull ’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’—’recommendation,neutral,neutral-POV’6655nullnullnull ’understanding,neutral,neutral-POV’—’finalization,support,dialogue’10115nullnullnull ’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’—’understanding,attack,neutral-POV’6755nullnullnull ’finalization,support,neutral-POV’3179nullnullnull Unsuccessfulsequencesrfcrankrfcfreqpredrankpredfreqdiffrank ’understanding,neutral,verifiability’—’understanding,neutral,verifiability’—’understanding,neutral,neutral-POV’11745375680 ’recommendation,neutral,verifiability’13343554878 ’questioning,neutral,verifiability’—’understanding,neutral,verifiability’12245455177 ’understanding,support,neutral-POV’—’understanding,neutral,dialogue’13443604474 ’understanding,neutral,verifiability’—’understanding,attack,verifiability’12844564872 ’understanding,attack,writing’10746405467 ’questioning,neutral,verifiability’—’understanding,neutral,neutral-POV’11446574657 ’understanding,neutral,verifiability’—’understanding,neutral,neutral-POV’—’understanding,neutral,verifiability’11645594457 ’understanding,attack,verifiability’—’understanding,neutral,neutral-POV’8850346054 ’understanding,attack,neutral-POV’—’understanding,neutral,dialogue’8551365849 ’understanding,attack,verifiability’—’understanding,neutral,verifiability’8251445238 ’understanding,neutral,dialogue’—’understanding,neutral,dialogue’—’understanding,neutral,dialogue’9448584536 ’understanding,neutral,neutral-POV’—’understanding,neutral,verifiability’—’understanding,neutral,neutral-POV’9150614430 ’understanding,attack,neutral-POV’—’understanding,attack,neutral-POV’7453465128 ’understanding,attack,dialogue’4564178128 ’understanding,neutral,verifiability’—’understanding,neutral,neutral-POV’—’understanding,neutral,neutral-POV’5757336024 ’understanding,neutral,neutral-POV’—’understanding,neutral,verifiability’—’understanding,neutral,neutral-POV’5559326123 ’understanding,support,verifiability’4863296519 ’finalization,neutral,neutral-POV’4962316218 ’understanding,neutral,neutral-POV’—’understanding,neutral,verifiability’—’understanding,neutral,verifiability’6855515017