Speaking patterns - MAS.662J, Fall 2004

1. Speaking patterns-MAS.662J, Fall 2004 Diane Hirsh & Xian Du Dec-07-2004

2. Outline • Introduction • Data at hand • Objectives • Applied Methods • Results • Conclusion and Comments • References

3. Introduction • Group debate always leads to only two kinds of final decision: right or wrong • Debating Member always influences each other by different speaking patterns • The speaker’s pattern and other speakers’ influences lead to the final decision of the debate • Identifying those patterns and influences can be helpful to the prediction of the debate result and member’s option

4. Data at hand • Raw data is Speaker ID and stamp time • Labeled data indicates the initial position and final outcome of the speakers Fig. 1 talking sequences of the four members in group: study_07_task1

5. Objectives • Find the distinct feature to discriminate the right-decision from wrong-decision group • Predict the winner of the project • Tell the individual’s position

6. Applied Methods • Preprocessing • “Turn” [1,2] : “for each unit of time we estimate how much time each of the participants speaks, the participants who has the highest fraction of speaking time is considered to hold the “turn” for that time unit. For a given interaction, we can easily estimate how a pair participating in the conversation transitions between turns.”

7. Applied Methods • Recognition techniques • Parzen Window & Linear Discriminant Function with one-leave-out validation • Hidden Markov Models (HMM)

8. Parzen Window & Linear Discriminant Function with one-leave-out validation • Goal: - to discriminate the right-decision group from wrong-decision group • Assumption: - Wrong decision group has a “wrong” density function (Parzen window) - There is a hyperplane H to divide the “turns” space into half spaces: right or wrong • Group 07 and 12 are two wrong groups in the training groups

9. Results for Parzen Window & Linear Discriminant Function application • Parzen window : - 07 group: 6 in 10 right groups are identified but Minimum error rate~0.5 - 12 group: fail (5 in 10 and ~0.5) • Linear Discriminant 07 group: 8 in 10 right groups are identified with Minimum error rates: 0.057~0.47, AVG=0.269 - 12 group: fail (5 in 10 and ~0.5)

10. Hidden Markov Models (HMM) • Single HMM • Identify Group option (wrong/right) • Parallel HMM • Identify Group option (wrong/right) • Identify members’ state option (probability of the final decision) • Influence model - Improve the result of parallel HMM by considering the influence between members in the group

11. Implementation of HMM • Assumptions: - each member in one group has influence on others by turns “amount” and more turns contribute to higher influence. - each member retains its initial state or changes to be opposite. The transition is strictly one direction. • Initializations: - randomize the initialization of transition matrix while keeping the HMM strictly left-to-right. - two states for each group: right or wrong; two observation symbols: 0 or 1. - The initial states of each member in the group are set according to the initial position in labeled data (e.g.1/4)

12. a11 a22=1 a11 a22=1 Influence model a12 a12 S1 S2 S1 S2 1 4 a11 a22=1 2 3 a11 a22=1 a12 a12 S1 S2 S1 S2 Fig. 2 The dynamic structure of influence model with four members’ HMMs (arrow in the influence model indicates the influence weight on c by c’) [3, 5] Implementation of HMM

13. Result for HMM • Single HMM for right-wrong groups’ separation: cannot find wrong groups e.g.. training data: [1,2,3,4,5,6,8,9,10,11] and [7,12] testing data: [1,2,3,5,6,7,8,9,10,12] and [4,11] Confusion Matrix for the Test Data (test 9) Recognized as right Recognized as wrong Class 1 8 0 Class 2 2 0

14. Result for HMM • Parallel HMM (group 8 and group 11 used) Recognition accuracy= 54.4% Members’ state transition matrix: - group 08: 1/4 meet labeled data - group 11: 3/4 meet labeled data • Influence HMM model Recognition accuracy= 61.0% Members’ state transition matrix: - group 08: 2/4 meet labeled data - group 11: 3/4 meet labeled data

15. Conclusion and Comments • The data set is not friendly for HMM because different training group has different members and each group has only one speech sequence; • Influence model improves the HMM recognition accuracy but its random initial state probability limits its application in this project (it needs more training) and its result up to now failed to find the winner; • Linear discriminant function recognizes some right-wrong group well but not all (more data needed for the testing); • The length of talking time varies a lot among different groups which limits the recognition; • More features may be helpful for this project.

16. References • Tanzeem Khalid Choudhury, “Sensing and modeling human networks”, PhD thesis, MIT, Cambridge, MA, 2004 • Chalee Asavathiratham, “the influence model: a tractable representation for the dynamics of networked Markov chains”, PhD thesis, MIT, Cambridge, MA, 2001 • A Pentland, Learning communications-understanding information flow in human networks, BT technology Journal, vol. 22, No4, October 2004 • Shi Zhong and Joydeep Ghosh, A new formulation of coupled hidden markov models, A new formulation of coupled hidden markov models, Tech. Report, June, 2001 • YongHong Tian, etc. Incremental learning for interaction dynamics with the influence model, IEEE, www-2.cs.cmu.edu/~dunja/LinkKDD2003/papers/Tian.pdf • Lawrence Rabiner and Biing-Hwang Juang, Fundamentals of speech recognition, Prentice Hall, 1993