Dependency Model Using Posterior Context

1. Dependency Model Using Posterior Context Kiyotaka Uchimoto † Masaki Murata † Satoshi Sekine ‡ Hitoshi Isahara † † Kansai Advanced Research Center, Communications Research Laboratory, Japan ‡ New York University, USA

2. dependency 太郎は 赤い 赤　い 太郎　は バラ　を 買い　ました。 バラを Taro_wa bara_wo kai_mashita Aka_i Taro red rose bought 買いました。 bunsetsu Background • Japanese dependency structure analysis 太郎は赤いバラを買いました。 Taro bought a red rose. • Preparing a dependency matrix • Finding an optimal set of dependencies for the entire sentence

3. Conventional (old) model :bunsetsu dependency  • Statistical approach • Each element in the dependency matrix is estimated as a probability. • Assigning one of two tags, a “1” or a “0,” to each relationship between two bunsetsus • Whether or not there is a dependency between two bunsetsus • Considers only the relationship between two bunsetsus. or “1” “0”

4. New model using posterior context dependent between beyond  or or “1” “0” “0” “2” • A relationship between two bunsetsus • The anterior bunsetsu can depend on • “0”: a bunsetsu between the two • “1”: the posterior bunsetsu • “2”: a bunsetsu beyond the posterior one • The dependency probability of two bunsetsus • Product of the probabilities of the relationship between the left bunsetsu and those to its right in a sentence • Overall dependencies in a sentence • Product of the probabilities of all the dependencies • Identified by analyzing a sentence from right to left

5. dpnd btwn btwn btwn btwn bynd dpnd btwn btwn btwn bynd bynd dpnd btwn btwn bynd bynd bynd dpnd btwn bynd bynd bynd bynd dpnd Bunsetsu :Current bunsetsu 1 3 4 5 2 :Modifiee candidates Normalized dependency probability 0.4 × 0.1 × 1.0 × 1.0 × 0.6 = 0.155 18.0% 0.6 × 0.3 × 1.0 × 1.0 × 0.6 = 0.329 38.1% 0.6 × 0.6 × 0 × 1.0 × 0.6 = 0 0.6 × 0.6 × 1.0 × 0 × 0.6 = 0 0.6 × 0.6 × 1.0 × 1.0 × 0.4 = 0.379 43.9%

6. Experiments • Implemented the models within a maximum entropy framework • Features: basically some attributes of a bunsetsu itself or those between bunsetsus • Using the Kyoto University text corpus (Kurohashi and Nagao, 1997) • a tagged corpus of the Mainichi newspaper • Training: 7,958 sentences (Jan. 1st to 8th) • Testing: 1,246 sentences (Jan. 9th) • The input sentences were morphologically analyzed and their bunsetsus were identified correctly.

7. Results of dependency analysis • The accuracy of the new model was about 1% better than that of the old model and there was a 3% improvement in sentence accuracy even using exactly the same features.

8. Relationship between the number of bunsetsus and accuracy • The accuracy of the new model is almost always better than that of the old model.

9. Amount of training data and accuracy • The accuracy of the new model is about 1% higher than that of the old model for any size of training data.

10. Conclusion • A new model for dependency structure analysis • Learns the relationship between two bunsetsus as three categories; “between,” “dependent,” and “beyond.” • Estimates the dependency likelihood by considering not only the relationship between two bunsetsus but also the relationship between the left bunsetsu and all of the bunsetsus to its right. • The dependency accuracy of the new model was • Almost always better than that of the old model for any sentence length. • About 1% higher than that of the old model for any size of training data used. • Future work • Applying the similar model to English sentences