The Power of Noticing in Teaching Legal Writing to LLMs

1. The Power of Noticing in Teaching Legal Writing to LLMs May 1, 2017 Global Legal Writing Skills Webinar “Teaching Global Skills to International & U.S. Law Students” Stephen Horowitz Director of Legal English Programs stjohnslegalenglish.com

2. About me: Director of Legal English Programs Started at St. John’s Law in 2014 MA TESOL (CUNY-Hunter College, 2015) CELTA English Language Teaching Certificate (2012) Worked in corporate bankruptcy law (1998-2010) J.D., Duke Law School + NY Bar (1998) Taught English in Japan on Japan Exchange Teaching (JET) Program (1992-94)

3. St. John’s Law School LLM programs for non-native English speakers: US Legal Studies (USLS) • Primarily foreign-trained lawyers who have emigrated to the NY area • Bar exam oriented Transnational Legal Practice (TLP) • 100% international students • Focus on cross-border legal practice • Pathway available for students who want to take the bar American Law Discourse & Analysis (ALDA) • One/two-semester law + language support LLM program • For students accepted to LLM who need extra language support • Also, certificate program offered for foreign-trained lawyers interested in non-degree legal English program

4. My role: Pre-LLM programs and courses • ALDA • 4-week summer English for American Law School course On-going support • Bar Exam Language & Strategies • Reading Support • Workshops for professors who work with NNES LLMs Credit courses • Drafting Litigation & Contract Documents (descriptivist approach)

5. “J’ever notice....” • Noticing • In the world of comedy • Passive • In teaching • Intentional act • Powerful tool for teaching and learning

6. The Noticing Hypothesis “Noticing is the essential starting point for acquisition.” “What happens within attentional space largely determines the course of language development, including the growth of knowledge and the development of fluency.” Schmitt, R. (2010). Attention, awareness, and individual differences in language learning.

7. Noticing & Descriptive Teaching Strategies Examples of ways to use noticing in teaching legal writing

8. Example #1: Capital letters Problem: Frequent mistakes with capitalization Strategy: • With student, co-construct a list of rules for why we capitalize letters. • Choose a text familiar to the student (e.g., a legal memo) • Have student read through text and identify all capital letters. • For each capital letter, student needs to identify the reason. • For any that don’t fit the “rules,” student must devise a new rule.

9. Example #2: Punctuation Problem: Punctuation errors Same Strategy: • With student, co-construct a list of rules for why we [e.g., use commas]. • Choose a text familiar to the student (e.g., a legal memo) • Have student read through text and identify all [commas]. • For each [comma], student needs to identify the reason. • For any that don’t fit the “rules,” student must devise a new rule.

10. Example #3: Grammar Problem: Errors with verb tense Strategy: • List the rules for the tense. • Select a text. Ideally the kind the student needs to write. • Identify all the verbs in the text. • Notice the tense of each verb. • Pay particular attention to shifts in tense, within a sentence as well as from section to section. • Devise rules or reasons for why the writer used a particular tense or changed tense. • Goal: Help student connect grammar form with communicative purpose.

11. Example #4: Vocabulary Problem: Incorrect use of “prima facie” Strategy: • Have student go to Google Scholar/Case law (or Google News). • Search for “prima facie”. • Gather sentences with “prima facie”. • List observations and patterns with regard to situation, syntax, collocations, etc. • e.g., About 90% of the time “prima facie” collocates with “case” or “showing” • Also can be used as an adverb (“prima facie valid”)

12. Example #5: Application of a rule Problem: In the Application section of an IRAC-style writing, student does not sufficiently connect the Rule to the Facts. Note on cultural differences: Reader responsible vs. Writer responsible Strategy: • Use model answers to help the student notice “mirroring,” i.e., where a writer incorporates key words from the rule and connects them to the facts.

13. Step 1: Identify key words in the rule The ordinance makes it unlawful “to view, send, or compose an electronic message...while operating a motor vehicle.” Here, Jefferson clearly composed and then sent an electronic message when she wrote and sent her friend a text saying “almost there.” Additionally, she did this while operating a motor vehicle since the facts state that she slowed down at a crosswalk at that time. If Jefferson slowed down, that means the car was still in drive and her foot was at best on the brakes.

14. Step 2: Notice how key words from the Rule match up with key facts The ordinance makes it unlawful “to view, send, or compose an electronic message...while operating a motor vehicle.” Here, Jefferson clearly composed and then sent an electronic message when she wrote and sent her friend a text saying “almost there.” Additionally, she did this while operating a motor vehicle since the facts state that she slowed down at a crosswalk at that time. If Jefferson slowed down, that means the car was still in drive and her foot was at best on the brakes.

15. Step 3: Notice words/language used to show equivalence of Rule and Facts words The ordinance makes it unlawful “to view, send, or compose an electronic message...while operating a motor vehicle.” Here, Jefferson clearly composed and then sent an electronic messagewhen shewrote and sent her friend a text saying “almost there.” Additionally, she did this while operating a motor vehiclesince the facts state that she slowed down at a crosswalk at that time. If Jefferson slowed down, that meansthe car was still in drive and her foot was at best on the brakes.

16. Example #6: Noticing as a tool for teaching plagiarism and citation Prescriptivist approach • Plagiarism is bad • Plagiarism is when you use someone else’s ideas without citing them. • Serious consequences. Problem: How do I know if I’m “plagiarizing”? Solution: Help students notice discourse and language clues of attribution and non-attribution.

17. Task: Noticing citation and non-citation In a legal memo or law review article, how many sentences: • do have citations/footnotes • don’t. What language or other patterns do you notice about sentences: • without citations/footnotes? • with citations/footnotes?

18. Task: Noticing citation and non-citation [1]The ABC Stadium sidewalk is most likely a designated public forum. [2]A designated public forum is one which is not a traditional open forum described above, but which the state has opened up as a place for public use and free expression, in a manner just like a traditional public forum. Perry, 460 U.S.at 45. [3]A designated public forum can only be created through government intent. [4]According to the Supreme Court, “[t]he government does not create a public forum by inaction or by permitting limited discourse, but only by intentionally opening a nontraditional forum for public discourse.” Cornelius, 473 U.S. at 802. [5]The Supreme Court and federal circuit courts applying Supreme Court doctrine rely on objective manifestations of government intent to determine whether the government intended to designate a particular place as a public forum. [6]For instance, in Paulsen v. County of Nassau, the Second Circuit considered whether a concert stadium had been designated a public forum, which would require a government agency to allow a religious organization to hand out leaflets on the sidewalks surrounding the stadium. 925 F.2d 65, 69 (2d Cir. 1991). [7]The Second Circuit first looked at the government instrument which created the stadium, which provided that the stadium was to be used for the “education, enlightenment, cultural development or betterment, . . . and . . . civic, community and general public interest.” Id. at 70. [8]Furthermore, the stadium had been used in the past for “parades, political rallies and speeches, religious weddings and circuses.” Id. [9]Last, the Second Circuit decided that the nature of the property is compatible with expressive activity, since it can “comfortably accommodate a wide variety of expressive activities at the same time.” Id. [10]As a result of these indicia of intent, the court held that the stadium premises had been designated a public forum, and the stadium authorities could not prohibit the religious group from distributing literature. -Source: Legal memo written by JD student

19. Sentences with no citation [1]The ABC Stadium sidewalk is most likely a designated public forum. [2]A designated public forum is one which is not a traditional open forum described above, but which the state has opened up as a place for public use and free expression, in a manner just like a traditional public forum. Perry, 460 U.S.at 45. [3]A designated public forum can only be created through government intent. [4]According to the Supreme Court, “[t]he government does not create a public forum by inaction or by permitting limited discourse, but only by intentionally opening a nontraditional forum for public discourse.” Cornelius, 473 U.S. at 802. [5]The Supreme Court and federal circuit courts applying Supreme Court doctrine rely on objective manifestations of government intent to determine whether the government intended to designate a particular place as a public forum.[6]For instance, in Paulsen v. County of Nassau, the Second Circuit considered whether a concert stadium had been designated a public forum, which would require a government agency to allow a religious organization to hand out leaflets on the sidewalks surrounding the stadium. 925 F.2d 65, 69 (2d Cir. 1991). [7]The Second Circuit first looked at the government instrument which created the stadium, which provided that the stadium was to be used for the “education, enlightenment, cultural development or betterment, . . . and . . . civic, community and general public interest.” Id. at 70. [8]Furthermore, the stadium had been used in the past for “parades, political rallies and speeches, religious weddings and circuses.” Id. [9]Last, the Second Circuit decided that the nature of the property is compatible with expressive activity, since it can “comfortably accommodate a wide variety of expressive activities at the same time.” Id. [10]As a result of these indicia of intent, the court held that the stadium premises had been designated a public forum, and the stadium authorities could not prohibit the religious group from distributing literature. -Source: Legal memo written by JD student

20. Sentences with citation [1]The ABC Stadium sidewalk is most likely a designated public forum. [2]A designated public forum is one which is not a traditional open forum described above, but which the state has opened up as a place for public use and free expression, in a manner just like a traditional public forum. Perry, 460 U.S.at 45. [3]A designated public forum can only be created through government intent. [4]According to the Supreme Court, “[t]he government does not create a public forum by inaction or by permitting limited discourse, but only by intentionally opening a nontraditional forum for public discourse.” Cornelius, 473 U.S. at 802. [5]The Supreme Court and federal circuit courts applying Supreme Court doctrine rely on objective manifestations of government intent to determine whether the government intended to designate a particular place as a public forum. [6]For instance, in Paulsen v. County of Nassau, the Second Circuit considered whether a concert stadium had been designated a public forum, which would require a government agency to allow a religious organization to hand out leaflets on the sidewalks surrounding the stadium. 925 F.2d 65, 69 (2d Cir. 1991). [7]The Second Circuit first looked at the government instrument which created the stadium, which provided that the stadium was to be used for the “education, enlightenment, cultural development or betterment, . . . and . . . civic, community and general public interest.” Id. at 70. [8]Furthermore, the stadium had been used in the past for “parades, political rallies and speeches, religious weddings and circuses.” Id.[9]Last, the Second Circuit decided that the nature of the property is compatible with expressive activity, since it can “comfortably accommodate a wide variety of expressive activities at the same time.” Id. [10]As a result of these indicia of intent, the court held that the stadium premises had been designated a public forum, and the stadium authorities could not prohibit the religious group from distributing literature. -Source: Legal memo written by JD student

21. Things you might have noticed Sentences • No citation: 4 sentences - 1, 3, 5, 10 • Citation: 6 sentences - 2, 4, 6, 7, 8, 9 Non-citation sentence features • Frequently first and last sentences of paragraphs • Statements of evaluation, opinion, prediction • “most likely” • Paraphrases or summaries of statements that have just been cited

22. More things you might have noticed Citation sentence features • Direct quotes • Definitions of key concepts • Paraphrased language from a court decision • Direct reference in the sentence to a source of authority • “According to the Supreme Court,...” • “For instance, in Paulsen v. County of Nassau...” • “The Second Circuit first looked at .....” • Use of Id. - What it means and when it is used

23. Big Picture: Advantages of using noticing Motivation: • Use actual language the students need to understand • Connection to meaning and function Develop intuition: The main thing NNES lack. Independence: Learning strategy for future use Descriptivism: • Help LLM students learn the language that NES-law-school-educated people actually use. • When a student asks you “why” + language/grammar question, you can say, “I’m not sure, but let’s find out!”

24. Try it at home! Step 1: • Sit down with actual texts, • look for examples of target language, grammar, discourse, etc., and.... • see what you notice. Step 2: • Take your students on the same journey.

25. Thank you! Questions, comments & continued conversation: Stephen B. Horowitz Director of Legal English Programs St. John’s University School of Law Office of Graduate Studies horowis1@stjohns.edu (718) 990-6625 StJohnsLegalEnglish.com