Regular Expression week 8

1. Regular Expressionweek 8

2. 1. What are Regular Expressions? • A regular expression (RE) is a pattern used to search through text. • It either matches the text (or part of it), or fails to match • you can easily extract the matching parts, or change them

3. REs are not easy to use at first • they're like a different programming language inside Java • But, REs bring so much power to string manipulation that they are worth the effort. • Look back at the "Discrete Math" notes on REs and UNIX grep.

4. 2. First Example • The RE "[a-z]+" matches a sequence of one or more lowercase letters • [a-z]means any character from a to z, and • + means “one or more” • Use this pattern to search "Now is the time" • it will match ow • if applied repeatedly, it will find is, the, time, then fail

5. Code Output: owisthetime import java.util.regex.*;public class RegexTest { public static void main(String args[]) { String pattern = "[a-z]+"; String text = "Now is the time"; Pattern p = Pattern.compile(pattern); Matcher m = p.matcher(text); while (m.find()) System.out.println( text.substring( m.start(), m.end() ) ); }}

6. Create a Pattern and Matcher • Compile the pattern • Pattern p = Pattern.compile("[a-z]+"); • Create a matcher for the text using the pattern • Matcher m = p.matcher("Now is the time");

7. Finding a Match • m.find() returns true if the pattern matches any part of the text string; false otherwise • If called again, m.find() will start searching from where the last match was found.

8. Printing what was Matched • After a successful match: • m.start() returns the index of the first character matched • m.end() returns the index of the last character matched,plus one • This is what most String methods require • e.g. "Now is the time".substring(m.start(), m.end())returns the matched substring

9. If the match fails, m.start() and m.end() throw an IllegalStateException • this is a RuntimeException, so you don’t have to catch it

10. Test Rig public class TestRegex { public static void main(String[] args) { if (args.length != 2) { System.out.println("Usage: java TestRegex string regExp"); System.exit(0); } System.out.println("Input: \"" + args[0] + "\""); System.out.println("Regular expression: \"" + args[1] + "\""); Pattern p = Pattern.compile(args[1]); Matcher m = p.matcher(args[0]); while (m.find()) System.out.println("Match \"" + m.group() + "\" at positions "+ m.start() + "-" + (m.end()-1)); } // end of main() } // end of TestRegex class

11. m.group() returns the string matched by the pattern • usually used instead of String.substring()

13. 3. Case Insensitive Matching String sentence ="The quick brown fox and BROWN tiger jumps over the lazy dog"; Pattern pattern = Pattern.compile("brown", Pattern.CASE_INSENSITIVE); Matcher matcher = pattern.matcher(sentence); while (matcher.find()) System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end()); a flag Text "brown" found at 10 to 15. Text "BROWN" found at 24 to 29.

14. Many flags can also be written as part of the RE: Pattern pattern = Pattern.compile("(?i)brown");

15. 4. Some Basic Patterns abcexactly this sequence of three letters [abc]any one of the letters a, b, or c [^abc]any character except one of the letters a, b, or c [a-z]any one character from a throughz [a-zA-Z0-9]any one letter or digit • The set of characters defined by [...] is called a character class.

16. Example Text "bat3" found at 12 to 16. Text "bat4" found at 18 to 22. Text "bat5" found at 24 to 28. Text "bat6" found at 30 to 34. Text "bat7" found at 36 to 40. // search for a string that begins with "bat" and a number in the range [3-7] String input = "bat1, bat2, bat3, bat4, bat5, bat6, bat7, bat8"; Pattern pattern = Pattern.compile( "bat[3-7]" ); Matcher matcher = pattern.matcher(input); while (matcher.find()) System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end());

17. 5. Built-in Character Classes .any one character except a line terminator \da digit: [0-9] \Da non-digit:[^0-9] \s a whitespace character: [ \t\n\x0B\f\r] \Sa non-whitespace character: [^\s] \wa word character:[a-zA-Z_0-9] \Wa non-word character: [^\w] Notice the space

18. In Java you will need to "double escape" the RE backslashes: \\d \\D \\S \\s \\W \\w when you use them inside Java strings • Note: if you read in a pattern from somewhere (the keyboard, a file), there's no need to double escape the text.

19. Example 1 // search for a whitespace, 'f', and any two chars Pattern pattern = Pattern.compile("\\sf.."); Matcher matcher = pattern.matcher( "The quick brown fox jumps over the lazy dog"); while (matcher.find()) { System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end()); Text " fox" found at 15 to 19.

20. Example 2 // match against a digit followed by a word Pattern p = Pattern.compile( "\\d+\\w+" ); Matcher m = p.matcher("this is the 1st test string"); if(m.find()) System.out.println("matched [" + m.group() + "] from " + m.start() + " to " + m.end() ); else System.out.println("didn’t match"); matched [1st] from 12 to 15

21. Subtraction • You can use subtraction with character classes. • e.g. a character class that matches everything from a to z, except the vowels (a, e, i, o, u) • written as [a-z&&[^aeiou]]

22. Search excluding vowels Pattern pattern = Pattern.compile( "[a-z&&[^aeiou]]" ); Matcher matcher = pattern.matcher("The quick brown fox."); while (matcher.find()) { System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end()); Text "h" found at 1 to 2. Text "q" found at 4 to 5. Text "c" found at 7 to 8. Text "k" found at 8 to 9. Text "b" found at 10 to 11. Text "r" found at 11 to 12. Text "w" found at 13 to 14. Text "n" found at 14 to 15. Text "f" found at 16 to 17. Text "x" found at 18 to 19.

23. 6. Sequences and Alternatives • Two patterns matches in sequence: • e.g., [A-Za-z]+[0-9] will match one or more letters immediately followed by one digit • The bar, |, is used to separate alternatives • e.g., abc|xyzwill match either abc or xyz • best to use brackets to make the scope clearer • (abc)|(xyz)

24. Search for 't' or 'T' Pattern pattern = Pattern.compile("[t|T]"); Matcher matcher = pattern.matcher( "The quick brown fox jumps over the lazy dog"); while (matcher.find()) System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end()); Text "T" found at 0 to 1. Text "t" found at 31 to 32.

25. 7. Some Boundary Matchers • ^the beginning of a line • $the end of a line • \ba word boundary • \Bnot a word boundary • \Gthe end of the previous match written as \\b, \\B, and \\G in Java strings

26. Find "dog" at End of Line Pattern pattern = Pattern.compile(“dog$”); Matcher matcher = pattern.matcher( "The quick brown dog jumps over the lazy dog"); while (matcher.find()) System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end()); Text "dog" found at 40 to 43.

27. Look for a Country ArrayList<String> countries = new ArrayList<String>(); countries.add("Austria"); : // more adds /* Look for a country that starts with "I" with any 2nd letter and either "a" or "e" in the 3rd position. */ Pattern pattern = Pattern.compile( "^I.[ae]" ); for (String c : countries) { Matcher matcher = pattern.matcher(c); if (matcher.lookingAt()) System.out.println("Found: " + c); } Found: Iceland Found: Iraq Found: Ireland Found: Italy

28. m.lookingAt() returns true if the pattern matches at the beginning of the text string, false otherwise.

29. Word Boundaries: \b \B • A word boundary is a position between \w and \W (non-word char), or at the beginning or end of a string. • A word boundary is zero length.

30. Examples String s = "A nonword boundary is the opposite of a word boundary, " + "i.e., anything other than a word boundary."; // match all words "word" Pattern p1 = Pattern.compile("\\bword\\b"); Matcher m1 = p1.matcher(s); while (m1.find()) System.out.println("p1 match: " + m1.group() + " at " + m1.start()); // match word ending with "word" but not the word "word" Pattern p2 = Pattern.compile("\\Bword\\b"); Matcher m2 = p2.matcher(s); while (m2.find()) System.out.println("p2 match: " + m2.group() + " at " + m2.start()); p1 match: word at 40 p1 match: word at 83p2 match: word at 5

31. 8. Grouping • A group treats multiple characters as a single unit. • a group is created by placing characters inside parentheses • e.g. the RE (dog) is the group containing the letters "d" "o" and "g".

32. Find the Words 'the' or 'quick' String text = "the quick brown fox jumps over the lazy dog"; Pattern pattern = Pattern.compile( "(the)|(quick)"); Matcher matcher = pattern.matcher(text); while (matcher.find()) { System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(), matcher.end()); Text "the" found at 0 to 3. Text "quick" found at 4 to 9. Text "the" found at 31 to 34.

33. 9. (Greedy) Quantifiers X represents some pattern: X?optional, X occurs once or not at all X*X occurs zero or more times X+X occurs one or more times X{n}X occurs exactly n times X{n,}X occurs n or more times X{n,m}X occurs at least n but not more than m times

34. Example String[] exprs = { "x?", "x*", "x+", "x{2}", "x{2,}","x{2,5}"}; String input = "xxxxxx yyyxxxxxx zzzxxxxxx"; for (String expr : exprs) { Pattern pattern = Pattern.compile(expr); Matcher matcher = pattern.matcher(input); System.out.println("--------------------------"); System.out.format("regex: %s %n", expr); while (matcher.find()) System.out.format("Text \"%s\" found at %d to %d.%n", matcher.group(), matcher.start(),matcher.end());

35. Output Regex: x? Text "x" found at 0 to 1. Text "x" found at 1 to 2. Text "x" found at 2 to 3. Text "x" found at 3 to 4. Text "x" found at 4 to 5. Text "x" found at 5 to 6. Text "" found at 6 to 6. Text "" found at 7 to 7. Text "" found at 8 to 8. Text "" found at 9 to 9. Text "x" found at 10 to 11. Text "x" found at 11 to 12. Text "x" found at 12 to 13. Text "x" found at 13 to 14. Text "x" found at 14 to 15. Text "x" found at 15 to 16. Text "" found at 16 to 16. Text "" found at 17 to 17. Text "" found at 18 to 18. Text "" found at 19 to 19. Text "x" found at 20 to 21. Text "x" found at 21 to 22. Text "x" found at 22 to 23. Text "x" found at 23 to 24. Text "x" found at 24 to 25. Text "x" found at 25 to 26. Text "" found at 26 to 26. ------------------------------

36. Regex: x* Text "xxxxxx" found at 0 to 6. Text "" found at 6 to 6. Text "" found at 7 to 7. Text "" found at 8 to 8. Text "" found at 9 to 9. Text "xxxxxx" found at 10 to 16. Text "" found at 16 to 16. Text "" found at 17 to 17. Text "" found at 18 to 18. Text "" found at 19 to 19. Text "xxxxxx" found at 20 to 26. Text "" found at 26 to 26. ------------------------------ Regex: x+ Text "xxxxxx" found at 0 to 6. Text "xxxxxx" found at 10 to 16. Text "xxxxxx" found at 20 to 26. ------------------------------ Regex: x{2} Text "xx" found at 0 to 2. Text "xx" found at 2 to 4. Text "xx" found at 4 to 6. Text "xx" found at 10 to 12. Text "xx" found at 12 to 14. Text "xx" found at 14 to 16. Text "xx" found at 20 to 22. Text "xx" found at 22 to 24. Text "xx" found at 24 to 26. ------------------------------ Regex: x{2,} Text "xxxxxx" found at 0 to 6. Text "xxxxxx" found at 10 to 16. Text "xxxxxx" found at 20 to 26. ------------------------------ Regex: x{2,5} Text "xxxxx" found at 0 to 5. Text "xxxxx" found at 10 to 15. Text "xxxxx" found at 20 to 25.

37. Matching SSN Numbers ArrayList<String> input = new ArrayList<String>(); input.add("123-45-6789"); input.add("9876-5-4321"); input.add("987-65-4321 (attack)"); input.add("987-65-4321 "); input.add("192-83-7465"); for (String ssn : input) if (ssn.matches("^(\\d{3}-?\\d{2}-?\\d{4})$")) System.out.println("Found good SSN: " + ssn); Found good SSN: 123-45-6789 Found good SSN: 192-83-7465

38. String.matches(String regex) returns true or false depending on whether the string matches the RE (regex). • str.matches(regex) is the same as:Pattern.matches(regex, str)

39. 10. Three Types of Quantifiers • 1. A greedy quantifier will match as much as it can, and back off if it needs to • see examples on previous slides • 2. A reluctant quantifier will match as little as possible, then take more if it needs to • you make a quantifier reluctant by adding a ?:X??X*?X+?X{n}?X{n,}?X{n,m}?

40. 3. A possessive quantifier will match as much as it can, and never lets go • you make a quantifier possessive by appending a +:X?+X*+X++X{n}+X{n,}+X{n,m}+

41. Quantifier Examples • The text is "aardvark". • 1. Use the pattern a*ardvark(a* is greedy) • the a*will first match aa, but then ardvarkwon’t match • the a* then “backs off” and matches only a single a, allowing the rest of the pattern (ardvark) to succeed

42. 2. Use the pattern a*?ardvark(a*? is reluctant) • the a*? will first match zero characters (the null string), but then ardvarkwon’t match • the a*? then extends and matches the first a, allowing the rest of the pattern (ardvark) to succeed

43. 3. Using the pattern a*+ardvark(a*+ is possessive) • the a*+ will match the aa, and will not back off, so ardvark never matches and the pattern match fails

44. Reluctant Example Pattern pat = Pattern.compile("e.+?d"); Matcher mat = pat.matcher("extend cup end table"); while (mat.find()) System.out.println("Match: " + mat.group()); Output: Match: extend Match: end

45. 11. Capturing Groups • Parentheses are used for grouping, but they also capture (keep for later use) anything matched by that part of the pattern. • Example:([a-zA-Z]*)([0-9]*)matches any number of letters followed by any number of digits • If the match succeeds: • \1holds the matched letters • \2 holds the matched digits • \0 holds everything matched by the entire pattern

46. Capturing groups are numbered by counting their opening parentheses from left to right: • ( ( A ) ( B ( C ) ) )1 2 3 4\0 = \1 = ((A)(B(C))), \2 = (A), \3 = (B(C)), \4 = (C) • Example: ([a-zA-Z])\1will match a double letter, such as letter

47. Matcher.group() • If mis a matcher that has just got a successful match, then • m.group(n)returns the String matched by capturing group n • this could be an empty string • this will be nullif the pattern as a whole matched but this particular group didn’t match anything • m.group(0) returns the String matched by the entire pattern (same as m.group()) • this could be an empty string

48. Examples • Move all the consonants at the beginning of a string to the end • "sheila"becomes "eilash" Pattern p = Pattern.compile( "([^aeiou]*)(.*)" );Matcher m = p.matcher("sheila");if (m.matches()) System.out.println(m.group(2) + m.group(1)); • (.*) means “all the rest of the chars”

49. 12. Escaping Metacharacters • A lot of special characters – parentheses, brackets, braces, stars, the plus sign, etc. – are used in REs • they are called metacharacters

50. Suppose you want to search for the character sequence a*(an a followed by an ordinary "*") • "a*"; doesn’t work; that means “zero or more a's” • "a\*"; doesn’t work; since a star doesn’t need to be escaped in Java String constants; Java ignores the \ • "a\\*"does work; it’s the three-char string a, \, * • Just to make things even more difficult, it’s illegal to escape a non-metacharacter in a RE.