Psychometric.AI: Building Intelligent Agents for IQ Testing

1. An Evening with PERI, "The IQ GURU" Selmer Bringsjord, Bettina Schimanski Cognitive Science / Computer Science Depts. RAIR (Rensselaer Artificial Intelligence and Reasoning) Lab selmer@rpi.edu ; schimb@rpi.edu October 17, 2002

2. Overview •  What is Psychometric AI? • Intelligence Tests and Artificial Intelligence • Description of the Block Design Task • PERI, PERI, and more PERI • Story Completion Task • Narratological Reasoning • Future Related Goals and Research • Object Assembly Task • Other tests, including TTT

3. An Answer to: What is AI? A New Kind of AI • Assume the ‘A’ part isn’t the problem: we know what an artifact is. • Psychometric AI offers a simple but radical answer: • Psychometric AI is the field devoted to building information-processing entities capable of at least solid performance on all established, validated tests of intelligence and mental ability, a class of tests that includes IQ tests, tests of reasoning, of creativity, mechanical ability, and so on. • Don’t confuse this with: “Some human is intelligent…” • Psychologists don’t agree on what human intelligence is. • Two notorious conferences. See The g Factor. • But we can agree that one great success story of psychology is testing, and prediction on the basis of it. (The Big Test) AI is the field devoted to building intelligent artificial agents, i.e., agents capable of solid performance on intelligence tests.

4. Psychometric AI • Psychologists don’t agree on what human intelligence is but we can agree that one great success story of psychology is testing, and prediction on the basis of it. •  Definition: Some artificial agent is intelligent if and only if it excels at all established, validated tests of [human] intelligence.

5. Intelligence Tests: Narrow vs. Broad Thurstone’s view of intelligence Spearman’s view of intelligence

6. Artificial Agent to Crack RPM ---------------- PROOF ---------------- 1 [] a33!=a31. 3 [] -R3(x)| -T(x)|x=y| -R3(y)| -T(y). 16 [] R3(a31). 24 [] T(a31). 30 [] R3(a33). 31 [] T(a33). 122 [hyper,31,3,16,24,30,flip.1] a33=a31. 124 [binary,122.1,1.1] $F. ------------ end of proof ------------- ----------- times (seconds) ----------- user CPU time 0.62 (0 hr, 0 min, 0 sec)

7. Artificial Agent to Crack RPM =========== start of search =========== given clause #1: (wt=2) 10 [] R1(a11). given clause #2: (wt=2) 11 [] R1(a12). given clause #3: (wt=2) 12 [] R1(a13). ... given clause #4: (wt=2) 13 [] R2(a21). given clause #278: (wt=16) 287 [para_into,64.3.1,3.3.1] R2(x)| -R3(a23)| -EmptyBar(y)| -R3(x)| -EmptyBar(x)| -T(a23)| -R3(y)| -T(y). given clause #279: (wt=16) 288 [para_into,65.3.1,8.3.1] R2(x)| -R3(a23)| -StripedBar(y)| -R3(x)| -StripedBar(x)| -EmptyBar(a23)| -R3(y)| -EmptyBar(y). Search stopped by max_seconds option. ============ end of search ============ Correct!

8.  One such well-established test is the WAIS IQ test (Wechsler Adult Intelligence Scale).  By attempting to score relatively well on this test with our AI Agent and being successful (as well on others with a general algorithm), what will this say about human intelligence, these IQ tests, etc.?

9. Possible objection to the Psychometric AI: “If one were offered a machine purported to be intelligent, what would be an appropriate method of evaluating this claim? The most obvious approach might be to give the machine an IQ test … However, [good performance on tasks seen in IQ tests would not] be completely satisfactory because the machine would have to be specially prepared for any specific task that it was asked to perform. The task could not be described to the machine in a normal conversation (verbal or written) if the specific nature of the task was not already programmed into the machine. Such considerations led many people to believe that the ability to communicate freely using some form of natural language is an essential attribute of an intelligent entity.” (Fischler & Firschein 1990, p. 12)

10. Our Rebuttal: • PERI will be able to solve many puzzles, without getting advance notice. • PERI should specifically solve all of the WAIS! • From there, we move to all other established tests. • PERI is not, and will not be, pre-engineered! •  Humans will communicate with PERI in natural language.

11. Pie Piece Design Task Basic Setup Solution Problem

12. Block Design Task

13. Very General Block Design Task Algorithm: • Interpret and symbolize the initial configuration of the pieces unless they will always be started in the same set-up. • Do the same for the goal configuration of the pieces. • Determine if the puzzle is solvable (there are enough pieces available, enough with correct sides needed and colors, etc.) • If so, pick a piece to start with and move it into position. • Continue until the solution has been reached. Complete Algorithm?

14. PERI System: Flow of Information Vision System Common Lisp Program Environment Robot Speech

15. PERI (Psychometric Experimental Robotic Intelligence)

16. Scorbot-ER IX Robot from the Intelitek Corporation • Capable of lifting objects of 4.4 lbs. or less • Five axes (human arm has seven) • Comes with several different software options • ACL (Advance Control Language) • ACL - Offline • ATS (Advanced Terminal Software) • Scorbase Pro • RoboCell

17. How PERI Moves • Pivot Points: • Base • Shoulder • Elbow • Wrist Bend • Wrist twist • Hand: Gripper (Open or Closed positions only) • Movement: Joint Coordinates vs. XYZ Coordinates

18. Joint coordinates are easier to code but are they easy to “guess” for dynamic programming for the robot in the future?

19. XYZ Coordinates are harder for humans to guess in this case and must be more accurate, but may be easier for a program to calculate.

20. Some Typical PERI Library Functions: special-write (writes specific strings to output file which is current communication) get-initial-setup-from-vision get-goal-setup-from-vision alert-piece-changed (vision system status report) find-piece-orientation find-piece-location get-piece-colors turn-piece-right turn-piece-left turn-piece-up turn-piece-down move-piece get-new-vision-input

21. A portion of the Robot ACL code: DEFP HM ;defines position HM – the HOME position DIMP ARR[33] ;defines vector ARR: 33 positions: ARR[1] - ARR[33] HOME ;drives all robot axes to their home position by ;searching for a microswitch on each axis SETPV HM ;records joint values for position HM -13543 -58700 16264 27779 -2 SETPV ARR[1] ;records joint values for position ARR[1] 26318 -58700 16264 27779 -2

22. PERI VIDEO

23. Picture Completion

24. Picture Completion Currently untouchable AI -- but we shall see.

25. Narratological Reasoning & Thwarting Terrorism… Terrorists struggle to make stories real. Their behavior can be anticipated, and thus thwarted. We need computers that can imagine future cvents in a (twisted) narrative.

26. Disadvantages without the Vision System • Blocks must be set-up the same way each time or input differently from a file • Input file tells set-up of all blocks including color, orientation, position • No dynamic feedback to Brain-program • == cannot solve general puzzle YET

27. Future Goals for PERI • Combine the CL Program, Vision Software, Robotic Software, and Speech system into one working package • Tackle the Story Completion Task. What would this mean if we’re successful? And for other tests? (abstract algorithms for any such problems) •Using PERI for instruction in classes

28. Object Assembly Task – the General Case • Want to be able to solve the general object assembly task • Any number of objects of any shape, color, orientation • Any goal configuration

29. General Algorithm for any type of puzzle pieces: (not necessarily a complete alg.)  1) Document the number of original pieces by color, dimension, size, and number of sides. Number the pieces in order when input. Create a flag for each that is initially not flagged. This will be used later to determine if the piece is already used in the solution. (Flagging is for non-vision input) 2) Input the goal configuration similarly to the original pieces. Also number each piece and create a flag for each. When a goal piece is not flagged, this will be used to signify whether the piece has been put in the solution already. (Flagging is for non-vision input) 3) Partition the goal into pieces that match available pieces in the original. (I.e. If there is a piece that does not match the dimensions of any original piece, try and break it up into pieces that will match smaller original pieces). Depending on the type of puzzle, you may be able to ignore the color of the pieces for now. 4) Once all pieces in the goal have been partitioned, make sure the goal is solvable (i.e. there are enough pieces with the right matching colors as the original pieces). If not, it is possible the goal has not been partitioned correctly. Either declare defeat or go back to step 3 and partition differently, making sure not to partition exactly like an earlier partitioning. If a partitioning of the goal is solvable, continue onto step 5. 5) Start with the lowest numbered goal piece that is not yet flagged. Now match it to an original piece that is also not yet flagged. Once such a match is found (which is a finite search since coming from step 4 the goal is known to be solvable) set the flag for the original piece and the goal piece and add the piece to the solution array by changing the coordinates of the original piece as well as the angle if this is an option. 6) Continue step 5 until no more pieces in the goal exist that are not flagged.