1 / 10

HAL: Human Activity language ( the grammars of human behavior )

HAL: Human Activity language ( the grammars of human behavior ). Yiannis Aloimonos: Big Brother problems Ken Nakayama: Visual Psychophysics of Action. Action Representations. Linguistic. Visuo-motor. Motoric. Visual. View-invariant eigenposes (filters) using PCA.

shaunaf
Download Presentation

HAL: Human Activity language ( the grammars of human behavior )

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HAL: Human Activity language(the grammars of human behavior) Yiannis Aloimonos: Big Brother problems Ken Nakayama: Visual Psychophysics of Action

  2. Action Representations Linguistic Visuo-motor Motoric Visual

  3. View-invariant eigenposes (filters) using PCA Filters for detecting humans Basic pipeline An example using depth Key poses (depth maps) TRAINING POSES Motion/Depth/Appearance from various viewpoints Dimensionality reduction (linear/nonlinear) Views View-invariant filters for detecting humans Apply to a new video

  4. Part of the HMM

  5. Being Feeling Thinking Moving Ground General Object General Person Specific Object Specific Person Group Motor Domain Verb Hierarchy

  6. Visuo-Motor Language Phonology jog := B0 G0 R2 Y3 B3 V0 G3 R0 Y0 jump := G0 R0 Y0 B2 G1 V0 B5 G3 R0 Y0 R10 Y5 B0 G0 R0 Y0 B0 G1 B0 G0 R0 Y1 B0 V0 B0 V0 G0 R0 Y0 B0 G0 R0 Y0 V0 run := B4 G7 R1 Y2 B1 G0 R3 Y5 V0 Y0 scuff := R0 V0 Y0V1 Y0 V0 R0 V0 Y0 V0 R0 V0 Y0 B0 V0 G0 V1 G0 V0 G0 0 V0 0 stomp := G0 R0 Y0 V0 R1 V0 Y2 V1 R0 V0 Y1V0 Y0 B0 G0 V0 B1 V0 G1 V0 B1 V0 G2 R2 Y0 B0 swagger := Y0 V0 R1 V0 Y0 V0 R0 V0 Y0 V0 0 R0 V0 Y0 V0 R0 V0 Y0 B2 G2 V0 B0 V0 G0 V0 B0 V0 G0 V0 G0 R1 tiptoe := B0 G1 V1 B0 V0 G1 0 B0 V0 G0 R0 V0 Y0V0 Y0 V0 R0 V0 Y0 B0 V0 G0 R0 V0 Y0 toe := R0 V1 R0 V0 Y0 V0 R0 Y0 B0 G1 V2 G1 R0 V0 Y0 B0 G0 troop := R0 Y0 B0 G0 R1 V3 Y3 V1 Y0 B5 G3 R0 Y0 B2 G3 R2 V0 Y2 B0 G0 walk := R0 V0 Y0V1 Y0 V0 R0 V0 Y0 B2 G2 V0 B0 V0 G0 R0 V0 Y0 0 R0 Y1 V0 R0 V0 Y1 Morphology L01 := V0 Y1L02 := R0 L01L03 := B1 G1L04 := V1 G1L05 := R1 Y1L06 := V0 L02L07 := B0 L04 L08 := L05 L03L09 := V0 L07L10 := L03 L09L11 := L01 L06L12 := L05 L06 L13 := L02 L11 L14 := L08 R2 L15 := V0 0L16 := B5 G3L17 := L16 L05L18 := L09 L08L19 := L12 L06L20 := L14 Y4 jog := B3 V0 G3 L20 jump := L08 V0 L17 R9 Y5 L03 L08 L03 L05 B0 L18 L05 L04 run := B4 G7 L20 L01 scuff := L13 L06 L07 L04 L04 0 L15 stomp := L19 L01 L10 L18 swagger := L15 L02 L06 L10 L09 L04 L19 tiptoe := L10 0 L07 L13 L07 L02 toe := R0 L06 V0 L08 L04 L02 L03 troop := L14 V3 Y3 L01 L17 B2 G3 R2 L01 L03 walk := L13 L10 L02 0 L12

  7. S := NP VP NP := N Adj VP := V Adv Human Activity Language - HAL St, j+1 Parallel (actuator) Nt, j  Nt, j+1 St, j NPt+1,j VPt,j VPt+1,j NPt,j St, j St+1, j Syntax Sequential (time) • Noun: Body parts active during the execution of a human activity • Verb: Changes each active joint experiences during the activity execution • Adjective: Specifies the initial state of the active joints (initial posture) • Adverb: String of multiplicative constants for execution time of coordinated segments

  8. We record 1000’s of actions Goal: Obtain the human Action grammar (universal grammar?)

More Related