Take a feature vector, fv=fv u fv m of SVD, let fv(t) = t*v (this is for the line search of SVD).

1. Take a feature vector, fv=fvufvm of SVD, let fv(t) = t*v (this is for the line search of SVD). So t=0 is one solution. The other two involve solving the quadratic equation, (v,m)Ratingst2fvu2fv2m = (v,m)Ratingsfvufvmru,m (v,m)Ratingspumru,m (v,m)Ratingsfvufvmru,m (v,m)Ratingsfvufvmru,m t=± t=± t2 = (v,m)Ratingsfvu2fv2m (v,m)Ratings p2um (v,m)Ratingsfvu2fv2m   To do the line search, mse(t) = 1/|Ratings| (v,m)Ratings(tfvu*tfvm - rum)2 mse(t) = 1/|Ratings| (v,m)Ratings(t2fvu*fvm - rum)2 mse(t) = 1/|Ratings| (v,m)Ratings( t4fvu2fvm2 - 2t2fvufvmru,m + rum2 ) mse/t = 1/|Ratings| (v,m)Ratings( 4t3fvu2fvm2 - 4t1fvufvmru,m ) = 0 mse/t = 4/|Ratings| (v,m)Ratings( t3fvu2fvm2 - t1fvufvmru,m ) = 0 mse/t = t * (v,m)Ratings( t2fvu2fvm2 - fvufvmru,m ) = 0

2. 1 r_________________ p______ e_____________________ e___________ 2 1 3 1 1 1 | 0 2 0 4 3 4 5 1 1 1 | 3 4 9 16 4 1 1 1 fv pr_____ pp____________________ 5 1 3 | 1 1 6 4 5 | 1 1 7.25 mse 7 1.802 =t 8 r_________________ p______ e_____________________ e___________ 9 1 3 1.802 3 3 | -2.25 -0.25 5.062 0 10 4 5 1.802 3 3 | 0.75 1.75 0.562 3.0 11 1.802 1.802 1.802 fvt pr_____ pp____________________ 12 3 9 |10.562 10.562 13 **** |10.562 10.562 2.187 mset 14 -1.80 =(-t) 15 r_________________ p______ e_____________________ e___________ 16 1 3 -1.80 3 9 | -2.25 -6.75 5.062 ** 17 4 5 -1.80 3 0 | 0.75 5 0.562 25 18 -1.80 0 -5.40 fv(-t)pr_____ pp____________________ 19 3 ** |10.562 95.062 20 **0 |10.562 0 19.04 mse(-t 61 r_________________ p______ e_____________________ e___________ 62 1 3 1.241 1 2 |-0.848 0.3950 0.720 0 63 4 5 2.433 3 4 |0.3763 0.0585 0.141 0.0 64 1.489 2.030 2.098 fv1 pr_____ pp_________________ 65 L 1 7 |3.4178 6.7856 66 0.18 **** |13.131 24.417 0.255 mse 67 1.011 =t 68 r_________________ p______ e_____________________ e___________ 69 1 3 1.255 1 2 |-0.891 0.3343 0.795 0 70 4 5 2.461 3 5 |0.2918 -0.056 0.085 0.0 71 1.506 2.054 2.122 fvt pr_____ pp_________________ 72 1 7 |3.5790 7.1057 73 **** |13.750 25.569 0.248 mset 74 -1.01 =(-t) 75 r_________________ p______ e_____________________ e___________ 76 1 3 -1.25 1 3 |-0.270 -0.811 0.073 0 77 4 5 -2.46 2 0 |1.5098 5 2.279 25 78 -1.01 0 -3.03 fv(-t)pr_____ pp_________________ 79 1 ** |1.6140 14.526 80 9 0 |6.2009 0 7.002 mse(-t 21 r_________________ p______ e_____________________ e___________ 22 1 3 1.352 2 2 |-1.071 0.6234 1.148 0 23 4 5 2.253 3 4 |0.5468 0.2265 0.299 0.0 24 1.532 2.118 1.757 fv1 pr_____ pp_________________ 25 L 2 7 |4.2926 5.6480 26 0.1 **** |11.924 22.785 0.471 mse 27 1.024 =t 28 r_________________ p______ e_____________________ e___________ 29 1 3 1.385 2 2 |-1.175 0.5047 1.381 0 30 4 5 2.309 3 5 |0.3743 -0.011 0.140 0.0 31 1.570 2.170 1.801 fvt pr_____ pp_________________ 32 2 7 |4.7323 6.2265 33 **** |13.145 25.119 0.444 mset 34 -1.02 =(-t) 35 r_________________ p______ e_____________________ e___________ 36 1 3 -1.38 1 4 |-0.419 -1.258 0.176 1 37 4 5 -2.30 2 0 |1.6339 5 2.669 25 38 -1.02 0 -3.07 fv(-t)pr_____ pp_________________ 39 1 ** |2.0153 18.138 40 9 0 |5.5982 0 7.357 mse(-t 81 r____ __________ p______ e__________________ e___________ 82 1 3 1.135 1 2 |-0.623 0.4993 0.389 0 83 4 5 2.523 3 5 |0.3918 -0.116 0.153 0.0 84 1.430 2.027 2.202 fv1 pr_____ pp____ ___________ 85 L 1 7 |2.6366 6.2532 86 0.19 **** |13.018 26.173 0.201 mse 87 1.010 =t 88 r____ __________ p______ e__________________ e___________ 89 1 3 1.147 1 2 |-0.659 0.4444 0.434 0 90 4 5 2.550 3 5 |0.3125 -0.228 0.097 0.0 91 1.445 2.049 2.226 fvt pr_____ pp____ ___________ 92 1 7 |2.7537 6.5309 93 **** |13.597 27.335 0.195 mset 94 -1.01 =(-t) 95 r____ __________ p______ e__________________ e___________ 96 1 3 -1.14 1 3 |-0.160 -0.481 0.025 0 97 4 5 -2.55 2 0 |1.4215 5 2.020 25 98 -1.01 0 -3.03 fv(-t)pr_____ pp____ ___________ 99 1 ** |1.3465 12.118 100 **0 |6.6486 0 6.819 mse(-t 41 r_________________ p______ e_____________________ e___________ 42 1 3 1.198 1 2 |-0.698 0.6745 0.487 0 43 4 5 2.421 3 5 |0.5678 -0.242 0.322 0.0 44 1.417 2.165 1.940 fv1 pr_____ pp_________________ 45 L 1 6 |2.8838 5.4079 46 0.2 **** |11.779 27.484 0.330 mse 47 1.011 =t 48 r_________________ p______ e_____________________ e___________ 49 1 3 1.211 1 2 |-0.736 0.6226 0.541 0 50 4 5 2.448 3 5 |0.4913 -0.359 0.241 0.1 51 1.433 2.189 1.962 fvt pr_____ pp_________________ 52 1 7 |3.0139 5.6518 53 **** |12.310 28.723 0.325 mset 54 -1.01 =(-t) 55 r_________________ p______ e_____________________ e___________ 56 1 3 -1.21 1 3 |-0.224 -0.674 0.050 0 57 4 5 -2.44 2 0 |1.5245 5 2.324 25 58 -1.01 0 -3.03 fv(-t)pr_____ pp_________________ 59 1 ** |1.5002 13.502 60 9 0 |6.1279 0 6.957 mse(-t 101 r____ __________ p______ e__________________ e___________ 102 1 3 1.156 1 2 |-0.682 0.2902 0.465 0 103 4 5 2.546 3 4 |0.2943 0.1205 0.086 0.0 104 1.454 1.915 2.343 fv1 pr_____ pp____ ___________ 105 L 1 8 |2.8297 7.3425 106 0.23 **** |13.731 23.809 0.162 mse 107 1.013 =t 108 r____ __________ p______ e__________________ e___________ 109 1 3 1.172 1 2 |-0.728 0.2154 0.530 0 110 4 5 2.581 3 5 |0.1920 -0.014 0.036 0.0 111 1.474 1.942 2.375 fvt pr_____ pp____ ___________ 112 1 8 |2.9881 7.7537 113 **** |14.500 25.142 0.153 mset 114 -1.01 =(-t) 115 r____ __________ p______ e__________________ e___________ 116 1 3 -1.17 1 3 |-0.188 -0.564 0.035 0 117 4 5 -2.58 2 0 |1.3827 5 1.912 25 118 -1.01 0 -3.04 fv(-t)pr_____ pp____ ___________ 119 1 ** |1.4115 12.704 120 **0 |6.8499 0 6.816 mse(-t

3. 121 r____ __________ p______ e__________________ e___________ 122 1 3 1.070 1 2 |-0.510 0.4074 0.260 0 123 4 5 2.627 3 5 |0.2936 -0.086 0.086 0.0 124 1.410 1.935 2.421 fv1 pr_____ pp____ ___________ 125 L 1 7 |2.2806 6.7212 126 0.18 **** |13.737 25.868 0.129 mse 127 1.009 =t 128 r____ __________ p______ e__________________ e___________ 129 1 3 1.080 1 2 |-0.539 0.3575 0.290 0 130 4 5 2.652 3 5 |0.2222 -0.184 0.049 0.0 131 1.423 1.954 2.444 fvt pr_____ pp____ ___________ 132 1 7 |2.3693 6.9825 133 **** |14.271 26.874 0.125 mset 134 -1.00 =(-t) 135 r____ __________ p______ e__________________ e___________ 136 1 3 -1.08 1 3 |-0.091 -0.274 0.008 0 137 4 5 -2.65 2 0 |1.3215 5 1.746 25 138 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 139 1 9 |1.1910 10.719 140 **0 |7.1739 0 6.707 mse(-t 181 r____ __________ p______ e__________________ e___________ 182 1 3 1.059 1 2 |-0.484 0.1860 0.234 0 183 4 5 2.722 3 4 |0.1875 0.0535 0.035 0.0 184 1.400 1.816 2.654 fv1 pr_____ pp____ ___________ 185 L 1 8 |2.2024 7.9185 186 0.17 **** |14.534 24.467 0.076 mse 187 1.007 =t 188 r____ __________ p______ e__________________ e___________ 189 1 3 1.068 1 2 |-0.507 0.1410 0.257 0 190 4 5 2.744 3 5 |0.1266 -0.025 0.016 0.0 191 1.411 1.831 2.676 fvt pr_____ pp____ ___________ 192 1 8 |2.2734 8.1736 193 **** |15.002 25.255 0.073 mset 194 -1.00 =(-t) 195 r____ __________ p______ e__________________ e___________ 196 1 3 -1.06 1 3 |-0.076 -0.230 0.005 0 197 4 5 -2.74 2 0 |1.2337 5 1.522 25 198 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 199 1 9 |1.1595 10.436 200 **0 |7.6522 0 6.645 mse(-t 141 r____ __________ p______ e__________________ e___________ 142 1 3 1.101 1 2 |-0.569 0.2273 0.324 0 143 4 5 2.644 3 4 |0.2305 0.0773 0.053 0.0 144 1.425 1.861 2.517 fv1 pr_____ pp____ ___________ 145 L 1 8 |2.4629 7.6873 146 0.19 **** |14.208 24.232 0.108 mse 147 1.010 =t 148 r____ __________ p______ e__________________ e___________ 149 1 3 1.112 1 2 |-0.601 0.1710 0.361 0 150 4 5 2.671 3 5 |0.1540 -0.022 0.023 0.0 151 1.439 1.880 2.543 fvt pr_____ pp____ ___________ 152 1 8 |2.5640 8.0028 153 **** |14.791 25.226 0.103 mset 154 -1.01 =(-t) 155 r____ __________ p______ e__________________ e___________ 156 1 3 -1.11 1 3 |-0.123 -0.370 0.015 0 157 4 5 -2.67 2 0 |1.3014 5 1.693 25 158 -1.01 0 -3.03 fv(-t)pr_____ pp____ ___________ 159 1 ** |1.2622 11.360 160 **0 |7.2819 0 6.711 mse(-t 201 r____ __________ p______ e__________________ e___________ 202 1 3 1.007 1 2 |-0.386 0.2771 0.149 0 203 4 5 2.768 3 5 |0.1902 -0.034 0.036 0.0 204 1.376 1.818 2.703 fv1 pr_____ pp____ ___________ 205 L 1 8 |1.9217 7.4139 206 0.18 **** |14.514 25.342 0.065 mse 207 1.007 =t 208 r____ __________ p______ e__________________ e___________ 209 1 3 1.015 1 2 |-0.408 0.2344 0.166 0 210 4 5 2.789 3 5 |0.1304 -0.113 0.017 0.0 211 1.387 1.832 2.724 fvt pr_____ pp____ ___________ 212 1 8 |1.9825 7.6484 213 **** |14.973 26.144 0.062 mset 214 -1.00 =(-t) 215 r____ __________ p______ e__________________ e___________ 216 1 3 -1.01 1 3 |-0.023 -0.069 0.000 0 217 4 5 -2.78 2 0 |1.1883 5 1.412 25 218 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 219 1 9 |1.0466 9.4200 220 **0 |7.9053 0 6.604 mse(-t 161 r____ __________ p______ e__________________ e___________ 162 1 3 1.030 1 2 |-0.433 0.3418 0.187 0 163 4 5 2.705 3 5 |0.2372 -0.055 0.056 0.0 164 1.390 1.868 2.579 fv1 pr_____ pp____ ___________ 165 L 1 7 |2.0538 7.0657 166 0.19 **** |14.158 25.558 0.090 mse 167 1.009 =t 168 r____ __________ p______ e__________________ e___________ 169 1 3 1.039 1 2 |-0.459 0.2928 0.211 0 170 4 5 2.730 3 5 |0.1679 -0.148 0.028 0.0 171 1.403 1.885 2.603 fvt pr_____ pp____ ___________ 172 1 8 |2.1302 7.3284 173 **** |14.684 26.509 0.086 mset 174 -1.00 =(-t) 175 r____ __________ p______ e__________________ e___________ 176 1 3 -1.03 1 3 |-0.049 -0.148 0.002 0 177 4 5 -2.73 2 0 |1.2445 5 1.549 25 178 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 179 1 9 |1.1013 9.9122 180 **0 |7.5922 0 6.643 mse(-t 221 r____ __________ p______ e__________________ e___________ 222 1 3 1.028 1 2 |-0.416 0.1547 0.173 0 223 4 5 2.785 3 4 |0.1622 0.0538 0.026 0.0 224 1.377 1.775 2.767 fv1 pr_____ pp____ ___________ 225 L 1 8 |2.0074 8.0956 226 0.18 **** |14.728 24.464 0.056 mse 227 1.007 =t 228 r____ __________ p______ e__________________ e___________ 229 1 3 1.035 1 2 |-0.438 0.1120 0.191 0 230 4 5 2.805 3 5 |0.1047 -0.020 0.010 0.0 231 1.388 1.789 2.787 fvt pr_____ pp____ ___________ 232 1 8 |2.0680 8.3399 233 **** |15.172 25.203 0.053 mset 234 -1.00 =(-t) 235 r____ __________ p______ e__________________ e___________ 236 1 3 -1.03 1 3 |-0.043 -0.130 0.001 0 237 4 5 -2.80 2 0 |1.1732 5 1.376 25 238 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 239 1 9 |1.0891 9.8024 240 **0 |7.9907 0 6.598 mse(-t

4. 241 r____ __________ p______ e__________________ e___________ 242 1 3 0.988 1 2 |-0.347 0.2256 0.120 0 243 4 5 2.823 3 5 |0.1527 -0.025 0.023 0.0 244 1.362 1.780 2.806 fv1 pr_____ pp____ ___________ 245 L 1 8 |1.8147 7.6972 246 0.16 **** |14.801 25.258 0.048 mse 247 1.006 =t 248 r____ __________ p______ e__________________ e___________ 249 1 3 0.994 1 2 |-0.363 0.1911 0.132 0 250 4 5 2.840 3 5 |0.1049 -0.088 0.011 0.0 251 1.371 1.791 2.823 fvt pr_____ pp____ ___________ 252 1 8 |1.8601 7.8897 253 **** |15.171 25.890 0.046 mset 254 -1.00 =(-t) 255 r____ __________ p______ e__________________ e___________ 256 1 3 -0.99 1 3 |-0.000 -0.002 0.000 0 257 4 5 -2.84 2 0 |1.1415 5 1.303 25 258 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 259 1 9 |1.0017 9.0157 260 **0 |8.1705 0 6.575 mse(-t 301 r____ __________ p______ e__________________ e___________ 302 1 3 0.981 1 2 |-0.320 0.1130 0.102 0 303 4 5 2.888 3 4 |0.1149 0.0276 0.013 0.0 304 1.345 1.721 2.940 fv1 pr_____ pp____ ___________ 305 L 1 8 |1.7447 8.3346 306 0.15 **** |15.093 24.724 0.032 mse 307 1.004 =t 308 r____ __________ p______ e__________________ e___________ 309 1 3 0.986 1 2 |-0.333 0.0848 0.111 0 310 4 5 2.902 3 5 |0.0770 -0.020 0.005 0.0 311 1.351 1.730 2.954 fvt pr_____ pp____ ___________ 312 1 8 |1.7789 8.4980 313 **** |15.389 25.208 0.031 mset 314 -1.00 =(-t) 315 r____ __________ p______ e__________________ e___________ 316 1 3 -0.98 0 2 |0.0085 0.0255 0.000 0 317 4 5 -2.90 2 0 |1.0837 5 1.174 25 318 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 319 0 8 |0.9830 8.8475 320 **0 |8.5043 0 6.543 mse(-t 261 r____ __________ p______ e__________________ e___________ 262 1 3 1.001 1 2 |-0.362 0.1391 0.131 0 263 4 5 2.838 3 4 |0.1390 0.0370 0.019 0.0 264 1.360 1.748 2.856 fv1 pr_____ pp____ ___________ 265 L 1 8 |1.8564 8.1843 266 0.17 **** |14.906 24.630 0.042 mse 267 1.006 =t 268 r____ __________ p______ e__________________ e___________ 269 1 3 1.007 1 2 |-0.379 0.1030 0.144 0 270 4 5 2.856 3 5 |0.0903 -0.025 0.008 0.0 271 1.368 1.759 2.874 fvt pr_____ pp____ ___________ 272 1 8 |1.9035 8.3921 273 **** |15.285 25.255 0.040 mset 274 -1.00 =(-t) 275 r____ __________ p______ e__________________ e___________ 276 1 3 -1.00 1 3 |-0.014 -0.042 0.000 0 277 4 5 -2.85 2 0 |1.1258 5 1.267 25 278 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 279 1 9 |1.0287 9.2589 280 **0 |8.2609 0 6.567 mse(-t 321 r____ __________ p______ e__________________ e___________ 322 1 3 0.956 1 2 |-0.277 0.1616 0.077 0 323 4 5 2.912 3 5 |0.1094 -0.012 0.011 0.0 324 1.335 1.721 2.967 fv1 pr_____ pp____ ___________ 325 L 1 8 |1.6331 8.0560 326 0.15 **** |15.136 25.121 0.028 mse 327 1.004 =t 328 r____ __________ p______ e__________________ e___________ 329 1 3 0.961 1 2 |-0.289 0.1350 0.084 0 330 4 5 2.925 3 5 |0.0728 -0.059 0.005 0.0 331 1.342 1.729 2.980 fvt pr_____ pp____ ___________ 332 1 8 |1.6639 8.2079 333 **** |15.422 25.594 0.027 mset 334 -1.00 =(-t) 335 r____ __________ p______ e__________________ e___________ 336 1 3 -0.96 0 2 |0.0343 0.1031 0.001 0 337 4 5 -2.92 2 0 |1.0603 5 1.124 25 338 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 339 0 8 |0.9323 8.3914 340 **0 |8.6415 0 6.534 mse(-t 281 r____ __________ p______ e__________________ e___________ 282 1 3 0.963 1 2 |-0.294 0.2115 0.086 0 283 4 5 2.871 3 5 |0.1403 -0.011 0.019 0.0 284 1.343 1.744 2.894 fv1 pr_____ pp____ ___________ 285 L 1 8 |1.6757 7.7757 286 0.2 **** |14.897 25.114 0.037 mse 287 1.006 =t 288 r____ __________ p______ e__________________ e___________ 289 1 3 0.969 1 2 |-0.312 0.1724 0.097 0 290 4 5 2.892 3 5 |0.0862 -0.081 0.007 0.0 291 1.353 1.757 2.915 fvt pr_____ pp____ ___________ 292 1 8 |1.7230 7.9951 293 **** |15.317 25.822 0.035 mset 294 -1.00 =(-t) 295 r____ __________ p______ e__________________ e___________ 296 1 3 -0.96 0 2 |0.0232 0.0697 0.000 0 297 4 5 -2.89 2 0 |1.0877 5 1.183 25 298 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 299 0 8 |0.9540 8.5864 300 **0 |8.4811 0 6.547 mse(-t 341 r____ __________ p______ e__________________ e___________ 342 1 3 0.963 1 2 |-0.283 0.1103 0.080 0 343 4 5 2.925 3 4 |0.1024 0.0179 0.010 0.0 344 1.332 1.703 3.000 fv1 pr_____ pp____ ___________ 345 L 1 8 |1.6466 8.3503 346 0.15 **** |15.190 24.821 0.025 mse 347 1.004 =t 348 r____ __________ p______ e__________________ e___________ 349 1 3 0.967 1 2 |-0.294 0.0846 0.086 0 350 4 5 2.938 3 5 |0.0678 -0.026 0.004 0.0 351 1.338 1.710 3.013 fvt pr_____ pp____ ___________ 352 1 8 |1.6759 8.4992 353 **** |15.461 25.263 0.024 mset 354 -1.00 =(-t) 355 r____ __________ p______ e__________________ e___________ 356 1 3 -0.96 0 2 |0.0283 0.0850 0.000 0 357 4 5 -2.93 2 0 |1.0487 5 1.099 25 358 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 359 0 8 |0.9441 8.4972 360 **0 |8.7099 0 6.526 mse(-t

5. 361 r____ __________ p______ e__________________ e___________ 362 1 3 0.927 1 2 |-0.217 0.1842 0.047 0 363 4 5 2.951 3 4 |0.1233 0.0171 0.015 0.0 364 1.313 1.688 3.037 fv1 pr_____ pp____ ___________ 365 L 1 8 |1.4825 7.9284 366 0.29 **** |15.028 24.829 0.024 mse 367 1.008 =t 368 r____ __________ p______ e__________________ e___________ 369 1 3 0.934 1 2 |-0.237 0.1375 0.056 0 370 4 5 2.975 3 5 |0.0590 -0.065 0.003 0.0 371 1.324 1.702 3.062 fvt pr_____ pp____ ___________ 372 1 8 |1.5321 8.1936 373 **** |15.531 25.659 0.020 mset 374 -1.00 =(-t) 375 r____ __________ p______ e__________________ e___________ 376 1 3 -0.93 0 2 |0.0575 0.1727 0.003 0 377 4 5 -2.97 3 0 |0.9994 5 0.998 25 378 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 379 0 8 |0.8881 7.9931 380 **0 |9.0030 0 6.508 mse(-t 421 r____ __________ p______ e__________________ e___________ 422 1 3 0.931 1 2 |-0.215 0.0926 0.046 0 423 4 5 2.997 3 4 |0.0870 0.0256 0.007 0.0 424 1.305 1.659 3.122 fv1 pr_____ pp____ ___________ 425 L 1 8 |1.4770 8.4529 426 0.2 **** |15.311 24.744 0.015 mse 427 1.004 =t 428 r____ __________ p______ e__________________ e___________ 429 1 3 0.935 1 2 |-0.226 0.0649 0.051 0 430 4 5 3.011 3 5 |0.0498 -0.021 0.002 0.0 431 1.311 1.667 3.137 fvt pr_____ pp____ ___________ 432 1 8 |1.5053 8.6145 433 **** |15.604 25.217 0.014 mset 434 -1.00 =(-t) 435 r____ __________ p______ e__________________ e___________ 436 1 3 -0.93 0 2 |0.0600 0.1802 0.003 0 437 4 5 -3.01 3 0 |0.9738 5 0.948 25 438 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 439 0 8 |0.8834 7.9509 440 **0 |9.1576 0 6.496 mse(-t 381 r____ __________ p______ e__________________ e___________ 382 1 3 0.945 1 2 |-0.247 0.0928 0.061 0 383 4 5 2.972 3 5 |0.0773 -0.001 0.005 0.0 384 1.319 1.682 3.075 fv1 pr_____ pp____ ___________ 385 L 1 8 |1.5561 8.4516 386 0.1 **** |15.387 25.018 0.018 mse 387 1.002 =t 388 r____ __________ p______ e__________________ e___________ 389 1 3 0.947 1 2 |-0.253 0.0781 0.064 0 390 4 5 2.980 3 5 |0.0574 -0.027 0.003 0.0 391 1.322 1.686 3.083 fvt pr_____ pp____ ___________ 392 1 8 |1.5719 8.5373 393 **** |15.543 25.272 0.018 mset 394 -1.00 =(-t) 395 r____ __________ p______ e__________________ e___________ 396 1 3 -0.94 0 2 |0.0499 0.1497 0.002 0 397 4 5 -2.98 2 0 |1.0123 5 1.024 25 398 -1.00 0 -3.00 fv(-t)pr_____ pp____ ___________ 399 0 8 |0.9026 8.1241 400 **0 |8.9258 0 6.512 mse(-t 441 r____ __________ p______ e__________________ e___________ 442 1 3 0.916 1 2 |-0.190 0.1126 0.036 0 443 4 5 3.017 3 4 |0.0797 0.0080 0.006 0.0 444 1.299 1.654 3.149 fv1 pr_____ pp____ ___________ 445 L 1 8 |1.4182 8.3365 446 0.2 **** |15.368 24.919 0.013 mse 447 1.004 =t 448 r____ __________ p______ e__________________ e___________ 449 1 3 0.920 1 2 |-0.201 0.0866 0.040 0 450 4 5 3.031 3 5 |0.0444 -0.036 0.001 0.0 451 1.304 1.661 3.163 fvt pr_____ pp____ ___________ 452 1 8 |1.4438 8.4873 453 **** |15.646 25.370 0.012 mset 454 -1.00 =(-t) 455 r____ __________ p______ e__________________ e___________ 456 1 3 -0.92 0 2 |0.0750 0.2251 0.005 0 457 4 5 -3.03 3 0 |0.9551 5 0.912 25 458 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 459 0 8 |0.8555 7.7000 460 **0 |9.2713 0 6.492 mse(-t 401 r____ __________ p______ e__________________ e___________ 402 1 3 0.933 1 2 |-0.225 0.1116 0.050 0 403 4 5 2.984 3 4 |0.0824 0.0014 0.006 0.0 404 1.312 1.674 3.094 fv1 pr_____ pp____ ___________ 405 L 1 8 |1.5012 8.3426 406 0.15 **** |15.346 24.985 0.017 mse 407 1.003 =t 408 r____ __________ p______ e__________________ e___________ 409 1 3 0.936 1 2 |-0.234 0.0899 0.054 0 410 4 5 2.995 3 5 |0.0530 -0.036 0.002 0.0 411 1.317 1.681 3.105 fvt pr_____ pp____ ___________ 412 1 8 |1.5238 8.4684 413 **** |15.578 25.362 0.016 mset 414 -1.00 =(-t) 415 r____ __________ p______ e__________________ e___________ 416 1 3 -0.93 0 2 |0.0595 0.1785 0.003 0 417 4 5 -2.99 3 0 |0.9929 5 0.985 25 418 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 419 0 8 |0.8845 7.9605 420 **0 |9.0422 0 6.505 mse(-t 461 r____ __________ p______ e__________________ ee__________ 462 1 3 0.923 1 2 |-0.195 0.0648 0.038 0 463 4 5 3.030 3 4 |0.0762 0.0320 0.005 0.0 464 1.294 1.639 3.179 fv1 pr_____ pp____ ___________ 465 L 1 8 |1.4283 8.6149 466 0.2 **** |15.396 24.680 0.012 mse 467 1.004 =t 468 r____ __________ p______ e__________________ ee__________ 469 1 3 0.926 1 2 |-0.205 0.0405 0.042 0 470 4 5 3.043 3 5 |0.0436 -0.009 0.001 0.0 471 1.300 1.646 3.192 fvt pr_____ pp____ ___________ 472 1 8 |1.4520 8.7583 473 **** |15.652 25.091 0.011 mset 474 -1.00 =(-t) 475 r____ __________ p______ e__________________ ee__________ 476 1 3 -0.92 0 2 |0.0692 0.2078 0.004 0 477 4 5 -3.04 3 0 |0.9442 5 0.891 25 478 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 479 0 8 |0.8662 7.7960 480 **0 |9.3373 0 6.484 mse(-t

6. 481 r____ __________ p______ e__________________ ee__________ 482 1 3 0.909 1 2 |-0.175 0.0930 0.030 0 483 4 5 3.048 3 5 |0.0592 -0.007 0.003 0.0 484 1.292 1.642 3.197 fv1 pr_____ pp____ ___________ 485 L 1 8 |1.3808 8.4505 486 0.13 **** |15.529 25.070 0.010 mse 487 1.002 =t 488 r____ __________ p______ e__________________ ee__________ 489 1 3 0.911 1 2 |-0.181 0.0778 0.032 0 490 4 5 3.056 3 5 |0.0387 -0.033 0.001 0.0 491 1.296 1.646 3.206 fvt pr_____ pp____ ___________ 492 1 8 |1.3953 8.5388 493 **** |15.691 25.332 0.010 mset 494 -1.00 =(-t) 495 r____ __________ p______ e__________________ ee__________ 496 1 3 -0.91 0 2 |0.0862 0.2586 0.007 0 497 4 5 -3.05 3 0 |0.9355 5 0.875 25 498 -1.00 0 -3.00 fv(-t)pr_____ pp____ ___________ 499 0 8 |0.8350 7.5152 500 **0 |9.3907 0 6.487 mse(-t 541 r____ __________ p______ e__________________ ee__________ 542 1 3 0.903 1 2 |-0.145 0.0454 0.021 0 543 4 5 3.080 3 4 |0.0914 0.0807 0.008 0.0 544 1.268 1.596 3.271 fv1 pr_____ pp____ ___________ 545 L 1 8 |1.3128 8.7295 546 0.43 **** |15.277 24.199 0.009 mse 547 1.007 =t 548 r____ __________ p______ e__________________ ee__________ 549 1 3 0.909 1 2 |-0.162 0.0023 0.026 0 550 4 5 3.102 3 4 |0.0344 0.0090 0.001 0.0 551 1.278 1.608 3.295 fvt pr_____ pp____ ___________ 552 1 8 |1.3513 8.9857 553 **** |15.725 24.909 0.006 mset 554 -1.00 =(-t) 555 r____ __________ p______ e__________________ ee__________ 556 1 3 -0.90 0 2 |0.0837 0.2513 0.007 0 557 4 5 -3.10 3 0 |0.8745 5 0.764 25 558 -1.00 0 -3.02 fv(-t)pr_____ pp____ ___________ 559 0 8 |0.8394 7.5548 560 **0 |9.7682 0 6.458 mse(-t 501 r____ __________ p______ e__________________ ee__________ 502 1 3 0.915 1 2 |-0.175 0.0497 0.030 0 503 4 5 3.055 3 4 |0.0759 0.0460 0.005 0.0 504 1.284 1.621 3.223 fv1 pr_____ pp____ ___________ 505 L 1 8 |1.3813 8.7036 506 0.25 **** |15.398 24.541 0.010 mse 507 1.004 =t 508 r____ __________ p______ e__________________ ee__________ 509 1 3 0.919 1 2 |-0.186 0.0222 0.034 0 510 4 5 3.069 3 5 |0.0392 -0.000 0.001 0.0 511 1.290 1.628 3.238 fvt pr_____ pp____ ___________ 512 1 8 |1.4072 8.8669 513 **** |15.687 25.001 0.009 mset 514 -1.00 =(-t) 515 r____ __________ p______ e__________________ ee__________ 516 1 3 -0.91 0 2 |0.0763 0.2290 0.005 0 517 4 5 -3.06 3 0 |0.9161 5 0.839 25 518 -1.00 0 -3.01 fv(-t)pr_____ pp____ ___________ 519 0 8 |0.8531 7.6782 520 **0 |9.5104 0 6.474 mse(-t 561 r____ __________ p______ e__________________ ee__________ 562 1 3 0.890 1 2 |-0.134 0.0647 0.018 0 563 4 5 3.108 3 5 |0.0394 -0.008 0.001 0.0 564 1.274 1.611 3.295 fv1 pr_____ pp____ ___________ 565 L 1 8 |1.2876 8.6156 566 0.095 **** |15.686 25.082 0.005 mse 567 1.001 =t 568 r____ __________ p______ e__________________ ee__________ 569 1 3 0.891 1 2 |-0.138 0.0559 0.019 0 570 4 5 3.113 3 5 |0.0275 -0.023 0.000 0.0 571 1.276 1.613 3.300 fvt pr_____ pp____ ___________ 572 1 8 |1.2954 8.6675 573 **** |15.780 25.232 0.005 mset 574 -1.00 =(-t) 575 r____ __________ p______ e__________________ ee__________ 576 1 3 -0.89 0 2 |0.1067 0.3201 0.011 0 577 4 5 -3.11 3 0 |0.8821 5 0.778 25 578 -1.00 0 -3.00 fv(-t)pr_____ pp____ ___________ 579 0 8 |0.7979 7.1817 580 **0 |9.7207 0 6.473 mse(-t 521 r____ __________ p______ e__________________ ee__________ 522 1 3 0.902 1 2 |-0.159 0.0749 0.025 0 523 4 5 3.074 3 5 |0.0483 -0.008 0.002 0.0 524 1.285 1.628 3.240 fv1 pr_____ pp____ ___________ 525 L 1 8 |1.3455 8.5561 526 0.1 **** |15.615 25.082 0.008 mse 527 1.001 =t 528 r____ __________ p______ e__________________ ee__________ 529 1 3 0.904 1 2 |-0.164 0.0643 0.026 0 530 4 5 3.080 3 5 |0.0340 -0.026 0.001 0.0 531 1.287 1.631 3.246 fvt pr_____ pp____ ___________ 532 1 8 |1.3553 8.6183 533 **** |15.728 25.264 0.008 mset 534 -1.00 =(-t) 535 r____ __________ p______ e__________________ ee__________ 536 1 3 -0.90 0 2 |0.0941 0.2825 0.008 0 537 4 5 -3.08 3 0 |0.9142 5 0.835 25 538 -1.00 0 -3.00 fv(-t)pr_____ pp____ ___________ 539 0 8 |0.8205 7.3846 540 **0 |9.5221 0 6.481 mse(-t 581 r_______________ p______ e_____________________ ee__________ 582 1 3 0.894 1 2 |-0.131 0.0343 0.017 0 583 4 5 3.112 3 4 |0.0635 0.0454 0.004 0.0 584 1.264 1.591 3.315 fv1 pr_____ pp____________________ 585 L 1 8 |1.2797 8.7952 586 0.3 **** |15.495 24.547 0.006 mse 587 1.004 =t 588 r_______________ p______ e_____________________ ee__________ 589 1 3 0.898 1 2 |-0.140 0.0089 0.019 0 590 4 5 3.125 3 4 |0.0298 0.0031 0.000 0.0 591 1.270 1.598 3.329 fvt pr_____ pp____________________ 592 1 8 |1.3017 8.9463 593 **** |15.762 24.968 0.005 mset 594 -1.00 =(-t) 595 r_______________ p______ e_____________________ ee__________ 596 1 3 -0.89 0 2 |0.0979 0.2937 0.009 0 597 4 5 -3.12 3 0 |0.8609 5 0.741 25 598 -1.00 0 -3.01 fv(-t)pr_____ pp____________________ 599 0 8 |0.8137 7.3240 600 **0 |9.8535 0 6.459 mse(-t

7. 601 r_______________ p______ e_____________________ ee__________ 602 1 3 0.883 1 2 |-0.119 0.0579 0.014 0 603 4 5 3.129 3 5 |0.0348 -0.006 0.001 0.0 604 1.266 1.599 3.330 fv1 pr_____ pp____________________ 605 L 1 8 |1.2522 8.6557 606 0.1 **** |15.722 25.067 0.004 mse 607 1.001 =t 608 r_______________ p______ e_____________________ ee__________ 609 1 3 0.884 1 2 |-0.122 0.0497 0.014 0 610 4 5 3.134 3 5 |0.0237 -0.020 0.000 0.0 611 1.268 1.601 3.335 fvt pr_____ pp____________________ 612 1 8 |1.2592 8.7041 613 **** |15.810 25.207 0.004 mset 614 -1.00 =(-t) 615 r_______________ p______ e_____________________ ee__________ 616 1 3 -0.88 0 2 |0.1141 0.3425 0.013 0 617 4 5 -3.13 3 0 |0.8612 5 0.741 25 618 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 619 0 7 |0.7846 7.0618 620 **0 |9.8515 0 6.468 mse(-t 661 r_______________ p______ e_____________________ ee__________ 662 1 3 0.881 1 2 |-0.108 0.0318 0.011 0 663 4 5 3.150 3 4 |0.0361 0.0108 0.001 0.0 664 1.258 1.583 3.368 fv1 pr_____ pp____________________ 665 L 1 8 |1.2288 8.8099 666 0.15 **** |15.712 24.892 0.003 mse 667 1.001 =t 668 r_______________ p______ e_____________________ ee__________ 669 1 3 0.882 1 2 |-0.112 0.0218 0.012 0 670 4 5 3.155 3 5 |0.0227 -0.006 0.000 0.0 671 1.260 1.586 3.374 fvt pr_____ pp____________________ 672 1 8 |1.2371 8.8696 673 **** |15.818 25.060 0.003 mset 674 -1.00 =(-t) 675 r_______________ p______ e_____________________ ee__________ 676 1 3 -0.88 0 2 |0.1159 0.3477 0.013 0 677 4 5 -3.15 3 0 |0.8386 5 0.703 25 678 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 679 0 7 |0.7815 7.0343 680 **0 |9.9939 0 6.459 mse(-t 621 r_______________ p______ e_____________________ ee__________ 622 1 3 0.887 1 2 |-0.118 0.0309 0.013 0 623 4 5 3.133 3 4 |0.0510 0.0314 0.002 0.0 624 1.260 1.585 3.346 fv1 pr_____ pp____________________ 625 L 1 8 |1.2503 8.8154 626 0.25 **** |15.594 24.686 0.004 mse 627 1.003 =t 628 r_______________ p______ e_____________________ ee__________ 629 1 3 0.890 1 2 |-0.125 0.0122 0.015 0 630 4 5 3.143 3 4 |0.0261 0.0001 0.000 0.0 631 1.264 1.590 3.356 fvt pr_____ pp____________________ 632 1 8 |1.2661 8.9269 633 **** |15.791 24.998 0.004 mset 634 -1.00 =(-t) 635 r_______________ p______ e_____________________ ee__________ 636 1 3 -0.89 0 2 |0.1071 0.3213 0.011 0 637 4 5 -3.14 3 0 |0.8466 5 0.716 25 638 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 639 0 8 |0.7972 7.1751 640 **0 |9.9434 0 6.457 mse(-t 681 r_______________ p______ e_____________________ ee__________ 682 1 3 0.870 1 2 |-0.092 0.0600 0.008 0 683 4 5 3.159 3 5 |0.0339 -0.000 0.001 0.0 684 1.255 1.582 3.377 fv1 pr_____ pp____________________ 685 L 1 8 |1.1937 8.6433 686 0.18 **** |15.729 25.005 0.003 mse 687 1.002 =t 688 r_______________ p______ e_____________________ ee__________ 689 1 3 0.872 1 2 |-0.097 0.0479 0.009 0 690 4 5 3.165 3 5 |0.0177 -0.021 0.000 0.0 691 1.257 1.586 3.384 fvt pr_____ pp____________________ 692 1 8 |1.2035 8.7143 693 **** |15.858 25.211 0.003 mset 694 -1.00 =(-t) 695 r_______________ p______ e_____________________ ee__________ 696 1 3 -0.87 0 2 |0.1260 0.3782 0.015 0 697 4 5 -3.16 3 0 |0.8276 5 0.685 25 698 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 699 0 7 |0.7637 6.8738 700 **0 |10.063 0 6.460 mse(-t To get the number of the round, divide the line number in the bottom left by 20. So in 35 rounds we drove all errors below .1 and the mse to .003 (avg indiv err ~.055 Note that each round involved a binary type search for the best LRATE and then a one-time calculation of t (to accomplishing the line search with one formula). I think I'm doing the line search incorrectly and the "best LRATE" search is doing a correct line search. I am searching the line through the origin generated by nfv while I think I should be searching the line through fv generated by the gradient (I believe that's what LRATE does. There should be a closed form formula for it, obviating the need for a looping search (neither a binary or fixed increment search). Next slide continues on to see how far it can go ;-) 641 r_______________ p______ e_____________________ ee__________ 642 1 3 0.877 1 2 |-0.106 0.0544 0.011 0 643 4 5 3.147 3 5 |0.0316 -0.005 0.001 0.0 644 1.260 1.590 3.357 fv1 pr_____ pp____________________ 645 L 1 8 |1.2234 8.6760 646 0.11 **** |15.747 25.058 0.003 mse 647 1.001 =t 648 r_______________ p______ e_____________________ ee__________ 649 1 3 0.878 1 2 |-0.109 0.0463 0.011 0 650 4 5 3.151 3 5 |0.0207 -0.019 0.000 0.0 651 1.262 1.592 3.362 fvt pr_____ pp____________________ 652 1 8 |1.2301 8.7239 653 **** |15.834 25.196 0.003 mset 654 -1.00 =(-t) 655 r_______________ p______ e_____________________ ee__________ 656 1 3 -0.87 0 2 |0.1204 0.3612 0.014 0 657 4 5 -3.15 3 0 |0.8442 5 0.712 25 658 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 659 0 7 |0.7736 6.9631 660 **0 |9.9588 0 6.464 mse(-t

8. 701 r_______________ p______ e_____________________ ee___________ 702 1 3 0.876 1 2 |-0.099 0.0306 0.0099 0 703 4 5 3.164 3 4 |0.0281 0.0018 0.0007 0. 704 1.255 1.579 3.388 fv1 pr_____ pp____________________ 705 L 1 8 |1.2091 8.8169 706 0.1 **** |15.775 24.981 0.0029 mse 707 1.001 =t 708 r_______________ p______ e_____________________ ee___________ 709 1 3 0.877 1 2 |-0.101 0.0246 0.0103 0 710 4 5 3.167 3 5 |0.0201 -0.008 0.0004 0. 711 1.256 1.580 3.392 fvt pr_____ pp____________________ 712 1 8 |1.2140 8.8525 713 **** |15.839 25.082 0.0028 mset 714 -1.00 =(-t) 715 r_______________ p______ e_____________________ ee___________ 716 1 3 -0.87 0 2 |0.1220 0.3661 0.0148 0 717 4 5 -3.16 3 0 |0.8288 5 0.6870 25 718 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 719 0 7 |0.7707 6.9371 720 **0 |10.056 0 6.4589 mse(-t 761 r_______________ p______ e_____________________ ee___________ 762 1 3 0.860 1 2 |-0.071 0.0520 0.0051 0 763 4 5 3.189 3 5 |0.0274 -0.000 0.0007 0. 764 1.245 1.567 3.425 fv1 pr_____ pp____________________ 765 L 1 8 |1.1489 8.6906 766 0.2 **** |15.780 25.001 0.0021 mse 767 1.001 =t 768 r_______________ p______ e_____________________ ee___________ 769 1 3 0.862 1 2 |-0.075 0.0412 0.0057 0 770 4 5 3.195 3 5 |0.0129 -0.018 0.0001 0. 771 1.247 1.570 3.431 fvt pr_____ pp____________________ 772 1 8 |1.1573 8.7542 773 **** |15.896 25.184 0.0019 mset 774 -1.00 =(-t) 775 r_______________ p______ e_____________________ ee___________ 776 1 3 -0.86 0 2 |0.1362 0.4087 0.0185 0 777 4 5 -3.19 3 0 |0.7987 5 0.6380 25 778 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 779 0 7 |0.7460 6.7146 780 **0 |10.247 0 6.4559 mse(-t 721 r_______________ p______ e_____________________ ee___________ 722 1 3 0.859 1 2 |-0.070 0.0772 0.0050 0 723 4 5 3.172 3 4 |0.0463 0.0172 0.0021 0. 724 1.246 1.570 3.401 fv1 pr_____ pp____________________ 725 L 1 8 |1.1467 8.5424 726 0.4 **** |15.631 24.828 0.0033 mse 727 1.004 =t 728 r_______________ p______ e_____________________ ee___________ 729 1 3 0.862 1 2 |-0.079 0.0528 0.0063 0 730 4 5 3.186 3 5 |0.0133 -0.024 0.0001 0. 731 1.251 1.576 3.415 fvt pr_____ pp____________________ 732 1 8 |1.1659 8.6857 733 **** |15.893 25.244 0.0024 mset 734 -1.00 =(-t) 735 r_______________ p______ e_____________________ ee___________ 736 1 3 -0.86 0 2 |0.1334 0.4003 0.0178 0 737 4 5 -3.18 3 0 |0.8006 5 0.6410 25 738 -1.00 0 -3.01 fv(-t)pr_____ pp____________________ 739 0 7 |0.7509 6.7582 740 **0 |10.235 0 6.4547 mse(-t 781 r_______________ p______ e_____________________ ee___________ 782 1 3 0.866 1 2 |-0.079 0.0220 0.0063 0 783 4 5 3.194 3 4 |0.0221 0.0023 0.0004 0. 784 1.245 1.564 3.435 fv1 pr_____ pp____________________ 785 L 1 8 |1.1654 8.8682 786 0.1 **** |15.823 24.976 0.0018 mse 787 1.000 =t 788 r_______________ p______ e_____________________ ee___________ 789 1 3 0.867 1 2 |-0.081 0.0173 0.0066 0 790 4 5 3.196 3 5 |0.0159 -0.005 0.0002 0. 791 1.246 1.565 3.437 fvt pr_____ pp____________________ 792 1 8 |1.1691 8.8961 793 **** |15.872 25.054 0.0017 mset 794 -1.00 =(-t) 795 r_______________ p______ e_____________________ ee___________ 796 1 3 -0.86 0 2 |0.1317 0.3952 0.0173 0 797 4 5 -3.19 3 0 |0.8008 5 0.6413 25 798 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 799 0 7 |0.7538 6.7846 800 **0 |10.234 0 6.4537 mse(-t 801 r_______________ p______ e_____________________ ee___________ 802 1 3 0.861 1 2 |-0.070 0.0368 0.0050 0 803 4 5 3.198 3 4 |0.0232 0.0002 0.0005 0. 804 1.243 1.563 3.440 fv1 pr_____ pp____________________ 805 L 1 8 |1.1469 8.7801 806 0.15 **** |15.814 24.997 0.0017 mse 807 1.001 =t 808 r_______________ p______ e_____________________ ee___________ 809 1 3 0.862 1 2 |-0.073 0.0294 0.0054 0 810 4 5 3.202 3 5 |0.0132 -0.012 0.0001 0. 811 1.244 1.565 3.444 fvt pr_____ pp____________________ 812 1 8 |1.1527 8.8241 813 **** |15.894 25.122 0.0016 mset 814 -1.00 =(-t) 815 r_______________ p______ e_____________________ ee___________ 816 1 3 -0.86 0 2 |0.1365 0.4095 0.0186 0 817 4 5 -3.20 3 0 |0.7936 5 0.6299 25 818 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 819 0 7 |0.7455 6.7103 820 **0 |10.280 0 6.4540 mse(-t 741 r_______________ p______ e_____________________ ee___________ 742 1 3 0.871 1 2 |-0.087 0.0213 0.0076 0 743 4 5 3.183 3 4 |0.0244 0.0037 0.0005 0. 744 1.248 1.569 3.419 fv1 pr_____ pp____________________ 745 L 1 8 |1.1828 8.8724 746 0.1 **** |15.804 24.962 0.0021 mse 747 1.000 =t 748 r_______________ p______ e_____________________ ee___________ 749 1 3 0.871 1 2 |-0.089 0.0163 0.0079 0 750 4 5 3.186 3 5 |0.0178 -0.004 0.0003 0. 751 1.249 1.570 3.422 fvt pr_____ pp____________________ 752 1 8 |1.1867 8.9019 753 **** |15.857 25.045 0.0021 mset 754 -1.00 =(-t) 755 r_______________ p______ e_____________________ ee___________ 756 1 3 -0.87 0 2 |0.1275 0.3826 0.0162 0 757 4 5 -3.18 3 0 |0.8108 5 0.6574 25 758 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 759 0 7 |0.7611 6.8507 760 **0 |10.170 0 6.4550 mse(-t

9. 821 r_______________ p______ e_____________________ ee__________ 822 1 3 0.863 1 2 |-0.073 0.0238 0.00533 * 823 4 5 3.202 3 4 |0.0203 0.0007 0.00041 0. 824 1.242 1.561 3.447 fv1 pr_____ pp____________________ 825 L 1 8 |1.1514 8.8572 826 0.1 **** |15.837 24.992 0.00158 mse 827 1.000 =t 828 r_______________ p______ e_____________________ ee__________ 829 1 3 0.864 1 2 |-0.074 0.0193 0.00557 * 830 4 5 3.204 3 5 |0.0142 -0.006 0.00020 0. 831 1.243 1.562 3.449 fvt pr_____ pp____________________ 832 1 8 |1.1549 8.8841 833 **** |15.885 25.068 0.00155 mset 834 -1.00 =(-t) 835 r_______________ p______ e_____________________ ee__________ 836 1 3 -0.86 0 2 |0.1353 0.4059 0.01830 * 837 4 5 -3.20 3 0 |0.7930 5 0.62896 25 838 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 839 0 7 |0.7476 6.7291 840 **0 |10.284 0 6.45301 mse( 881 r_______________ p______ e_____________________ ee__________ 882 1 3 0.840 1 2 |-0.025 0.0106 0.00063 * 883 4 5 3.273 3 5 |0.0064 -0.000 0.00004 0. 884 1.220 1.527 3.557 fv1 pr_____ pp____________________ 885 L 1 8 |1.0510 8.9361 886 0.072 **** |15.948 25.002 0.00019 mse 887 1.000 =t 888 r_______________ p______ e_____________________ ee__________ 889 1 3 0.840 1 2 |-0.025 0.0088 0.00066 * 890 4 5 3.274 3 5 |0.0041 -0.003 0.00001 0. 891 1.220 1.528 3.558 fvt pr_____ pp____________________ 892 1 8 |1.0523 8.9468 893 **** |15.967 25.032 0.00019 mset 894 -1.00 =(-t) 895 r_______________ p______ e_____________________ ee__________ 896 1 3 -0.84 0 2 |0.1592 0.4776 0.02534 * 897 4 5 -3.27 3 0 |0.7248 5 0.52547 25 898 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 899 0 7 |0.7069 6.3623 900 **0 |10.726 0 6.44474 mse( 841 r_______________ p______ e_____________________ ee__________ 842 1 3 0.861 1 2 |-0.069 0.0269 0.00487 * 843 4 5 3.205 3 5 |0.0194 -0.000 0.00037 0. 844 1.241 1.560 3.451 fv1 pr_____ pp____________________ 845 L 1 8 |1.1445 8.8392 846 0.1 **** |15.845 25.009 0.00149 mse 847 1.000 =t 848 r_______________ p______ e_____________________ ee__________ 849 1 3 0.862 1 2 |-0.071 0.0223 0.00510 * 850 4 5 3.207 3 5 |0.0133 -0.008 0.00017 0. 851 1.242 1.561 3.453 fvt pr_____ pp____________________ 852 1 8 |1.1480 8.8663 853 **** |15.893 25.085 0.00146 mset 854 -1.00 =(-t) 855 r_______________ p______ e_____________________ ee__________ 856 1 3 -0.86 0 2 |0.1372 0.4117 0.01883 * 857 4 5 -3.20 3 0 |0.7898 5 0.62391 25 858 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 859 0 7 |0.7443 6.6990 860 **0 |10.304 0 6.45307 mse( 901 r_______________ p______ e_____________________ ee__________ 902 1 3 0.840 1 2 |-0.023 0.0062 0.00053 * 903 4 5 3.274 3 4 |0.0149 0.0108 0.00022 0. 904 1.217 1.523 3.561 fv1 pr_____ pp____________________ 905 L 1 8 |1.0466 8.9622 906 0.4 **** |15.880 24.892 0.00022 mse 907 1.001 =t 908 r_______________ p______ e_____________________ ee__________ 909 1 3 0.841 1 3 |-0.025 -0.000 0.00063 * 910 4 5 3.277 3 4 |0.0064 0.0001 0.00004 0. 911 1.218 1.525 3.565 fvt pr_____ pp____________________ 912 1 9 |1.0511 9.0006 913 **** |15.948 24.998 0.00016 mset 914 -1.00 =(-t) 915 r_______________ p______ e_____________________ ee__________ 916 1 3 -0.84 0 2 |0.1576 0.4729 0.02485 * 917 4 5 -3.27 3 0 |0.7188 5 0.51677 25 918 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 919 0 7 |0.7095 6.3860 920 **0 |10.765 0 6.44132 mse( . . . 981 r_______________ p______ e_____________________ ee__________ 982 1 3 0.838 1 2 |-0.020 0.0047 0.00040 * 983 4 5 3.282 3 4 |0.0054 0.0008 0.00002 0. 984 1.216 1.522 3.572 fv1 pr_____ pp____________________ 985 L 1 8 |1.0408 8.9712 986 0.1 **** |15.956 24.991 0.00011 mse 987 1.000 =t 988 r_______________ p______ e_____________________ ee__________ 989 1 3 0.838 1 2 |-0.020 0.0036 0.00042 * 990 4 5 3.283 3 5 |0.0039 -0.001 0.00001 0. 991 1.217 1.523 3.573 fvt pr_____ pp____________________ 992 1 8 |1.0416 8.9782 993 **** |15.968 25.010 0.00011 mset 994 -1.00 =(-t) 995 r_______________ p______ e_____________________ ee__________ 996 1 3 -0.83 0 2 |0.1612 0.4838 0.02601 * 997 4 5 -3.28 3 0 |0.7161 5 0.51283 25 998 -1.00 0 -3.00 fv(-t)pr_____ pp____________________ 999 0 7 |0.7034 6.3310 1000 **0 |10.783 0 6.44323 mse( 861 r_______________ p______ e_____________________ ee__________ 862 1 3 0.835 1 2 |-0.001 0.0776 0.00000 * 863 4 5 3.214 3 4 |0.1443 0.1839 0.02082 0. 864 1.199 1.498 3.498 fv1 pr_____ pp____________________ 865 L 1 8 |1.0037 8.5403 866 2.3 **** |14.866 23.194 0.01517 mse 867 1.017 =t 868 r_______________ p______ e_____________________ ee__________ 869 1 3 0.849 1 3 |-0.036 -0.024 0.00136 * 870 4 5 3.270 3 4 |0.0092 0.0152 0.00008 0. 871 1.220 1.524 3.559 fvt pr_____ pp____________________ 872 1 9 |1.0753 9.1490 873 **** |15.925 24.847 0.00057 mset 874 -1.01 =(-t) 875 r_______________ p______ e_____________________ ee__________ 876 1 3 -0.84 0 2 |0.1353 0.4061 0.01832 * 877 4 5 -3.27 3 0 |0.6725 5 0.45233 25 878 -1.01 0 -3.05 fv(-t)pr_____ pp____________________ 879 0 7 |0.7475 6.7282 880 **0 |11.071 0 6.40889 mse(

10. After 60 rounds, the mse is.000025and all individual errors show2 zero places to the right of the decimal. The resulting feature vector: ufv1 ufv2 ufv3 mfv1 mfv2 1.21168 1.51538 3.59832 0.83349 3.29954 There are only two unrated pairs, (u2,m1) and (u3,m2) and this vector predicts them as: 1181r_____ ____________ ___ e_____________________ ee__________ 1182 1 3 0.83313 * * |-0.009 0.0033 0.000082 1183 4 5 3.29813 ** |0.0054 0.0042 0.000029 0. 11841.21117 1.51473 3.59678 fv1 r__ pp____________________ 1185 L * * |1.0182 8.9797 1186 0.4 ** |15.956 24.957 0.000035 mse 11871.00042 =t 1188r_____ ____________ ___ e_____________________ ee__________ 11891 3 0.83349 * * |-0.009 0.0008 0.000098 11904 5 3.29954 ** | 0.002 -0.000 0.000003 0. 1191 1.21168 1.51538 3.59832 fvt r__ pp____________________ 1192 * * |1.0199 8.9951 1193 ** |15.984 25.000 0.000025 mse 1194-1.0004 =(-t) 1195r_____ ____________ ___ e_____________________ ee__________ 1196 1 3 -0.8334 * * |0.1661 0.4984 0.027605 1197 4 5 -3.2995 ** |0.6990 5 0.488661 25 1198-1.0004 0 -3.0012 fv(-t) )r__ pp____________________ 1199 * * |0.6953 6.2577 1200 ** |10.896 0 6.441178 mse p(u2,m1)=1.51538*.83349= 1.26306 p(u3,m2)=3.59832*3.29813=11.87283 The last one must be truncated into the [0,5] range at 5. As a final note, we can say we have mined almost every last bit of information from this training set (down to a mse=.000025) so that the fv "models" the training set nearly perfectly. Let's see if we can convince ourselves of that. u1 u2 u3 mfv m1 ->11.263063 0.83349 m2 ->4 511.87283 3.29954 ufv-> 1.21168 1.51538 3.59832 What does training row m2 tell us about p(u2,m1)? It tells us that it should be ~25% higher than r(u1,m1)=1 or ~1.25 What does training column u1 tell us about p(u2,m1)? It tells us that it should be 1/4th of r(u2,m2)=5 or ~1.25 What does training column u1 tell us about p(u3,m2)? It tells us that it should be ~4 times r(u3,m1)=3 or ~12 What does training row m1 tell us about p(u3,m2)? It tells us that it should be ~3 times r(u1,m2)=4 or ~12 I am searching the line through the origin generated by nfv while I think I should be searching the line through fv generated by the gradient (I believe that's what LRATE does. There should be a closed form formula for it, obviating the need for a looping search (neither a binary or fixed increment search). The next slide investigates that.

11. nfv ≡ fv + GR = ( fvu+GRu, fvm+GRm ), GR=Gradient of mse of p(u,m)=nfvunfvm fvt ≡ ( fvu+tGRu, fvm+tGRm ) umRtngs 2( (fvu+tGRu)(fvm+tGRm) - rum) mse/t( (fvu+tGRu)(fvm+tGRm) - 0 ) umRtg 2( fvufvm-rum+t(GRmfvu+GRufvm)+t2GRuGRm) mse/t( t2GRuGRm+t(GRmfvu+GRufvm)+fvufvm) umRtg ( fvufvm-rum+t(GRmfvu+GRufvm)+t2GRuGRm) (2tGRuGRm+ GRmfvu+GRufvm) = 0 g=GRmGRu h=GRmfvu+GRufvm p=fvufvm-rum umRtg umRtg (2tpg+2t2hg+2t3g2+ph+th2+t2gh) = (p+th+t2g)(2tg+h) = t3umRtg t2umRtg 2g2 + + tumRtg 3hg (2pg+h2) + umRtg ph = 0 b a d c GR= ( nSUPPu1 ER(u1,n)*mfvn, ..., nSUPPulastu ER(ulastu,n)*mfvn, vSUPPm1 ER(v,m1)*ufvv, ..., vSUPPlastm1 ER(v,mlastm)*ufvv ) mse(t) = 1/|Ratings|(u,m)Ratings (fvtu fvtm - rum)2 0=mse/t= 1/|Rtgs|(u,m)Ratings 2(fvtu fvtm - rum)1 mse/t( fvtu fvtm - rum ) 1 r________ p________ e________ ee_______ mse g________ h____ 2 1 3 1 1 1 0 2 0 4 7.25 6 4 2 5 3 4 5 1 1 1 3 4 9 16 t 21 7 7 ** 8 4 1 1 1 fv 1 ***a **b ***c **d -0 p 0.6195 3 1 2 GR 5 0. -0 0. -0 -0 0. q 0 r 1 6 7 r________ p________ e________ ee______ 8 1 3 2. 6. 5. -5 -2 29 4. 43.287 mset 9 4 5 5. 15 8. ***-3 ***13 10 2. 1. 2. fvt 21 r________ p________ e________ ee_______ mse g________ h______ 22 1 3 ****** 66 *** *** *** *** 3E+06 *** ****** ** 23 4 5 ********* ****** ****** t ****** *** ******* 24 ******-3 fv 1 ***a **b ***c **d -0 p 0.4825 *********GR 25 0. -0 0. -0 -0 0. q 0 r 1 26 27 r________ p________ e________ ee______ 28 1 3 ****** *** *** *** *** *** 3E+19 mset 29 4 5 ********* ****** ****** 30 *********fvt

12. y=ax3 + bx2 + cx + d with one real solutions and two complex y=ax3 + bx2 + cx + d with three real solutions y=x3

17. WOLFRAM

25. Line Search Details Delta mse -0.05 5.571428571 4.787528571 3.956937381 3.124580148 2.370216629 1.827544757 1.712786894 0.001 1.694534417 1.683969759 1.683990218 0.001 1.683990218 1.683817009 1.683713544 1.683680240 a b c d e f g h i j k l m n ... LRATE MSE a b c d e f g h i j k l m n ... LRATE MSE a b c d e f g h i j k l m n ... LRATE MSE Line Search Details Delta mse -0.1 2.742857142 1.757028571 1.726552903 -0.001 1.725127451 1.722583169 1.720124307 1.717750375 1.715460883 1.713255343 0.0001 1.712786894 0.0001 1.683990218 0.0001 1.683680240 0.0001 2.742857142 0.0001 1.726552903 0.0001 1.725127451 0.0001 486.9428571 0.0001 1.684954260 0.0001 1.683737218 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 ... 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 1.78 ... 5 5 5 5 5 5 5 5 5 5 5 5 5 5 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 1.72 ... Line Search Details Delta mse -0.1 486.9428571 300.2339285 182.0785776 108.0757974 62.37291917 34.69604087 18.40735293 9.235135416 4.444223928 2.294761042 1.689520299 1.684954260 0.001 1.683737218 1.683737131 Using the new dataset with 20 movies and 51 users. Initial: 1 1 1 1 .... 1 Line Search Details Delta mse -0.1 5.8 2.848214285 1.780423107 1.744331269 -0.001 1.741238800 1.738234884 1.735319015 1.732490689 1.729749404 1.727094662 5 5 5 5 5 5 5 0.0001 5.8 4.00 4.00 4.00 4.00 4.00 4.00 4.00 0.0001 1.744331269 3.98 3.98 3.98 3.98 3.98 3.98 3.98 0.0001 1.727094662 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Note: 3.0625 = 1.75*1.75 3.1442 = 0.79*3.98

26. a b c d e f g h i j k l m n o p q r s t 1 2 3 4 5 6 7 8 LRATEMSE .52 1.47 .52 1 1.47 1 1.31 .68 1 .52 1.47 1 1.15 1.63 .52 1 .84 1.15 1.63 1.15 3.04 3.02 3.06 3.07 2.95 3.00 3.05 3.04 .05250.4470620 We came up with an approximately minimized mse at LRATE=,030. Going from this line search resulting from LRATE=.03, we do another round round: .76 1.47 .51 .99 1.47 .99 1.40 .67 .99 0.51 1.47 .99 1.19 1.63 .51 0.99 .79 1.15 1.63 1.16 3.06 3.03 3.07 3.08 2.93 3.00 3.04 3.03 .0300.368960 Going from this line search resulting from LRATE=.02, we the same for the next round: .75 1.47 .50 .99 1.47 .98 1.44 .66 .99 0.51 1.47 .99 1.21 1.63 .50 0.98 .76 1.15 1.63 1.17 3.06 3.03 3.07 3.08 2.92 2.99 3.04 3.03 .0200.351217 Here is the result after 1 round when using a fixed increment line search to find minimize mse with respect to the LRATE used: Without line search, using Funk's LRATE=.001, to arrive at ~ same mse (and a nearly identical feature vector) it takes 81 rounds: .76 1.38 .61 .99 1.38 .99 1.34 .74 0.99 .61 1.38 .99 1.16 1.50 .61 .99 .82 1.12 1.50 1.13 3.04 3.01 3.04 3.06 2.92 2.98 3.02 3 .001 0.44721854 Going from the round 1 result (LRATE=.0525) shown here, we do a second round and again do fixed increment line search: .52 1.47 .52 1 1.47 1 1.31 .68 1 .52 1.47 1 1.15 1.63 .52 1 .84 1.15 1.63 1.15 3.04 3.02 3.06 3.07 2.95 3.00 3.05 3.04 .05250.447062 .92 1.48 .50 .99 1.47 .98 1.46 .66 .98 0.50 1.47 .98 1.22 1.63 .50 0.98 .75 1.15 1.63 1.17 3.07 3.03 3.07 3.09 2.92 2.99 3.04 3.03 .0500.387166 .84 1.47 .50 .99 1.47 .99 1.43 .66 .99 0.50 1.47 .99 1.21 1.63 .50 0.98 .77 1.15 1.63 1.17 3.06 3.03 3.07 3.08 2.93 3.00 3.04 3.03 .0400.371007 .76 1.47 .51 .99 1.47 .99 1.40 .67 .99 0.51 1.47 .99 1.19 1.63 .51 0.99 .79 1.15 1.63 1.16 3.06 3.03 3.07 3.08 2.93 3.00 3.04 3.03 .0300.368960 .76 1.47 .51 .99 1.47 .99 1.40 .67 .99 0.51 1.47 .99 1.19 1.63 .51 0.99 .79 1.15 1.63 1.16 3.06 3.03 3.07 3.08 2.93 3.00 3.04 3.03 .0200.380975 .75 1.47 .50 .99 1.47 .98 1.44 .66 .99 0.51 1.47 .99 1.21 1.63 .50 0.98 .76 1.15 1.63 1.17 3.06 3.03 3.07 3.08 2.92 2.99 3.04 3.03 .0200.351217 .75 1.47 .50 .99 1.47 .99 1.42 .66 .99 0.51 1.47 .99 1.20 1.63 .50 0.99 .77 1.15 1.63 1.17 3.06 3.03 3.07 3.08 2.92 3.00 3.04 3.03 .0100.362428 .74 1.47 .50 .99 1.47 .98 1.46 .66 .98 0.50 1.47 .98 1.22 1.63 .50 0.98 .75 1.15 1.63 1.17 3.07 3.04 3.07 3.09 2.91 2.99 3.04 3.02 .0100.351899 LRATE=.02 stable, near-optimal? (No further line search). After 200 rounds at LRATE=.02. (note that it took ~2000 rounds without line search and with line search ~219): .83 1.39 .48 .91 1.40 .86 1.52 .61 .98 0.60 1.51 .98 1.15 1.74 .48 0.99 .64 1.10 1.62 1.45 3.28 3.28 3.04 3.48 1.69 2.98 3.12 2.65 .0200.199358 Comparing this feature vector to the one we got with ~2000 rounds at LRATE=.001 (without line search) we see that we arrive at a very different feature vector: a b c d e f g h i j k l m n o p q r s t 1 2 3 4 5 6 7 8 LRATE 1.48 2.54 .90 1.6 2.6 1.55 2.68 1.15 1.73 1.08 2.67 1.73 2.06 3.08 .90 1.76 1.16 2.07 2.90 2.75 1.86 1.74 1.71 1.94 .87 1.61 1.7 1.5 .001, no ls .83 1.39 .48 .91 1.40 .86 1.52 .61 .98 .60 1.51 .98 1.15 1.74 .48 0.99 .64 1.10 1.62 1.45 3.28 3.28 3.04 3.48 1.69 2.98 3.12 2.65 .020, w ls However, the UserFeatureVector protions differ by constant multiplier and the MovieFeatureVector portions differ by a different constant. If we divide the LR=.001 vector by the LR=.020, we get the following multiplier vector (one is not a dialation of the other but if we split user portion from the movie portion, they are!!! What does that mean!?!?!?! 1.77 1.81 1.85 1.75 1.84 1.79 1.76 1.86 1.75 1.78 1.76 1.75 1.79 1.76 1.85 1.76 1.81 1.86 1.78 1.89 .56 .53 .56 .55 .51 .54 .54 .56".001/.020" 1.80 avg 0.04 std 0.54 avg 0.01 std Another interesting observation is that 1 / 1.8 = .55, that is, 1 / AVGufv = AVGmfv. They are reciporicals of oneanother!!! This makes sense since it means, if you double the ufv you have to halve the mfv to get the same predictions. The bottom line is that the predictions are the same! What is the nature of the set of vectors that [nearly] minimize the mse? It is not a subspace (not closed under scalar multiplication) but it is clearly closed under "reciporical scalar multiplication" (multiplying the mfv's by the reciporical of the ufv's multiplier). Waht else can we say about it? So, we get an order of magnitude speedup fromline search. It may be more than that since we may be able to do all the LRATE calculations in parallel (without recalculating the error matrix or feature vectors????). Or we there may be a better search mechanism than fixed increment search. A binary type search? Othere?

27. A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AAAB AC AD 1 \a=Z2 2 3 3 5 2 5 3 3 /rvnfv~fv~{goto}L~{edit}+.005~/XImse<omse-.00001~/xg\a~ 3 2 5 1 2 3 5 3 .001~{goto}se~/rvfv~{end}{down}{down}~ 4 3 3 3 5 5 2 /xg\a~ 5 5 3 4 3 6 2 1 2 1 7 4 1 1 4 3 8 4 3 2 5 3 9 1 4 5 3 2 LRATE omse 10 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3 1 3 3 2 0.001 0.1952090 fv A22: +A2-A$10*$U2 /* error for u=a, m=1 */ A30: +A10+$L*(A$22*$U$2+A$24*$U$4+A$26*$U$6+A$29*$U$9) /* updates f(u=a) */ U29: +U9+$L*(($A29*$A$30+$K29*$K$30+$N29*$N$30+$P29*$P$30)/4)/* updates f(m=8 */ AB30: +U29 /* copies f(m=8) feature update in the new feature vector, nfv */ W22: @COUNT(A22..T22) /* counts the number of actual ratings (users) for m=1 */ X22: [W3] @SUM(W22..W29) /*adds ratings counts for all 8 movies = training count*/ AD30: [W9] @SUM(SE)/X22 /* averages se's giving the mse */ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AAAB AC AD 21 working error and new feature vector (nfv) 22 0 0 0 **0 ** 3 6 35 23 0 0 ** 0 ** 0 3 6 24 0 0 0 ** 0 2 5 25 0 ** ** 3 3 26 0 0 **1 3 27 **** ** 0 3 4 28 ** 1 0 ** 3 4 29 ** ** 0 0 2 4 L mse 30 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.1952063 nfv A52: +A22^2 /*squares all the individual erros */ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AAAB AC AD 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 square errors 53 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 58 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 59 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 SE 60 --------------------------------------------------------------- 61 0 1 0 0 1 0 1 0 0 0 1 0 1 1 0 0 0 1 1 1 3 3 3 3. 2 2 3 2 0.125 0.225073 62 0 1 0 0 1 0 1 0 0 0 1 0 1 1 0 1 0 1 1 1 3 3 3 3. 1 2 3 2 0.141 0.200424 63 0 1 0 0 1 0 1 0 0 0 1 0 1 1 0 1 0 1 1 1 3 3 3 3. 1 3 3 2 0.151 0.197564 64 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.151 0.196165 65 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.151 0.195222 66 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195232 67 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195228 68 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195224 69 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195221 70 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195218 71 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195214 72 0 1 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 1 1 1 3 3 2 3. 1 3 3 2 0.001 0.195211 {goto}se~/rvfv~{end}{down}{down}~ "value copy" fv to output list Notes: In 2 rounds mse is as low as Funk gets it in 2000 rounds. After 5 rounds mse is lower than ever before (and appears to be bottoming out). I know I shouldn't hardcode parameters! Experiments should be done to optimize this line search (e.g., with some binary search for a low mse). Since we have the resulting individual square_errors for each training pair, we could run this, then for mask the pairs with se(u,m) > Threshold. Then do it again after masking out those that have already achieved a low se. But what do I do with the two resulting feature vectors? Do I treat it like a two feature SVD or do I use some linear combo of the resulting predictions of the two (or it could be more than two)? We need to test out which works best (or other modifications) on Netflix data. Maybe on those test pairs for which the training row and column have some high errors, we apply the second feature vector instead of the first? Maybe we invoke CkNN for test pairs in this case (or use all 3 and a linear combo?) This is powerful! We need to optimize the calculations using pTrees!!! /rvnfv~fvcopies fv to nfv after converting fv to values. {goto}L~{edit}+.005~increments L by .005 /XImse<omse-.00001~/xg\a~IF mse still decreasing, recalc mse with new L .001~ Reset L=.001 for next round /xg\a~ Start over with next round

28. A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AAABACADAEAFAGAHAIAJ 1 3 4 3 1 3 3 3 2 3 2 2 5 3 3 3 3 5 1 4 4 4 1 1 3 4 5 5 2 3 3 2 6 3 1 3 4 1 7 3 2 1 3 1 1 3 4 8 4 5 5 9 2 2 2 3 3 1 10 3 1 1 2 3 2 11 4 4 2 3 5 3 3 12 3 13 5 3 3 5 3 3 3 1 14 2 3 3 2 5 15 3 3 2 16 1 1 5 3 1 5 1 3 17 3 2 3 1 2 2 18 2 3 3 1 3 3 19 1 3 4 2 4 3 3 1 4 1 20 1 2 5 3 1 4 4 2 3 AKALAMANAOAPAQARASATAUAVAWAXAY BA 5 2 5 3 1 1 2 3 5 1 3 3 5 5 1 3 4 1 1 2 1 1 4 1 3 4 2 5 1 3 5 1 1 5 1 5 5 5 2 1 1 3 3 2 5 1 4 4 1 1 3 1 5 2 1 4 4 3 5 1 5 1 3 2 5 1 4 1 4 1 5 2 4 5 3 4 1 O P Q R S T U V W X Y Z AA AB 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.89 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.88 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.88 0.66 0.97 0.72 0.89 0.97 0.97 0.97 0.56 0.72 0.53 0.80 1.62 0.97 0.88 0.66 0.99 0.98 0.99 0.99 0.99 0.99 0.97 0.98 0.98 0.99 1.01 0.99 0.99 0.99 0.99 0.77 0.92 0.99 0.99 0.99 0.61 0.77 0.62 0.84 1.45 0.99 0.92 0.85 BB BC BD BE BF BG BH BI BJ BK BL BM BN BO 1.68 1.68 1.69 1.67 1.68 1.68 1.68 1.68 1.68 1.67 1.68 1.67 1.70 1.68 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.69 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 2.81 2.80 2.83 2.80 2.81 2.77 2.81 2.80 2.81 2.80 2.81 2.80 2.83 2.82 2.84 2.84 2.84 2.84 2.84 2.83 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 2.84 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.01 3.01 3.02 3.01 3.01 3.01 3.01 3.01 3.01 3.01 3.01 3.01 3.02 3.01 AC AD AE AF AG AH AI AJ AK AL AM 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.97 0.64 0.56 0.97 0.97 0.86 0.97 1.13 1.07 1.59 1.13 0.99 0.99 0.97 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.01 0.99 0.76 0.61 0.99 0.99 0.90 0.99 1.14 1.08 1.28 1.15 BP BQ BR BS BT Lrate MSE 3.09 3.09 3.09 3.09 3.09 0.0079 1.252787373 3.09 3.09 3.09 3.09 3.09 0.0001 1.252778817 3.09 3.09 3.09 3.09 3.09 0.0001 1.252777738 3.09 3.09 3.09 3.09 3.09 0.0001 1.252777438 3.09 3.09 3.09 3.09 3.09 0.0001 1.252777289 3.09 3.09 3.09 3.09 3.09 0.0001 1.252777139 3.09 3.09 3.09 3.09 3.09 0.0001 1.252776991 3.09 3.09 3.09 3.09 3.09 0.0001 1.252776843 3.09 3.09 3.09 3.09 3.09 0.0001 1.252776695 3.09 3.09 3.09 3.09 3.09 0.0001 1.252776548 3.00 3.00 3.00 3.00 0.0005 1.749577428 3.01 3.02 3.01 3.01 0.0035 1.278489789 A B C D E F G H I J K L M N 102 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 103 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 104 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 105 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 106 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 107 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 108 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 109 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 110 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 111 0.76 0.97 0.97 0.75 0.72 1.29 0.88 0.86 0.97 1.18 0.86 0.72 0.97 1.29 0.97 0.99 0.99 0.99 0.98 1.00 0.99 0.99 0.99 1.00 0.99 0.98 0.99 1.01 0.78 0.99 0.99 0.81 0.77 1.22 0.92 0.90 0.99 1.18 0.90 0.77 0.99 1.27 AN AO AP AQ AR AS AT AU AV AW AX AY AZ BA 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.97 0.48 0.97 1.29 0.80 1.07 0.80 1.29 0.97 0.89 1.53 1.42 3.09 0.99 0.98 0.99 1.00 0.99 1.00 0.99 1.01 0.99 0.99 1.03 1.03 3.00 3.00 0.99 0.65 0.99 1.22 0.84 1.08 0.84 1.29 0.99 0.92 1.52 1.43 3.02 3.01 A larger example: 20 movies, 51 users (same as last time except I found errors in my code, which I corrected. 2 2 2 1 0 1 2 1 2 1 2 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 21 1 1 1 1 1 2 1 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 2 The last two red lines are printouts of the two steps in the initial line search (on the way to the first result line at MSE=1.252787373). The two vectors should be co-linear (generate the same line) or else I am not doing line search!! They are clearly not co-linear. Thus I have a more code mistake. This is why a C# versions is desparately needed!! How is that coming?

29. Where are we now wrt PSVD? Clearly line search is a good idea. How good? (speedup?, accuracy comparisons?) What about 2nd [3rd?, 4th?, ...] feature vector training? How to generate those? (Probably just a matter of understanding Funk's code). What "retraining under mask" steps are breakthroughs? improve accuracy markedly? improve speed markedly? What speedup shortcuts can we [as mindless engineers ;-) ] come up with. By "mindless" I mean only that trial and error is probably the best way to find these speedups, unless you can understand the mathematics). Maybe Dr. Ubhaya? What speedup shortcuts can we come up with to execute Md's PTreeSet Algebra Procedures? These speedups can be "mindless" or "magic" - we'll take them anyway!. Again, by "mindless" I mean that trial and error is used to find lucky speedups - unless you can fully understand the mathematics, it's mindless ;-) Maybe Dr. Ubhaya can do the math for us? I will suggest the following: "The more the Mathematics is understood the better the mindless engineering tricks work!" What speedup shortcuts can we come up with? Involving Md's PTreeSet Algebra? These speedups can be "mindless" or "magic", we'll take them anyway!. By "mindless" I mean that trial and error is used to find lucky speedups - unless you can fully understand the mathematics, it's mindless ;-) Maybe Dr. Ubhaya can do the math for us? I will suggest the following: "The more the Mathematics is understood the better the mindless engineering tricks work!" In RECOMMENDERs, we have people (users, customers, websearchers...) and things (products, movies, items, documents, webpages or?) We also often have text (product descriptions, movie features, item descriptions, document contents, webpage contents...), which can be handled as entity description columns or by introducing a third entity, terms (content terms, stems of content terms, ...). So we have three entities and three relationships in a cyclic 2 hop rolodex structure (or what we called BUP "Bi-partite, Uni-partite on Part" structure). A lifetime of fruitful research lurks in this arena. We can use one relationship to restrict (mask entities instances in) an adjacent relationship. I firmly believe pTree structuring is the way to do this. We can add a people-to-people relationship also (ala, facebook friends) and richen the information content significantly. We should add tweats to this somehow. Since I don't tweat, I'm probably not the one to suggest how this should fit in, but I will anyway ;-) Tweats (seem to be) mini-documents describing documents or mini-documents describing people, or possibly even mini-documents describing terms (e.g, if a buzzword becomes hot in the media, people tweat about it????) Let's call this research arena the VERTICAL RECOMMENDER arena. It's hot! Who's going to be the Master Chef in this Hell's Kitchen?