school/csci5521
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

1b progress

Browse source
This commit is contained in:
Michael Zhang 2023-11-10 00:42:46 -06:00
parent 15481ebb04
commit 9eee507d73
1 changed files with 24 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
assignments/hwk03/hw3_sol.typ
View file

@ -38,4 +38,28 @@ Solved as:
&= - sum_t (r^t - y^t) v_h x_h (1-tanh^2(ww_2 dot xx)) \ &= - sum_t (r^t - y^t) v_h x_h (1-tanh^2(ww_2 dot xx)) \
$ $
Updates:
- $Delta v_h = eta sum_t (r^t-y^t) z^t_h$
- $Delta w_(1,j) = eta sum_t (r^t - y^t) v_h x_h cases(0 "if" ww_1 dot xx <0, 1 "otherwise")$
- $Delta w_(2,j) = eta sum_t (r^t - y^t) v_h x_h (1-tanh^2(ww_2 dot xx))$
= Problem 1b = Problem 1b
- $E(ww,vv|XX) = - sum_t r^t log y^t + (1 - r^t) log (1 - y^t)$
- $y^t = "sigmoid"(v_2 z_2 + v_1 z_1 + v_0)$
- $z^t_1 = "ReLU"(w_2 x^t_2 + w_1 x^t_1 + w_0)$
- $z^t_2 = tanh(w_2 x^t_2 + w_1 x^t_1 + w_0)$
Updates:
-
Same as above:
$Delta v_h = eta sum_t (r^t-y^t) z^t_h$
- $
frac(diff E, diff w_j) &= - sum_t (frac(diff E, diff y^t) frac(diff y^t, diff z^t_1) frac(diff z^t_1, diff w_j)) + (frac(diff E, diff y^t) frac(diff y^t, diff z^t_2) frac(diff z^t_2, diff w_j)) \
&= - sum_t frac(diff E, diff y^t) (frac(diff y^t, diff z^t_1) frac(diff z^t_1, diff w_j) + frac(diff y^t, diff z^t_2) frac(diff z^t_2, diff w_j)) \
&= - sum_t (frac(r^t, y^t) - frac(1-r^t, 1-y^t)) (frac(diff y^t, diff z^t_1) frac(diff z^t_1, diff w_j) + frac(diff y^t, diff z^t_2) frac(diff z^t_2, diff w_j)) \
&= - sum_t (frac(r^t-y^t, y^t (1-y^t))) (frac(diff y^t, diff z^t_1) frac(diff z^t_1, diff w_j) + frac(diff y^t, diff z^t_2) frac(diff z^t_2, diff w_j)) \
$