Skip to content

Commit bae54c3

Browse files
Update AdaGard.md for equation rendering
1 parent 331aaaa commit bae54c3

File tree

1 file changed

+9
-10
lines changed
  • docs/Deep Learning/Optimizers in Deep Learning

1 file changed

+9
-10
lines changed

docs/Deep Learning/Optimizers in Deep Learning/AdaGard.md

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,21 +30,20 @@ The update rule for AdaGrad is as follows:
3030

3131
1. Accumulate the squared gradients:
3232

33-
$
33+
$$
3434
G_t = G_{t-1} + g_t^2
35-
$
35+
$$
3636

3737
2. Update the parameters:
3838

39-
$
40-
\theta_t = \theta_{t-1} - \frac{\eta}{\sqrt{G_t} + \epsilon} \cdot g_t
41-
$
39+
40+
$$η = \theta_{t-1} - \frac{\eta}{\sqrt{G_t} + \epsilon} \cdot g_t$$
4241

4342
where:
44-
- $ G_t $ is the accumulated sum of squares of gradients up to time step $ t $
45-
- $ g_t $ is the gradient at time step $ t $
46-
- $ \eta $ is the learning rate
47-
- $ \epsilon $ is a small constant to prevent division by zero
43+
- $G_t$ is the accumulated sum of squares of gradients up to time step $t$
44+
- $g_t$ is the gradient at time step $t$
45+
- $\eta$ is the learning rate
46+
- $\epsilon$ is a small constant to prevent division by zero
4847

4948
## Implementation in Keras
5049

@@ -107,4 +106,4 @@ The results of the training process, including the loss and accuracy, will be di
107106

108107
## What Next
109108

110-
To address these issues, various optimization algorithms have been developed, such as Adam, which incorporate techniques. Which we'll see in next section .
109+
To address these issues, various optimization algorithms have been developed, such as Adam, which incorporate techniques. Which we'll see in next section .

0 commit comments

Comments
 (0)