Day-19 of Machine Learning:

I. Basic template of TensorFlow implementation:

#### 1. construct the network

```
model = Sequential(
[
tf.keras.Input(shape=(400,)), #specify input size
Dense(25, activation='sigmoid'),
Dense(15, activation='sigmoid'),
Dense(1, activation='sigmoid')
], name = "my_model"
)
```

Keras Sequential model and Dense Layer with sigmoid activations.

#### 2. loss function

```
model.compile(
loss=tf.keras.losses.BinaryCrossentropy(),
optimizer=tf.keras.optimizers.Adam(0.001),
)
```

Here for **binary classification**, BinaryCrossentropy() is used. We can also use MeanSquareError() for Linear regression.

#### 3. gradient descent to fit the weights of the model to the training data

```
model.fit(
X,y,
epochs=20
)
```

II. Got to know about different Activation

##### - Linear Activation:

Activation **a = g(Z) = Z**

where Z = W.X + b

**Output y** might be an Integer number **(+ve/-ve)**

##### - Sigmoid Activation:

Activation **a = g(Z) = 1 / (1 + e ^ (-Z))**.

**Output y** might be **0 or 1 i.e binary classification**

##### - ReLU Activation (Rectified Linear Activation):

Activation **a = g(Z) = max (0, Z)**.

**Output y** will be any **Whole number**

III. How to choose Activation?

We can choose different activation within a Neural Network for separate layers and activations can be chosen accordingly requirement and goal of the Neural Network. However some recommendations are,

- A neural network with many layers but no activation function is not effective. A Neural network with
**only linear activation**is the same as**no activation function**. - ReLU are often use than Sigmoid activation. It is because firstly
**ReLU is a bit faster as it does less computation (max of 0 and Z)**than sigmoid which does exponential then inverse and so on. Secondly Gradient Descent goes slow for flat and ReLU goes flat in one place whereas Sigmoid in 2 places. - Use
**ReLU**instead of Linear Activation in**Hidden layers**.

## Top comments (0)