/
Neural Network Activations and Perceptrons
Save to my account
Sign up
Neural Network Activations and Perceptrons
Neural Network Activations and Perceptrons
Study
1
Question
What is activation function in neural networks?
Answer
Nonlinear function applying to neurons output
2
Question
Why are activation functions needed in neural networks?
Answer
Introduce nonlinearity to model complex patterns
3
Question
What is the Sigmoid activation function definition?
Answer
σ(x) = 1/(1+e^{-x})
4
Question
State a key limitation of the sigmoid function.
Answer
Vanishes gradients for extreme inputs
5
Question
What is the ReLU function definition?
Answer
f(x) = max(0, x)
6
Question
What is a major advantage of ReLU?
Answer
Computational simplicity and sparse activation
7
Question
What is a downside of ReLU?
Answer
Dying ReLU problem (dead neurons)
8
Question
What is the tanh function definition?
Answer
tanh(x) = (e^x - e^{-x})/(e^x + e^{-x})
9
Question
How does tanh compare to sigmoid in output range?
Answer
Output in [-1, 1] vs [0, 1] for sigmoid
10
Question
What is the SoftMax function used for in neural nets?
Answer
Convert vector to probability distribution over classes
11
Question
What is the Swish activation function?
Answer
f(x) = x * sigmoid(x)
12
Question
Why is Swish considered smooth and non-monotonic?
Answer
Smooth derivative; can improve training performance
13
Question
What is the purpose of activation functions in perceptrons?
Answer
Introduce nonlinearity for complex decision boundaries
14
Question
Define a perceptron.
Answer
A simple linear classifier with a step activation
15
Question
What is the perceptron learning rule?
Answer
Adjust weights based on misclassified examples
16
Question
What does the convergence theorem for perceptrons guarantee?
Answer
If data linearly separable, algorithm converges
17
Question
What is the role of non-linear activation in multi-layer nets?
Answer
Enable learning of complex mappings across layers
18
Question
What is a multi-layer perceptron (MLP)?
Answer
Feedforward network with multiple hidden layers
19
Question
What is backpropagation in neural networks?
Answer
Efficiently computing gradients via chain rule for all weights
20
Question
What is a common issue in backpropagation called?
Answer
Vanishing/exploding gradients
21
Question
What is a residual network (ResNet)?
Answer
NN with skip connections to mitigate gradient issues
22
Question
What does backpropagation through time (BPTT) involve?
Answer
Unfolding recurrent nets in time and applying backpropagation
23
Question
What is a common activation used in CNNs?
Answer
ReLU and variants
24
Question
What is the purpose of pooling layers in CNNs?
Answer
Downsample feature maps and reduce computation
25
Question
What is the difference between a convolution and a fully connected layer?
Answer
Convolution uses local receptive fields and weight sharing
26
Question
What is a key property of the sigmoid function in classification?
Answer
Probabilistic interpretation as class probability
27
Question
What problem does vanishing gradient cause in deep networks?
Answer
Poor weight updates in early layers during training
28
Question
What is the advantage of Leaky ReLU over ReLU?
Answer
Small gradient for negative inputs prevents dead neurons
29
Question
What is a backpropagated error signal?
Answer
δ error term propagated to adjust weights
30
Question
What does the term 'gradient explosion' refer to?
Answer
Rapid growth of gradients causing unstable training