25 AI interview questions, with the answers

Bias measures how far a model's average predictions are from the true values; variance measures how much predictions fluctuate across different training sets. A model with too few parameters (too simple) has high bias and low variance, it underfits. A model with too many parameters has low bias but high variance, it overfits. The goal is to find the sweet spot that minimizes total error (bias² + variance + irreducible noise) through tuning model complexity, regularization, or ensembling.

Gradient descent is an iterative optimization algorithm that minimizes a loss function by computing its gradient with respect to the model's parameters and updating those parameters in the opposite direction (steepest descent). The step size is controlled by the learning rate. It's used when parameters can't be solved analytically (e.g., deep networks). Variants include batch GD, stochastic GD (one sample per step), and mini-batch GD (standard in practice).

Underfitting: model is too simple to capture the underlying pattern, high training and test error. Overfitting: model memorizes training data and fails to generalize, low training error, high test error. Combat underfitting: increase model capacity, add features, reduce regularization. Combat overfitting: add L1/L2 regularization, dropout, early stopping, data augmentation, gather more training data, use cross-validation to tune complexity.

Regularization adds a penalty term to the loss function to discourage overly complex models. It reduces variance at the cost of a slight increase in bias. Common methods: L2 (Ridge), penalizes sum of squared weights, shrinks all coefficients toward zero but never to exactly zero, good for multicollinear features. L1 (Lasso), penalizes sum of absolute weights, can zero out coefficients producing sparse models useful for feature selection. Elastic Net combines both. Dropout is a regularization technique specific to neural networks.

PCA is an unsupervised dimensionality reduction technique. It finds orthogonal directions (principal components) of maximum variance in the data by computing the eigenvectors of the covariance matrix. The data is then projected onto the top-k eigenvectors, retaining the most variance with fewer dimensions. Steps: (1) Standardize data. (2) Compute covariance matrix. (3) Eigen-decompose to get eigenvectors/values. (4) Sort by eigenvalue descending. (5) Project data onto top-k eigenvectors. Useful for visualization, noise reduction, and reducing compute before downstream models.

Both add a penalty on the weights to the loss to reduce overfitting. L2 (ridge) penalizes the sum of squared weights, shrinking them smoothly toward zero without eliminating them, and handles correlated features by sharing weight among them. L1 (LASSO) penalizes the sum of absolute weights, which tends to drive some weights exactly to zero, producing sparse models and performing feature selection. Elastic Net combines both to get sparsity plus stability under correlation.

Logistic regression is a linear model for classification. It computes a linear combination of features and passes it through the sigmoid function to produce a probability between 0 and 1, then thresholds it for a class. It is trained by minimizing cross-entropy (log loss), usually via gradient descent, and the coefficients are log-odds contributions of each feature. It is a discriminative model, interpretable, and extends to multiclass with softmax.

Bagging trains base learners independently and in parallel on bootstrap samples and averages them, mainly reducing variance. Boosting trains learners sequentially, each focusing on the errors of the previous ones and combining them into a weighted sum, mainly reducing bias and building a strong learner from weak ones. Bagging is robust and parallelizable and resists overfitting, boosting is more accurate but more sensitive to noise and outliers and harder to parallelize.

A neural network is a model of layered units, each computing a weighted sum of its inputs plus a bias followed by a nonlinear activation. Stacking layers lets it learn hierarchical, nonlinear feature representations that map inputs to outputs. It is trained by forward-propagating to compute a loss and backpropagating gradients to update weights via gradient descent. With enough capacity it is a universal function approximator, powering vision, language, and more.

Three main reasons: (1) Vanishing gradient: Sigmoid saturates near 0 or 1, producing near-zero gradients that kill learning in deep networks. ReLU has gradient exactly 1 for positive inputs, so gradients flow freely. (2) Computational efficiency: ReLU is just max(0, x), a simple threshold, making forward and backward passes much faster than computing exp(). (3) Sparsity: ReLU outputs exactly 0 for negative inputs, producing sparse activations that improve efficiency and generalization. Downside of ReLU: 'dying ReLU' (neurons stuck at 0), addressed by Leaky ReLU or ELU.

Dropout randomly zeroes a fraction of unit activations during each training step, so the network cannot rely on any single unit and must learn redundant, robust features. It acts like training an ensemble of many thinned subnetworks that share weights, which reduces co-adaptation and overfitting. At test time no units are dropped and activations are scaled so the expected value matches training. The drop probability is a hyperparameter, commonly around 0.5 for hidden layers.

Batch normalization normalizes the activations of a layer across the current mini-batch to zero mean and unit variance, then rescales and shifts them with learned parameters gamma and beta. It stabilizes and speeds training by reducing internal covariate shift, allows higher learning rates, and adds a mild regularizing effect from batch noise. At inference it uses running averages of mean and variance collected during training instead of batch statistics.

In deep networks gradients are products of many factors during backpropagation, so they can shrink toward zero (vanishing) or grow without bound (exploding), especially with saturating activations or many layers and recurrent steps. Mitigations include ReLU-family activations, careful initialization such as Xavier or He, batch or layer normalization, residual connections that give gradients a shortcut, gradient clipping for explosions, and gated architectures like LSTM or GRU for sequences.

No. If all weights start at zero (or any identical value), every neuron in a layer computes the same output and receives the same gradient, so they update identically and remain identical forever. This symmetry means the layer never learns diverse features and effectively behaves like a single neuron. You must break symmetry with small random initialization, such as Xavier or He, while biases can be initialized to zero.

Pooling downsamples feature maps by summarizing local regions, most commonly max pooling which takes the maximum in each window and average pooling which takes the mean, with global pooling reducing a whole map to one value. It reduces spatial resolution and computation, enlarges the receptive field, and provides small translation invariance and robustness to minor shifts. Modern networks sometimes replace pooling with strided convolutions to learn the downsampling.

Self-attention lets each token in a sequence attend to every other token in the same sequence to build a contextual representation. For each token you compute Query, Key, and Value vectors, score the Query against all Keys, scale by the square root of the key dimension, softmax the scores, and take a weighted sum of the Values. The result is that each position's output is a mixture of the whole sequence, weighted by relevance.

Each token is projected into three vectors. The Query represents what a token is looking for, the Key represents what a token offers, and the Value is the information actually carried. Attention scores come from the dot product of a Query with every Key, are scaled and softmaxed into weights, and those weights are used to take a weighted sum of the Values. So Q and K decide where to look, and V decides what gets aggregated.

Attention is permutation invariant, it treats the input as a set, so without extra signal the model cannot tell word order. Positional encoding injects position information into the token embeddings, either through fixed sinusoidal functions or learned position vectors added to the embeddings, or through relative and rotary schemes applied inside attention. This lets the model distinguish 'dog bites man' from 'man bites dog'.

For large key dimension d_k the dot products of Query and Key vectors grow large in magnitude, since variance scales with d_k. Feeding large values into softmax pushes it into saturated regions where gradients vanish and attention becomes nearly one-hot. Dividing by the square root of d_k keeps the score variance roughly constant regardless of dimension, so softmax stays in a well-behaved range and gradients flow.

A residual connection adds a sub-layer's input to its output, so the layer learns a residual rather than a full transformation. In Transformers each attention and feed-forward sub-layer is wrapped in a residual with normalization. This gives gradients a direct path back through the network, mitigating vanishing gradients and enabling very deep stacks, and it lets a layer default to an identity mapping if that is best.

Precision is the fraction of predicted positives that are truly positive, TP over TP plus FP. Recall, or sensitivity, is the fraction of actual positives that were caught, TP over TP plus FN. The ROC curve plots true positive rate (recall) against false positive rate as the threshold varies, so it captures the recall-versus-false-alarm trade-off but not precision directly. On highly imbalanced data a precision-recall curve is often more informative than ROC.

A confidence interval is a range around an estimate that expresses how uncertain the estimate is. A 95 percent confidence interval means that if you repeated the study many times, about 95 percent of the intervals you computed would contain the true value. It is not a 95 percent probability that this particular interval holds the truth, but a statement about the reliability of the procedure. Wider intervals mean more uncertainty.

It depends on the cost of each error in the domain. In disease screening a false negative can mean a missed diagnosis and lack of treatment, so you tolerate more false positives and follow up with confirmatory tests. In spam filtering a false positive can discard a legitimate email, which is costly, so you tolerate more false negatives. You set the decision threshold to minimize the more expensive error type for the use case.

Statistical power is the probability that a hypothesis test correctly rejects the null hypothesis when the alternative is true, that is, it detects an effect that really exists. It equals one minus the Type II error rate. Power rises with larger sample size, larger effect size, lower variance, and a less strict significance level. Studies aim for adequate power, commonly 0.8, so they are likely to find real effects.

Both learn word embeddings from co-occurrence. CBOW predicts the center word from its surrounding context words, averaging context, which trains faster and works better for frequent words. Skip-gram predicts the surrounding context words from the center word, which is slower but represents rare words better and captures more fine-grained relationships. Both are trained efficiently with negative sampling or hierarchical softmax rather than a full vocabulary softmax.