Learning Python Ecosystem for Machine Learning

Notes from learning the three core Python ML libraries: NumPy, Pandas, and Scikit-learn. More of a personal reference than a tutorial, written to solidify what I actually understood vs. what I just copied.

NumPy: Numerical Operations Foundation

For me, NumPy was the layer that made the rest of the ML stack make sense. Through multidimensional ndarray objects, it performs vector and matrix operations at C-language level speed. All ML data, from image data (pixel matrices) to text embeddings (vectors), is ultimately represented as NumPy arrays.

When I started, .shape and broadcasting did not click. I only internalized that a (100,) vector and a (100, 1) vector behave differently after the errors bit me.

1import numpy as np
2
3# Simple array operations with NumPy
4arr = np.array([[1, 2, 3], [4, 5, 6]])
5print("Array shape:", arr.shape)
6print("Array dimensions:", arr.ndim)
7
8# Complex array operations
9matrix_mul = np.dot(arr, arr.T)
10print("Matrix multiplication result:\n", matrix_mul)

Pandas: Data Cleaning and Exploration

Pandas operates on top of NumPy, providing DataFrame, a 2D data structure similar to Excel. Like SQL, it's essential for 'cleaning' and 'exploring' data before putting it into ML models through filtering, grouping, joining, etc.

1import pandas as pd
2
3# Basic DataFrame manipulation
4df = pd.read_csv('my_dataset.csv')
5
6# Missing value handling and advanced analysis
7df_cleaned = df.dropna()
8grouped_data = df_cleaned.groupby('category').agg({
9    'value': ['mean', 'median', 'std']
10})
11print("Statistics by category:\n", grouped_data)

Scikit-learn: Machine Learning Models with Consistent Interface

What helped most was the same .fit() / .predict() flow across models, so swapping baselines stayed cheap. That consistency made quick model experiments much easier.

1from sklearn.model_selection import train_test_split
2from sklearn.preprocessing import StandardScaler
3from sklearn.linear_model import LogisticRegression
4from sklearn.metrics import classification_report
5
6# Data splitting and preprocessing
7X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
8
9# Data scaling
10scaler = StandardScaler()
11X_train_scaled = scaler.fit_transform(X_train)
12X_test_scaled = scaler.transform(X_test)
13
14# Logistic regression model training
15model = LogisticRegression(max_iter=1000)
16model.fit(X_train_scaled, y_train)
17
18# Model evaluation
19predictions = model.predict(X_test_scaled)
20print("Classification report:\n", classification_report(y_test, predictions))

Thoughts While Learning

Honestly, NumPy was the hardest of the three to internalize. Reading it in a book vs. hitting errors in practice felt completely different. Scikit-learn felt easier sooner because the .fit() → .predict() interface is unified.

What stuck with me: understanding why libraries are designed this way lasts longer than memorizing API calls.

Future Plans

I want to keep going with ensembles (Random Forest, XGBoost) and enter one Kaggle competition. Theory alone hides where you get stuck; I need a real scoreboard to see the next gap.