LG Aimers Demand Forecasting Hackathon: What I Did (Ensemble Strategy & Experiments)

I joined the LG Aimers hackathon on restaurant menu demand forecasting. With a final private score of 0.48718, I placed 61st out of 1,500 (top 4.2%). I did not advance to the finals, but I want to reflect on how systematic experiments led to a meaningful ensemble strategy.

When the first strategy failed, I pivoted

At first I sketched a heavy design: rich feature engineering plus a tree/deep hybrid. Debugging dragged on, performance stayed below expectations, and I could not see a clear direction.

Pivot: Instead of building one giant model from scratch, I froze a validated baseline (V44), a simple average of three models with different strengths:

• V19: Feature-based expert (LGBM, XGBoost, CatBoost)
• V31: Per-menu time-series pattern expert (PatchTST)
• V34.2: Global time-series pattern expert (N-BEATS)

The goal became obvious: how to combine this strong baseline more intelligently.

Systematic experiments and what I learned

I set one rule: leave each baseline model as-is and change only how predictions are merged, then watch the score.

Failed hypothesis: editing the model internals

The first try was adding new features inside V19.

Result: Score worsened (public 0.5030 → 0.5092).

Why: V19 grew more complex and its predictions grew more similar to the deep models (V31, V34.2). That broke ensemble diversity: the property that each model should be “reasonably good yet different.” I felt firsthand how much that diversity matters.

Winning hypothesis: optimize the outer combination

From that failure I focused purely on the outer combination.

Hypothesis 1 (weights differ): Tree model V19 anchors the ensemble. Raising its weight beat a plain average.

Hypothesis 2 (exploit SMAPE): SMAPE penalizes predicted zeros heavily while actual zeros are cheap. Setting final predictions of 0 to 1 was the move that pulled the score into the 0.4 range.

Final outcome: weighted ensemble + post-processing → about 15% RMSE reduction vs. the baseline.

The clearest lesson: when you already have solid models, tuning how you combine them often beats rewriting their insides.

Final submission code (simplified)

1import pandas as pd
2import numpy as np
3
4# Load each baseline model’s predictions
5pred_v19 = pd.read_csv('submission_v19.csv')['answer']
6pred_v31 = pd.read_csv('submission_v31.csv')['answer']
7pred_v34_2 = pd.read_csv('submission_v34_2.csv')['answer']
8
9# Weighted average: 0.5 on V19, 0.25 on each of the others
10weighted_pred = 0.5 * pred_v19 + 0.25 * pred_v31 + 0.25 * pred_v34_2
11
12# SMAPE defense: map exact zeros to 1
13final_pred = np.where(weighted_pred == 0, 1, weighted_pred)
14
15submission = pd.read_csv('sample_submission.csv')
16submission['answer'] = final_pred
17submission.to_csv('final_submission.csv', index=False)

Retrospective

Final private score: 61st / 1,500 (top 4.2%). Missing the finals stings, but the biggest lesson was that performance only started climbing when I stopped chasing a perfect monolith and focused on mixing proven models well.

Next competition I plan to add architectures like Temporal Fusion Transformer (TFT) into the ensemble to widen diversity further.