> For the complete documentation index, see [llms.txt](https://pseudo-lab.gitbook.io/1st-moa/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://pseudo-lab.gitbook.io/1st-moa/solution-review/public-46th-private-34th-solution.md).

# Public 46th / Private 34th Solution

## Method

![](/files/-MQ7MeTYlHTqS4cOA7A1)

### pre-processing

* Add stat feature
* Add PCA feature
* Variance Threshold
  * Feature selector that removes all low-variance features.
* Rankgauss

  * Assign a spacing between -1 and 1 to the sorted features
  * Apply inverse error function → makes a gaussian distribution

![](/files/-MQ7MpjussXWFECPCCGe)

### modeling

* Label Smoothing
* Transfer Learning by nonscored for NN, ResNet
* Shallow model
  * Short epoch, learning until limit before loss is NaN by NN
  * n\_steps=1, n\_shared=1 by TabNet
* Thresholdlng NN: input → linear → tanh → NN

### post-processing

* Ensemble. In particular, Tabnet and NN's ensemble is effective.
* 2 Stage Stacking by MLP, 1D-CNN, Weight Optimization

## What doesn't work

* pre-processing
* modeling
* post-processing

## Code Structure

1. Dataset Structure:

* Model Weights
* Inference codes for each models
* Python Packages

![](/files/-MQ7N-8xomF4rf45aQ_M)

2\. Install python packages → Inference stage1 models → get the predictions of each models

![](/files/-MQ7N843r7BT7PvyJ7Eq)

![](/files/-MQ7NCiIx75Zc8pWz9W6)

3\. Stacking (MLP, 1D CNN, Weight Optimization)

![](/files/-MQ7NItCcyU-y4F39AHp)

* **Not enough time**
  * Target Encoding to g-,c- bin's feature
  * XGBoost, CatBoost, CNN model for single model (Stage 1)
  * GCN model for stacking model (Stage 2)
  * Netflix Blending
  * PostPredict by LGBMWe noticed that there are columns that NN can't predict, but LGBM can (e.g. cyclooxygenase\_inhibitor). Therefore, we came up with the idea of repredicting only the columns that are good at LGBM. But not enough time.

## Takeover

* Clean Inference Code
  * Ensemble using various different models
  * Stacking

## \[Update] Private 3rd Rank with Various Stacking

![](/files/-MQ7NXN0rmpkGmVdjM6l)

0.01608 → 0.01599

### 2D-CNN Stacking

![](/files/-MQ7NbG-pWJLdPuzJaZT)

### GCN Stacking

![](/files/-MQ7Nek1Q9W77MZhstsV)

* Adjacency Matrix: Matrix of ones / (# of classes)^2
* Node: (1, 5)