# Public 46th / Private 34th Solution

## Method

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7MeTYlHTqS4cOA7A1%2Fimage.png?alt=media\&token=74066be3-03be-40af-aabf-197e95170d26)

### pre-processing

* Add stat feature
* Add PCA feature
* Variance Threshold
  * Feature selector that removes all low-variance features.
* Rankgauss

  * Assign a spacing between -1 and 1 to the sorted features
  * Apply inverse error function → makes a gaussian distribution

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7MpjussXWFECPCCGe%2Fimage.png?alt=media\&token=f0591b49-c21e-48b7-87de-32ffaeadec36)

### modeling

* Label Smoothing
* Transfer Learning by nonscored for NN, ResNet
* Shallow model
  * Short epoch, learning until limit before loss is NaN by NN
  * n\_steps=1, n\_shared=1 by TabNet
* Thresholdlng NN: input → linear → tanh → NN

### post-processing

* Ensemble. In particular, Tabnet and NN's ensemble is effective.
* 2 Stage Stacking by MLP, 1D-CNN, Weight Optimization

## What doesn't work

* pre-processing
* modeling
* post-processing

## Code Structure

1. Dataset Structure:

* Model Weights
* Inference codes for each models
* Python Packages

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7N-8xomF4rf45aQ_M%2Fimage.png?alt=media\&token=db599f39-5afc-4c0c-a09b-161247b53ac6)

2\. Install python packages → Inference stage1 models → get the predictions of each models

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7N843r7BT7PvyJ7Eq%2Fimage.png?alt=media\&token=581ef76d-d214-485e-95de-4aea5bd93cda)

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7NCiIx75Zc8pWz9W6%2Fimage.png?alt=media\&token=82cc3fcd-6df6-482b-8892-b1df73411625)

3\. Stacking (MLP, 1D CNN, Weight Optimization)

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7NItCcyU-y4F39AHp%2Fimage.png?alt=media\&token=f1f02aed-4282-4442-994b-e992d99b949b)

* **Not enough time**
  * Target Encoding to g-,c- bin's feature
  * XGBoost, CatBoost, CNN model for single model (Stage 1)
  * GCN model for stacking model (Stage 2)
  * Netflix Blending
  * PostPredict by LGBMWe noticed that there are columns that NN can't predict, but LGBM can (e.g. cyclooxygenase\_inhibitor). Therefore, we came up with the idea of repredicting only the columns that are good at LGBM. But not enough time.

## Takeover

* Clean Inference Code
  * Ensemble using various different models
  * Stacking

## \[Update] Private 3rd Rank with Various Stacking

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7NXN0rmpkGmVdjM6l%2Fimage.png?alt=media\&token=b1c51644-0e65-4e1f-8be7-1a4f6c66ce22)

0.01608 → 0.01599

### 2D-CNN Stacking

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7NbG-pWJLdPuzJaZT%2Fimage.png?alt=media\&token=af75cf4d-d7aa-4a71-91a5-166fa9fc9249)

### GCN Stacking

![](https://3092297284-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MOYwbgbzxMoGaqcwNGj%2F-MQ7KZmZz9oMJg_rEpLH%2F-MQ7Nek1Q9W77MZhstsV%2Fimage.png?alt=media\&token=6a5eb273-1343-4cfc-9686-6366537adf47)

* Adjacency Matrix: Matrix of ones / (# of classes)^2
* Node: (1, 5)