https://towardsdatascience.com/hyperparameter-optimization-in-python-part-2-hyperopt-5f661db91324
드디어 내가 손튜닝을 벗어나는 날이 오다니...감동스럽다..
https://en.wikipedia.org/wiki/Kernel_density_estimation
kde라고 아는가?? 여기 분야는 공부할 것이 끝도 없다..끝도 없어...
https://en.wikipedia.org/wiki/Q-Gaussian_distribution
이것도 추가~~~ q의 등장!
The distribution is often favored for its heavy tails in comparison to the Gaussian for 1 < q < 3.
HyperOpt로 머리 아프려고 하는 찰나에 Ray를 발견했다.
https://docs.ray.io/en/latest/tune/index.html
https://arxiv.org/pdf/1905.05583.pdf
-> fine tune BERT
Large Batch Training Of Convolutional Networkshttps://towardsdatascience.com/understanding-fixup-initialization-6bf08d41b427
Fixup initialization --> 이건 또 뭐람?? 이거 다 공부하려면 잘 시간이 없겠는데...(하지만 자야지~)
Proper initialization of weight matrices is extremely important. 아 weight matrix initialization을 말하는 거군.
출처: 위의 글
What is Fixup?
Fixup (fixed-update initialization) is a recent initialization method for ResNets created by Hongyi Zhang, Yann N. Dauphin and Tengyu Ma. In their paper, the authors showed that it’s possible to train a residual network without batch norm layers. What is more, the authors managed to achieve state of the art performance in image classification and machine translation.
만약 resnet에 normalization layers 가 없다면 ???
그러나, 이런 일이 발생 안 된다고 함. (잘 이해가 안 되긴 한데... 밑의 논문 발췌를 보시라)
데이터 증강하기 Slidding Window with stride
https://arxiv.org/pdf/1908.03265
ON THE VARIANCE OF THE ADAPTIVE LEARNING RATE AND BEYOND
Gradient distribution을 아는가???너무 귀엽다..ㅎㅎ
OOF STACKING
'AI' 카테고리의 다른 글
Rethinking Personalized Ranking at Pinterest: An End-to-EndApproach 리뷰 (0) | 2024.01.23 |
---|---|
Of Spiky SVDs and Music Recommendation 리뷰 (0) | 2024.01.16 |
Feature Enigneering 공부 (0) | 2024.01.12 |
GLORY 리뷰 (0) | 2024.01.09 |
9주차 회고 (0) | 2024.01.05 |