[D] For Anyone Who Has Clocked More Than 50+ Days Of DL Model Training Time, Do You Use Anything Other Than Adam or AdamW?

Jump in cancer diagnoses at 65 implies patients wait for Medicare: study
April 8, 2021
LastPass Pricing Changes
April 8, 2021

[D] For Anyone Who Has Clocked More Than 50+ Days Of DL Model Training Time, Do You Use Anything Other Than Adam or AdamW?


For almost All ML projects which had DL, I used AdamW and it just worked. So fucking well. So a few questions to fellow Redditors who might be training models frequently :

Do you use a different optimizer? Why?

Do you tune the Beta values?

Have you consciously ever chosen not to use Adam? Why?

I have seen some recent fancy optimizers like PCGrad but never found the need to use it. When did you use them if you had to?

submitted by /u/thunder_jaxx
[link] [comments]

Source

Comments are closed.