Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Randomly Initialized ESM's

See original GitHub issue

Hello friends,

Requesting a very minimal feature addition. Could we get an esm.pretrained.esm_random (or similarly named), maybe with 4 or 5 seeds for completeness? It’s not hard to randomly initialize one’s own ESM models, but it would be a nice quality of life improvement. It would also ensure some consistency in these “random ESM” baselines 😃

Thanks! Nick

Issue Analytics

State:
Created 2 years ago
Comments:5 (2 by maintainers)

Top GitHub Comments

1reaction

nickbhatcommented, Apr 6, 2022

@ulupo feel free to take this issue! Thanks for helping out.

0reactions

ulupocommented, Apr 6, 2022

@tomsercu thanks a lot for the clarification!

Top Results From Across the Web

On generalization capability of randomly initialized vs pre ...

TLDR; Q: “Can training neural networks from randomly initialized weights lead to better generalization than from pre-trained weights?” A: They can.

Issues · facebookresearch/esm - GitHub

Evolutionary Scale Modeling (esm): Pretrained language models for proteins - Issues · facebookresearch/esm.

Conservation laws in a neural network architecture - GMD

The 5000 independent days include randomly initialized values for active species concentrations at the beginning of each day of simulation. Cosine of zenith ......

Random initialization - Machine Learning Glossary

Random initialization refers to the practice of using random numbers to initialize the weights of a machine learning model.

Synopsys True Random Number Generators

TRNGs are used to generate keys, initialization vectors and nonces used in many security standards. The Internet of Things (IoT) is a fast-growing...