Can large language models explore in-context?

Arxiv Papers - Can large language models explore in-context?